Fast way to pick among thinInstance positions from screen point?

Questions
1

I have a single mesh with up to about 25000 thinInstances that are just spheres. I have access to both the thinInstance matrices and the positions.

What is the best way to find the closest (depth from screen) thinInstance/position to a screen coordinate?

I thought about using GetFinalMatrix from the viewport and camera view then TransformCoordinates on each position with that single matrix. Finally, iterate and test each result within a square or radius (in screen coordinates). If I iterate in depth order, then I think I end up with a depth-ordered list of picked positions (or stop at the first). If this works, I like that I can choose the size of the (virtual) “picking ray” (via the square size or radius).

Is there a faster/better way?

I don’t really have a need to test all triangles of each thinInstance sphere.

2

With so many spheres, chances are that you will often have a sphere at the screen location you pick. In that case, you could use GPU picking to find out what this sphere is. If no sphere exists at this location, you could try multi-picking, or just do the calculation yourself as you described.

3

Works pretty well, but selects all thinInstances whether they are behind another object or not, and regardless of depth.

It’s a good start because the picked objects are sometimes tiny.

        const finalMatrix = BABYLON.Matrix.GetFinalMatrix(
            camera.viewport.toGlobal(
                engine.getRenderWidth(),
                engine.getRenderHeight(),
            ),
            BABYLON.Matrix.Identity(),
            camera.getViewMatrix(),
            camera.getProjectionMatrix(),
            camera.minZ,
            camera.maxZ,
        )

Then within the loop:

 BABYLON.Vector3.TransformCoordinatesToRef(vis.position,finalMatrix,screenPos);

Note that I compare by ignoring screenPos.z, and getting the distance to the screen position of the object. The following “return” is within an array.filter().

return 20>clickPoint.subtractFromFloats(screenPos.x,screenPos.y,0).length()

“20” is a somewhat arbitrary number of pixels. Ideally, I should derive it from the screen dpi and the resolution of the canvas.

Edit: Forgot to add: use evt.offsetX and Y from observable event:

const clickPoint = new BABYLON.Vector3(evt.offsetX,evt.offsetY,0)

Here it is in action:

(click on an orbit regime, then tap near or on some satellites. Labels will appear on “picked” satellites. Use Settings > Reset to clear labels. The regime “LEO” is most stressful to the picking and draw engine because of the 10000+ satellites really close together, and labels are a burden to draw.)

4

It works pretty well!

I’m not sure how it can happen, as GPU picking works by rendering the scene with special materials, so it should not be possible to pick non visible objects, and you should always get the closest object… If you can setup a repro, we will be able to look for the problem.

5

I wasn’t very clear. I implemented my own “picking” by converting each thinInstance world position into screen position then calculating distance to click position. My calculation includes all thinInstances whether they are obscured by other objects or not. I haven’t tried GPUPicking.

1 Like