Speeding up scene.pick() on renderloop

I’m trying to allow the user to navigate through the scene from a FPS camera view (Universal camera), while pointer locked, and any mesh they hover will show a description of the mesh. I have this code all written already, but out of the blue it’s extremely laggy (see: Javascript behaves differently when Chrome console is open/closed for more info on that). At this point the only way to reduce the lag to a playable state is to remove scene.pick entirely, or just only update the picked mesh every x-number of frames. Removing it is basically out of the question, and only updating every x-number of frames is pretty ugly at runtime.

I have looked into and tried using ActionManagers, but they won’t work in this case because the only way to pick a hovered mesh from the scene is by using the pointer coordinates. When the pointer is locked, the user is expecting the center of the screen to be the focal point, not where the pointer was last when they locked to the screen. There is no way to center the pointer, and no way (that I’m aware of) to change the ActionManager’s coordinates to pick from, so I’m stuck with running scene.pick() every render.

I’ve tried picking with a ray that’s limited to the length of the user’s reach, but this doesn’t really help with the lag issue.

Any ideas on the most efficient way to constantly be picking from the scene?

You could define exactly which meshes to pick every frame:

meshes.forEach(mesh=> {
  const pick = ray.intersectsMesh(mesh, false);
  // do stuff with pick
});
1 Like

The scene.pick() method returns the closest mesh, which is still the functionality I’m needing. If I displayed a hover for each mesh that the ray intersects it would display the hover text for every mesh that the ray intersects, as opposed to only displaying for the closest mesh.

In your case, a GPU picking could work because the picking occurs always at the same location: the center of the screen.

So, you could render each mesh with a different flat color to a render target texture and setup a camera viewport / scissor test to only render a very small part of the scene, namely the very center of it (maybe 3x3 pixels). Then you can retrieve the 3x3 texture to the CPU and read the color at the center of this texture, that will give the mesh picked. Maybe a 1x1 texture would work too…

It should be fast because given the very small dimension of the render target, most of the meshes will be culled early.

5 Likes

That would be ideal! I’m not seeing any documentation for GPU picking anywhere, is that something I will have to build from scratch or do you know where would be the best place to find more information on this?

You will need to build it from scratch. From this forum:

1 Like

Awesome, thanks! I’m surprised a built in GPU picking doesn’t already exist :thinking:

Well, you are free to contribute :wink:

1 Like

hey
i test it in 2016

https://www.babylonjs-playground.com/#JAG8B#24

2 Likes

Wow, great work! It’s way over my head at the moment, but I’ll start working on it. Thank you!

1 Like