Speeding up scene.pick() on renderloop

I’m trying to allow the user to navigate through the scene from a FPS camera view (Universal camera), while pointer locked, and any mesh they hover will show a description of the mesh. I have this code all written already, but out of the blue it’s extremely laggy (see: Javascript behaves differently when Chrome console is open/closed for more info on that). At this point the only way to reduce the lag to a playable state is to remove scene.pick entirely, or just only update the picked mesh every x-number of frames. Removing it is basically out of the question, and only updating every x-number of frames is pretty ugly at runtime.

I have looked into and tried using ActionManagers, but they won’t work in this case because the only way to pick a hovered mesh from the scene is by using the pointer coordinates. When the pointer is locked, the user is expecting the center of the screen to be the focal point, not where the pointer was last when they locked to the screen. There is no way to center the pointer, and no way (that I’m aware of) to change the ActionManager’s coordinates to pick from, so I’m stuck with running scene.pick() every render.

I’ve tried picking with a ray that’s limited to the length of the user’s reach, but this doesn’t really help with the lag issue.

Any ideas on the most efficient way to constantly be picking from the scene?

You could define exactly which meshes to pick every frame:

meshes.forEach(mesh=> {
  const pick = ray.intersectsMesh(mesh, false);
  // do stuff with pick
});
1 Like

The scene.pick() method returns the closest mesh, which is still the functionality I’m needing. If I displayed a hover for each mesh that the ray intersects it would display the hover text for every mesh that the ray intersects, as opposed to only displaying for the closest mesh.

In your case, a GPU picking could work because the picking occurs always at the same location: the center of the screen.

So, you could render each mesh with a different flat color to a render target texture and setup a camera viewport / scissor test to only render a very small part of the scene, namely the very center of it (maybe 3x3 pixels). Then you can retrieve the 3x3 texture to the CPU and read the color at the center of this texture, that will give the mesh picked. Maybe a 1x1 texture would work too…

It should be fast because given the very small dimension of the render target, most of the meshes will be culled early.

5 Likes

That would be ideal! I’m not seeing any documentation for GPU picking anywhere, is that something I will have to build from scratch or do you know where would be the best place to find more information on this?

You will need to build it from scratch. From this forum:

1 Like

Awesome, thanks! I’m surprised a built in GPU picking doesn’t already exist :thinking:

Well, you are free to contribute :wink:

1 Like

hey
i test it in 2016

https://www.babylonjs-playground.com/#JAG8B#24

2 Likes

Wow, great work! It’s way over my head at the moment, but I’ll start working on it. Thank you!

1 Like

Hey morterra,

any update on this? I’m currently using scene.pick and for 10-15 elements that have to be picked the whole scene freezes for about 300 to 1500 ms when I’m doing this.
scene.pick (ray.intersectsMesh is also slower than scene.pick) is not ideal at all :frowning:

a second for only 15 elements sounds like a lot :frowning: could you provide a repro in the playground ?

1 Like

@Robin_McLaut scene.pick should work with little to no problem on pretty advance setups and scenes. Id suspect there is something underlying not being addressed. If you could do like @sebavan requested and post a small PG we could help out more.

1 Like

It’s been a long time, so I don’t remember what I ended up doing exactly. But I do remember multiple times having weird rotational camera lags which came from Chome. I never found a solution for that problem, they just appeared and fixed by themselves basically. Must’ve been an issue in Chome.

Even though the issue was probably with Chrome, I did spend hundreds of hours analyzing stress points and optimizing as much as possible, incase I could help mitigate the lag. I remember reducing DOM elements and their listeners, I used bounding boxes when possible, and wrote custom picker functions to limits the meshes I’m testing for collisions.

Your situation sounds a bit different, as you shouldn’t have that kind of lag with only a few objects. If you can replicate the issue in a Playground, it shouldn’t be too difficult to pinpoint the cause.