Most 3D Controls aren't pickable by WebXR ControllerPointerSelection

Bugs
1

Test scene here: WebXR basic example with teleportation | Babylon.js Playground (babylonjs-playground.com)

After entering VR, you can see that the ray cast by the controller passes right through the holographic button. This is because the button’s meshes are created on the scene’s utility layer, which WebXRControllerPointerSelection doesn’t check during its ray-casting phase. This in turn means that instead of the ray stopping on the surface and rendering a cursor, it passes right through without rendering a cursor.

Note: You can still interact with the button using the ray + motion controller trigger button, you just need to guess as to if the ray is hitting the button.

I feel either controllers shouldn’t interact with 3D controls (though this doesn’t make much sense to me), or the utility layer meshes should be included with the main scene’s meshes for the pick (this seems to already be the case for 2D interactions?).

Note 2: The meshes created in the scene when preparing a MeshButton3D are the only 3D control that WebXRControllerPointerSelection can pick, as the mesh is created on the main scene. This creates a visual inconsistency between the behaviour of different 3D controls.

1 Like
2

Adding @RaananW

3

That’s a good catch.

If I understand correctly you can interact with the controls, but there is no visual feedback related to the ray distance or the selection disc (like a “hover” effect). The ray goes right through them.

This is probably the right solution, but that would require changes to the picking mechanism. We could also pick twice (once for the scene, once for the utility layer), but that would probably have performance consequences.

Pointer events are passed to the utility layer (and this is why you can still interact with the buttons), but pointer pick isn’t. I honestly don’t see a way around picking inside the utility layer as well