VRHelper onNewMeshSelected not triggering on Android


I’ve been plaing around with the VR function of babylon and tried to get mesh interaction to work.
The problem I’m having is, that I don’t seem to get notified when gaze is starting to select something. I have tried adding an ExecuteCodeAction to the actionManager of the Mesh with a OnPointerOverTrigger, and also tried using the VRHelper.onNewMeshSelected function.
It works on Chrome on Windows, but on Chrome on Android none of these functions seem to get triggered.
Interestingly a ExecuteCodeAction with OnPickTrigger is triggered after staring at a mesh for a while.
This playground is not from me, but it is close enough to my code and also has the issue of working on windows in vr mode, but not on android in vr mode:

All clues are welcome :smiley:


I would recommend to use the WebXR helper instead, which has a gaze mode that updates the scene meshUnderPointer for you. This page will help you get started - WebXR Experience Helpers | Babylon.js Documentation , and there are a few examples on the playground as well.

The VR experience helper is deprecated and is using code that is no longer maintained.

Awesome Thanks!

So I assume OnPointerOverTrigger and OnPointerOutTrigger not being triggered by the gaze mode is by design and is something I would have to do manually by observing changes in scene.meshUnderPointer?

Never mind, it seems that the Triggers are being triggered correctly, but the window.requestAnimationFrame doesn’t seem to work while in vr mode, that’s why nothing happened. So different problem xP

Since we emulate pointer events for you, all pointer events should trigger correctly. And no, requestAnimationFrame is different in XR. You should either use the observable that triggers on each frame (in the xr session manager) or scene.onBeforeRenderObservable