Best way to listen for touch controller hit in WebXR

Questions
1

Hey all, I have something working great on screen. I’m using scene.onPointerObservable to pick a mesh and get the point on the mesh where the ray from the mouse hit. To get the initial X, Y to cast from the screen, I use scene.pointerX/Y. I’m also trying the clientX,Y on the pointerInfo.event coming in on the observable which also seems to work reasonably well.

The problem is that I’m really trying to target the the Oculus Touch controllers. I’m running Chrome Canary and the scene itself works great, but the X,Y values coming in from the pointer observable don’t seem to change (I’m suspecting that it’s just the last known mouse position prior to entering WebXR. The pointermove event is firing just fine however.

What’s the correct way to accomplish hovering over a mesh and getting the point on the mesh? In the pointerInfo itself, there’s a pickedMesh, hit flag, etc, but all seem to be null. I see that there is a ray I might be able to use - is that it? Do I need to manually cast the ray? I question because the pointerInfo object has so much info coming in, I feel like I’m missing something really basic.

thanks!

2

Hello! to get the same kind of interaction you need to use scene.pickWithRay which can use a ray coming from the controllers (like here with the vrExperienceHelper: Babylon.js/vrExperienceHelper.ts at 7dffc724755edddc6a0b980e125b6168eaf7690b · BabylonJS/Babylon.js · GitHub)

To get the ray, just use controller.getForwardRay(length)

3

Thanks! That sounds easy enough. Does the call to get the ray just get thrown in the render loop as needed? As in, having a onPointerObservable is pointless for this use case? I wasn’t sure if the onPointerObservable just normalized the currently dominant hand with touch controller, and you could get the ray from the pointerInfo object.

thanks again!

4

Yep no worries it will be handled flawlessly. You can use it when you want (like once a frame for isntance)

5

Awesome, thanks much!

6

OK, FYI, just got this working. Unfortunately pickedRay is not a property of controller. The ray must be constructed manually AFAIK:

 const ray = new BABYLON.Ray(controller.pointer.absolutePosition, controller.pointer.forward, 1000);

I’m making some assumptions here on the pointer properties, but given that they have the same accuracy as my mouse pointer events in place right now, I think my assumptions are correct.

7

If it works then it’s probably correct :slight_smile:

You can also see what the WebXRControllerPointerSelection class is doing. It is a more advanced example that is tightly coupled to the base experience helper.

8

Great! Thanks! I’ll check it out, I just wanted to post my solution in case anyone else came looking

2 Likes