Multi Touch Events in WebXR

Hi,
Anyone can help me with getting panning events and multi-touch panning events in a mobile WebAR environment? For now, I can only get pointer down, up and move events but not sure how can I check or implement single touch panning and multi-touch panning. I’ve also tried to reimplement this Babylon.js/arcRotateCameraPointersInput.ts at master · BabylonJS/Babylon.js · GitHub and add controls to web XR camera but it seems not trigger onMultiTouch function when I test it. Any thoughts or help is highly appreciated!

@PolygonalSun might be able to help but I am not sure why the arcRotateInput you tried would not work.

Could you provide a repro Playground ?

Hey @sebavan I don’t have a playground example yet will try to get you one later. But just give you something to might be helpful to start with.


This is the pointerInfo I got from desktop

This is the pointerInfo I got from android in WebXR session
Seems like there are some info missing in the webxr pointerInfo(for example if I am multi touching, I guess the buttons in the pointerInfo callback should be something other than 0? )

Hi,
I got a simple XR playground setup here https://playground.babylonjs.com/#9K3MRA#692, checking the pointer event callback I see the button and buttons all return 0. how would I implement multi-touch event in that case?

Thanks a lot for the repro, this will help @RaananW once he ll be back from his break to have a look as I unfortunately do not know XR enough for this part.

Our WebXR pointer event emulation will run different pointer events for both controllers, if this flag is set to true when enabling it:

Babylon.js/WebXRControllerPointerSelection.ts at master · BabylonJS/Babylon.js (github.com)

the pointerId will be different between the two controllers. The button and buttons will return 0, since this is an emulated event, it is not the native event that webxr provides us (since webxr doesn’t work with pointer events).

Hope this helps!

okay, that makes sense, so what do you think is the best approach I should take if I want to implement gestures say panning and two-finger rotate in WebXR.

You will need to measure the distance between the two controllers constantly, just like you implement a zoom gesture.

https://playground.babylonjs.com/#9K3MRA#692 okay still see this demo on PG, when I am on a mobile device and doing some two finger gesture I only see one pointer event with same pointerId

As i wrote in my last comment, you need to enable enablePointerSelectionOnAllControllers in order for that to work:

WebXR Pointer Event Test | Babylon.js Playground (babylonjs.com)

2 Likes

ohhh thanks! will give it a try