How to properly setup WebXR (immersive-ar) Touch Input?

Hey there! I’m learning BabylonJS and I was trying to make something similar to this demo Babylon.js Playground work on my own.

However, I can’t seem to make the touch input to work properly.
It seems like the onPointerDown event is triggered perfectly fine when in the 3d-viewer, but as soon as the immersive-ar scene is started and the camera/fullscreen AR window opens, the touch input doesn’t work anymore.

I’m using the WebXRExperienceHelper, is there any extra setup needed? Any features enabled or anything else? Any suggestions on how to properly set this type of input?

Thanks in advance!

1 Like


Would be great to see the actual scene you are trying to implement. The demo you shared (the demo from the docs) uses touch input to generate an anchor:

scene.onPointerDown = (evt, pickInfo) => {
        if (hitTest && anchors && xr.baseExperience.state === BABYLON.WebXRState.IN_XR) {

If you got it to work, the touch input does work correctly.

Using the experience helper (along with the babylon pointer system, which is enabled per default) should allow you to handle touch events correctly. Please share your playground (the one you are working on that doesn’t work) so we can help you find out what goes wrong there :slight_smile:

1 Like

Thank you so much for replying!

Yes that’s what I can not get to work, it’s only detecting input (triggering the OnPointerDown when on the ‘normal webpage’, but as soon as the immersive-ar ‘layer’/page opens up with the camera view there’s no touch input, not even in the onPointerObservable.

I’m super confused regarding the setup for this.
In this Docs Page (WebXR Experience Helpers | Babylon.js Documentation) it reads that the Default Experience setup automatically sets up some Controller Input:
(from the docs) -
Basic usage of default experience
The default xr experience will:

  • Create a basic experience helper and initialize it
  • Create an HTML UI button to enter XR
  • Init the input source class which will initialize controllers
  • Enable the pointer selection and teleportation features

…but I’m using the Experience Helper, and for that it doesn’t state any sort of Controller Input initialization:
(from the same docs page)-
The basic experience helper will:

  • Initialize the XR scene
  • Create an XR Camera
  • Initialize the features manager

There is also this page, regarding Pointer Inputs that states the need to install and import PEPjs:

As well as this other page, where it states that we need to access some ‘xr.input’ object, which I guess belongs to the object (xrHelper) that comes back from the WebXRExperienceHelper.CreateAsync() Premise:

So, with all of these and some more reading here on the forums I’ve got super confused and have no Idea of what I need to setup/how to implement the OnPointerDown (touch input) within a session running with the Experience Helper.

Unfortunately, I’m not writing this in a Playground - which also makes me wonder if it somehow uses/imports that PEPjs library by default and has some extra setup stuff - since the suggested playground demo works fine. I’m implementing this in a VueJS Application and everything else has been working fine (the Hit-Test, Plane Detection, Anchors Setup, etc).

A few quick notes -

  • No need for PEP at all for device input in XR.
  • Pointer events should work out of the box when using the default webxr experience helper.

If you don’t use the default experience helpr you will need to initialize the pointer selection feature using the features manager. To do that you need to add this line to your code:

featuresManager.enableFeature(WebXRFeatureName.POINTER_SELECTION, "stable" /* or latest */, {
  xrInput: xr.input,
  // other options here

to get the xrInput you will need to initialize the class (WebXRInput) using the session and the camera created by the experience helper.

If pointer down events are not triggered when in AR there is a bug on our side. But as the playground you initially pasted is working (and is using pointer events), i will only be able to help if you can reproduce it somewhere. The playground will be the best place to start, but even a working project or a reproduction online (please with unminified code if possible) will suffice.

1 Like

Hah! I’m not using the Default Experience, but I’m using the Basic Experience Helper (WebXR Experience Helpers | Babylon.js Documentation).
So, for that, I should enable the Pointer Selection feature in the FM.
Could you please elaborate a bit more on the xrInput setup? Or share a Docs link or demo example?

I will try to implement with those and give you some feedback here.
If it doesn’t work I’ll try to ‘pastebin’ the code somewhere to share it.

Thanks again!

Of course! It is all in the docs page:

WebXRInput | Babylon.js Documentation

1 Like

Awesome! Thanks for the link to the api docs!
One more thing, what’s the preferable way to get the xrCamera from the Basic Experience Helper?

The camera is a property of the xr experiencer helper: WebXRExperienceHelper | Babylon.js Documentation
So you can get it right after constructing it

Thank you!
I will give this a try.
I’ll let you know how it goes ASAP!

1 Like

Got it working! Thanks a lot for your help!
I had some mistakes in my setup, but I’ll refactor the code and if I find something specific that I was missing out beyond the general setup you suggested here I’ll update here :slightly_smiling_face:.
Thank you!!!

1 Like

Awesome! always glad to read that :slight_smile:

1 Like

Hey There! So revisiting the project, I’ve noticed there is this warning:

babylon.js?633b:16 BJS - [16:58:50]: Could not find a matching motion controller for the registered input source
e._WarnEnabled @ babylon.js?633b:16
e.Warn @ babylon.js?633b:16
eval @ babylon.js?633b:16
Promise.then (async)
e @ babylon.js?633b:16
e._addAndRemoveControllers @ babylon.js?633b:16
_onInputSourcesChange @ babylon.js?633b:16

Any idea what might be causing this?

I was sure i solved this already. You can ignore this, this is because there is no controller for, well, your finger :slight_smile:

I will make sure this is not displayed on mobile. Just for me to be sure - what version of babylon are you using?

I’m asking because I also noticed that the ‘onPointerObservable’, triggers every Event Type everytime there’s an interaction with the touch input (POINTERDOWN, POINTERUP, POINTERMOVE, POINTERPICK, POINTERTAP). But IDK if this is related to the warning

I’m using version 4.2.0.
from package.json:

“babylonjs”: “^4.2.0”,

It will trigger on every pointer down event, because a new input source is created every time you are touching the screen. You can ignore it, nothing will fail. In any case it shouldn’t happen in mobile.
Will take of that in a future version of the framework.

1 Like

Hah I see! Thank you so much for clarifying!
Thanks a lot for all the help and suggestions! :smiley:

1 Like