Handle multi touch in AR

Hello, newbie here!

Apologies if my question is a bit basic.

I’m trying to add AR support to my existing Babylon.js setup. So far, I’ve managed to successfully load a sample car model.

Here’s the code I’m working with:

public async testARSupport() {
    // Hide other meshes since I only want to show the car
    let meshToHide = this.scene.getMeshById("Sphere_low");
    if (meshToHide) {
      meshToHide.isVisible = false;
    }

    // Enable WebXR AR support with the 'hit-test' feature
    const xr = await this.scene.createDefaultXRExperienceAsync({
      uiOptions: {
        sessionMode: "immersive-ar",
      },
      optionalFeatures: true,
    });

    const xrFeaturesManager = xr.baseExperience.featuresManager;

    xrFeaturesManager.enableFeature(WebXRBackgroundRemover.Name);

    const domOverlayFeature = xrFeaturesManager.enableFeature(
      WebXRFeatureName.DOM_OVERLAY,
      1,
      {
        element: "#rootUI",
      },
      undefined,
      false
    ) as WebXRDomOverlay;

    xr.baseExperience.onStateChangedObservable.add((eventData: WebXRState) => {
      console.log(domOverlayFeature.domOverlayType);
    });

    // Enable pointer selection (handles multi-touch and ray pointer interaction)
    xrFeaturesManager.enableFeature(
      WebXRFeatureName.POINTER_SELECTION,
      "latest",
      {
        xrInput: xr.input,
        enablePointerSelectionOnAllControllers: true,
      }
    );
}

I’ve also added multi-touch support so the user can interact with the car using pan, rotate, and pinch gestures. For that, I’m using scene.onPointerObservable like this:

this.scene.onPointerObservable.add((pointerInfo, eventState) => {
      console.log(pointerInfo);
      if (pointerInfo.type === PointerEventTypes.POINTERMOVE) {
        console.log('AR: POINTER X:', pointerInfo.event.clientX);
        console.log('AR: POINTER Y:', pointerInfo.event.clientY);
      }
    });

However, it’s only detecting one touch at a time. Am I missing something here?
I’d like to be able to capture at least two touches to detect gestures like pan, rotate, or pinch.

Any help would be greatly appreciated!

Pinging XR expert @RaananW – thank you in advance!

Hello and welcome to the forum! Totally right to ping @RaananW here.
In the meantime… it looks like you’ve seen these already, but just in case, here are some related threads on multi-touch in AR:

2 Likes

Hey @alexchuber, thank you for the response!
I’ll have a look into these and get back to you.

1 Like

Would you be able to share a playground? something to test with? to see what exactly you are trying to do?

Thanks :slight_smile:

Thank you so much @RaananW for the reply,
and sorry for the late response. I’ve been trying out the solutions suggested by @alexchuber, and they’re working great!

Thanks again for all the help!

1 Like