Issue with Image Tracking in WebXR

Questions
1

Hello ,

I’m encountering an issue with the WebXR Image Tracking feature in Babylon.js where the onTrackedImageUpdatedObservable event does not seem to trigger as expected. Here’s the context and the problem I’m facing:

Setup:

  • Babylon.js Version: 7.12.0
  • Browser: 17.5.1
  • Device: IPhone 13 mini

Description:

I’m trying to use WebXR’s image tracking capabilities to track a custom image marker in a WebXR experience. I followed the Babylon.js documentation and used the sample code provided, with modifications to fit my use case.

Here’s a simplified version of my code:

javascript

Copy code

const xrHelper = await scene.createDefaultXRExperienceAsync();
const featuresManager = xrHelper.baseExperience.featuresManager;

const imageTracking = featuresManager.enableFeature(BABYLON.WebXRFeatureName.IMAGE_TRACKING, "latest", {
    images: [
        {
            src: "path/to/my-image.png",
            estimatedRealWorldWidth: 0.2 // Adjusted for my marker size
        }
    ],
    disableOnExit: false
});

imageTracking.onTrackedImageUpdatedObservable.add((image) => {
    console.log("Image tracking updated:", image);
    // Additional handling for when the image is updated
});

imageTracking.onTrackedImageAddedObservable.add((image) => {
    console.log("Image tracking added:", image);
});

imageTracking.onTrackedImageRemovedObservable.add((image) => {
    console.log("Image tracking removed:", image);
});

Issue:

The onTrackedImageAddedObservable and onTrackedImageRemovedObservable events are firing correctly when the image is detected and lost, respectively. However, the onTrackedImageUpdatedObservable event never seems to trigger, even though I expect it to provide updates on the position and status of the tracked image.

Steps Taken:

  • Checked the Image: Verified that the image has clear, distinctive features suitable for tracking.
  • Updated Babylon.js: Ensured I’m using the latest stable version of Babylon.js.
  • Tested on Different Devices: Tried it on various devices to rule out device-specific issues.
  • Console Logs: Added console logs to ensure the observable is set up, but still no output.

Questions:

  1. Is there any specific requirement for the image or the environment that might affect the onTrackedImageUpdatedObservable event triggering?
  2. Could there be any recent changes or bugs in the WebXR Image Tracking implementation in Babylon.js that might be causing this issue?
  3. Are there any additional configurations or debugging steps I should consider to get the onTrackedImageUpdatedObservable to trigger?

Any guidance or suggestions on how to resolve this issue would be greatly appreciated!

Thanks in advance for your help.

2

I am abit confused to be honest :slight_smile:

iOS does not support WebXR (at least not on the iPhone), unless apple has updated something and I am not updated. I know it is available in the vision pro, so everything is possible?

Can you enter a WebXR session? Or are you using babylon native?

Also, if I may ask - the onTrackedImageAddedObservable does not exist on the image tracking feature. can you share the code in which is it working? Can you reproduce that in the playground?

3

Apologies for the confusion! I forgot to mentioned about variant launch app to launch my experience in Ios, I am bit new to all let me know if you have any suggestions to use webxr image tracking with variant launch!

4

oh, that might be an issue on their side as well, so it is quite hard for me to debug. Would you be able to share a variant link so I will at least be able to see the behavior?