WebXRLightEstimation not working properly if it was created on BABYLON.WebXRState.IN_XR

Questions
1

Hello everyone,

I’m encountering a somewhat complex issue with light estimation in Babylon.js WebXR and would appreciate any guidance or insights. Here’s a detailed description of my problem:

I attempted to enable light estimation using the following code:

scene.createDefaultXRExperienceAsync({
    uiOptions: xrOptions,
    optionalFeatures: true // Required for hit-test
}).then(async (xr) => {
    lightEstimation = await xr.baseExperience.featuresManager.enableFeature(BABYLON.WebXRFeatureName.LIGHT_ESTIMATION, "latest", {
        setSceneEnvironmentTexture: true,
        createDirectionalLightSource: true,
        reflectionFormat: 'srgba8'
    }, true, false);
});

However, this approach consistently returns a “valid” lightEstimation variable, making it unclear whether the feature will function correctly or fail on the current device.

Alternative Approach with BABYLON.WebXRState.IN_XR:
When implementing light estimation within the BABYLON.WebXRState.IN_XR state, I encounter a peculiar behavior:

  1. On the first attempt to enter AR mode, lightEstimation.directionalLight appears to have zero power and no orientation.
  2. If I exit AR mode and then re-enter, the light estimation works perfectly.

This leads me to believe that initializing light estimation during the IN_XR step might not properly set up certain aspects, which are correctly initialized on a second attempt.

While I understand that this issue might be a messy, I hope I’ve explained it clearly. I’m looking for any advice on how to ensure reliable initialization of light estimation in WebXR.

Thank you in advance!

2

You have to enable it before, as the xr session need to start with the feature flag. Some features behave like that. But you are right, there should be a warning. I’ll take care of that

3

The problem is that the onError of xrOptions triggers this error:
this._xrSessionManager.session.requestLightProbe is not a function

if WebXRLightEstimation is not available on that device (like a computer)

4

that is because you don’t enable it before the session starts. then the function will not be available, or will fail because it wasn’t enabled.

Enable it before the session starts on a supporting device, and it will work as expected

5

No no, on a supported device it works ok the problem is on unsupported devices.

My intention is:

  • If it is supported enable it, if not disable it.
  • On WebXRState.IN_XR if it was not enabled create a regular direct light.
6

Sorry, here is a more detailed answer :slight_smile:

Your alternative approach mentioned in the first message will not work (see my last messages :slight_smile: ). It does work as you mention, as the first session doesn’t have it enabled, and the second has, since the feature was initialized before creating it (the 2nd session).
Your code (the one included in the first message) should initialize light estimation to be included in the next session. If you set light estimation to be required, the session will not start if the device doesn’t support it. this is the only fail-safe way to be sure that the feature is enabled. there is no way for us to know if light estimation is supported until that point. We can check if the objects needed are available, but that’s about it.

The feature should not attach if the session doesn’t support it (it has an attached variable). If marked as optional, the session will start, but without the feature attached. Looking at the code it seems like it will attach nonetheless, which is something I need to solve. After solving this you will be able to check if the feature is attached (i.e. - session supports it and it is enabled) or not (system doesn’t support or created after the session has started). Then you will be able to run your own code doing what you want to do with the experience.

7

Yes, my concern was that it should be attached in a mandatory way even if I mark it as optional, that’s why the error is triggered on my side.

1 Like
8

This PR is dedicated to you :slight_smile:

Better feature handling by RaananW · Pull Request #14732 · BabylonJS/Babylon.js (github.com)

1 Like
9

I owe you a beer (actually more than one!)

1 Like
10

Sounds good to me! :beers:

11

I can confirm that it works flawlessly now

1 Like