WebXR Light Estimation and Realtime Reflections

I’m using 8th Wall with BabylonJS to create a somewhat realistic demo that can use light estimation and realtime reflections based off the real environment the camera is displaying. They use this: Realtime Reflections | 8th Wall | 8th Wall, which is great but it’s used with A-Frame. I went into the BJS docs to see if there was something similar and found this: WebXR Augmented Reality Features | Babylon.js Documentation

I had trouble getting this example to run on my iPhone: https://playground.babylonjs.com/#NAZYHG#5

It just shows up as it does in a desktop browser. I copied this code block:

    scene.createDefaultXRExperienceAsync({
        uiOptions: {
            sessionMode: 'immersive-ar',
            referenceSpaceType: 'local-floor'
        }
    }).then((defaultXRExperience) => {
        const le = defaultXRExperience.baseExperience.featuresManager.enableFeature(BABYLON.WebXRFeatureName.LIGHT_ESTIMATION, 'latest', {
            setSceneEnvironmentTexture: true,
            // cubeMapPollInterval: 1000,
            createDirectionalLightSource: true,
            // reflectionFormat: 'srgba8',
            // disableCubeMapReflection: true
        });
        const shadowGenerator = new BABYLON.ShadowGenerator(512, le.directionalLight)
        shadowGenerator.useBlurExponentialShadowMap = true;
        shadowGenerator.blurScale = 2;
        shadowGenerator.setDarkness(0.1);
        shadowGenerator.getShadowMap().renderList.push(sphere, sphere2, sphere3);
    });

and I got this in the console:
BJS - [14:33:23]: light-estimation is an experimental and unstable feature.

Even so, the scene loads up but nothing different appears. I also get this error:

Unhandled promise rejection: TypeError: undefined is not an object (evaluating 'defaultXRExperience.baseExperience.featuresManager'

Does BabylonJS not support these features yet? Is there another way to set this up?

As a follow up to this. It looks like what 8th Wall is doing here: Realtime Reflections | 8th Wall | 8th Wall
Is essentially creating a render texture in the form of a cube map and running it every frame. I’ve used render textures before in Unity to mimic viewing a secondary camera output (like a security camera displaying on a TV) but never with something like an env texture/HDRI or with an AR camera within BabylonJS. Has anyone ever done something like this?

IOS does not support webXR which explains why it can not work on iPhone. I hope this will soon-ish be working as they added a flag recently :slight_smile:

On the other hands 8th wall is relying on its own tracking and such behavior for this very reason.

Light estimation are only available on latest android chromium under a flag I believe at the moment.

Ok, that’s good to know. I was going to try to replicate the scripts they’re using for light estimation and real-time reflections to be used with BabylonJS.