WebXR Optimization

Hi, I’m trying to learn how to optimize my WebXR experience with BabylonJS. I noticed the guys at SketchFab seems to be able to output very high FPS with seemingly lower rendering resolution. I tried to engine.setHardwareScaling in Babylonjs which seems to work fine when not in VR mode. but Once in VR mode everything seems to be rendered in high fidelity and is very laggy. My scene can run very fine on my android mobile.
Not sure If i’m doing everything right.

Do anyone have examples of optimization applied to WebXR with babylonjs?
Much appreciated! :slight_smile:

cc @RaananW (please be patient as Raanan is on vacation)

I also noticed that hardware scaling is ignored after entering xr, at least on Oculus Quest.

And depending on the project, I would prefer to tweak scaling instead of disabling some things.

Regardless, I always do:

  • use low poly as much as possible;
  • use as little materials as possible;
  • lower texture resolution as much as possible;
  • merge any static meshes that share same material;
  • bake as much of the lighting as possible;
  • use shadows only where absolutely necessary;
  • not use hdr and postprocess pipeline;
  • disable scene antialias

Edit: just exposed the property on the engine creation, and will create a PR soon. It is working nicely here, hope you guys will accept that. Thanks.

It is like this:

this._engine = new BABYLON.Engine(this._canvas, USE_ANTIALIAS, {framebufferScaleFactor: 0.5});

----------8<-------------

old text:

Ok, so I did a preliminar and ugly hack in the engine, and got scaled down rendering on the Quest. Yes, sure, less resolution, but man that becomes fast… I loved to see the little fractal from the template in 90fps… =)

Depending on project I honestly prefer lower resolution and more effects than higher resolution but disabling effects. That is just me, though!

Ok, but this was a quick hack, I’m now seeing how this could work nicely and then who knows, create a PR.

Edit: if devs are reading: could the framebufferScaleFactor from WebXRManagedOutputCanvasOptions be exposed to the user? Even if just once on initialization?

I believe that some users (including myself) will check their experiences, and if they can’t run at full resolution, will want to reduce rendering res from the start. If that property is exposed, the problem is solved. I just edited that property default value, and it worked nicely on Quest 2. Loved the performance gain… =)

3 Likes

Thee biggest thing to do is use multiview of XRLayers. My code in is:

const layers = featuresManager.enableFeature(BABYLON.WebXRFeatureName.LAYERS, "latest", {
    preferMultiviewOnInit: true
}, true, false);

This has no documentation, & there is no “stable” version available. Not sure it is working, but too busy to check right now. It is the way to code it though, so might as well put it in.

2 Likes

I added the above and got a black screen.

Not sure if I used it in the correct place, though – on my little template:

    // create a default xr experience
    createDefaultXr() : void {
        this._scene.createDefaultXRExperienceAsync(
            {
                floorMeshes: this._grounds
            }).then((xrHelper) => {
                this._xrhelper = xrHelper;

                const layers = this._xrhelper.baseExperience.featuresManager.enableFeature(BABYLON.WebXRFeatureName.LAYERS, "latest", {
                    preferMultiviewOnInit: true
                }, true, false);

                this._xrhelper.baseExperience.onStateChangedObservable.add((state) => {
                    if (state === BABYLON.WebXRState.IN_XR) {
                        // not working yet, but we hope it will in the near future
                        this._engine.setHardwareScalingLevel(HW_SCALE_VR);

                        // force xr camera to specific position and rotation
                        // when we just entered xr mode
                        this._scene.activeCamera.position.set(0, 0.1, 2);
                        (this._scene.activeCamera as BABYLON.WebXRCamera).setTarget(new BABYLON.Vector3(0, 0.1, 0));
                    }
                });
            }, (error) => {
                console.log("ERROR - No XR support.");
            });
    }

Results in black screen (Quest 2). Without those lines, normal XR.

As a side note, that exposed name “framebufferScaleFactor” might be misleading to the user, as now we would have “setHardwareScalingLevel” which works on non-XR rendering, and “framebufferScaleFactor” which only works on XR rendering.

Maybe that could be called “xrFramebufferScaleFactor” or something like that to avoid confusion?

Yes, I can confirm this. I only test on device once a week if possible, and had not tried after changing from stable to latest yet.

There is obviously a reason that there is no stable version, but this is way better than just an optimization. Optimization implies doing trade offs in favor of your circumstances, not just pure gain. This reduces much of the cpu required to be run twice by the multiple angles / displays.

Ranaan is rumored to be hiding out at a remote location for his vacation, but probably just coding it with the version “stable” & running CDR version of BJS will probably mean you would not have to do anything once it is up.

Sure, totally agree that multiview is best, but even with that, if pushing limits, maybe tweaking framebuffer could be useful as well?

Even at 0.5 resolution – imho, of course – it still looks pretty good and is so faster… could allow a lot of extra hardware abuse. =)

This is amazing, thanks alot for this info. Let me try this out.

I have tried multiview but cant see any benefits so far…

Just out of curiosity, this build is using a tweaked half-resolution framebuffer (0.5):

https://imerso.github.io/babylon.js/template

I used both setHardwareScalingLevel() for non-XR and framebufferScaleFactor for XR to test it on Quest 2. All effects and antialiasing intentionally disabled to make the lower resolution more visible.

1 Like

In the end, RaananW shown that this feature was already available through createDefaultXRExperience.

        scene.createDefaultXRExperienceAsync(
            {
                ...
                outputCanvasOptions: { canvasOptions: { framebufferScaleFactor: yourXrScaleHere } }
                ...
            }).then((xrHelper) => {
              ...
            });

So, the feature was already available and there was no need to change anything!

1 Like

Sorry, was just about to answer here :slight_smile:

Yes, you are in full control of your initialization variables. If you are using oculus you can also use other features like updating the target frame rate - Babylon.js/webXRSessionManager.ts at master · BabylonJS/Babylon.js (github.com)

1 Like