Registering views and using conditional rendering

Liam_Parker · July 1, 2020, 4:00am

Hey,

I’m currently using engine.registerView to attach my canvas for rendering. I used to have code which conditionally rendered the scene when I knew it would be changing otherwise I just didn’t clear and retained the last frame buffer that was rendered…

After switching the registerView this behaviour seems to no longer work. If I don’t render the canvas always goes flat white as if the buffer was cleared, even with autoClear disabled and preserveFrameBuffer enabled.

Is this behaviour a technicality of how registerView works? Or is it something I can control with a little more tweaking?

Thanks,
Liam

sebavan · July 1, 2020, 1:13pm

I am thinking that the easiest would be to unregister the unused view and register it again when needed ? Would that work for you ?

Liam_Parker · July 7, 2020, 8:06am

Hey sebavan,

Thanks for taking the time to reply. I think we’re not on the same page. I’ve included a short video of the behavior.

Screen Recording 2020-07-07 at 5.52.24 pm

if(sceneHasChanged) {
    scene.render();
}

I essentially use the above logic To only render the scene if the user is interacting. If the user isn’t interacting I would expect whatever was last rendered to just persist. Despite this, it seems both autoClear = false and preserveDrawingBuffer = true don’t solve the issue. Does this make my problem clearer?

If not I tried to get a code sandbox running for this particular problem but I was running into seperate issues. I will try again if need be.

Regards,
Liam

Liam_Parker · July 7, 2020, 10:58pm

I did some further digging with Spector.js, my new favorite tool. It appears that EngineViews are treated as render targets and implement their own clear behavior that must be ignoring autoClear configuration? The other issue I’ve found is that these EngineViews don’t appear to be respecting device pixel ratio. The frame buffer for this render target appears to be ignoring device pixel ratio despite scaling being set to 1 / devicePixelRatio. So on mobile I get a very blurry canvas.

sebavan · July 8, 2020, 1:06pm

Yup multiviews have their own lifecycle and are always rendererd.

Would you want to contribute some changes to allow a bit more customization ?

Deltakosh · July 8, 2020, 2:15pm

They should stick with the autoclear behavior as we are just calling the scene rendering system:

github.com

BabylonJS/Babylon.js/blob/master/src/Engines/Extensions/engine.views.ts#L150


        parent.width = canvas.clientWidth;

        parent.height = canvas.clientHeight;

        this.resize();

    }



    if (!parent.width || !parent.height) {

        return false;

    }



    // Render the frame

    this._renderFrame();



    // Copy to target

    context.drawImage(parent, 0, 0);



    // Restore

    if (previewCamera && scene) {

        scene.activeCamera = previewCamera;

    }

}

each view only swap the active camera (if required) and do a copy of the working canvas to the target canvas.

the device pixel ratio is also conserved when we call resize:

github.com

BabylonJS/Babylon.js/blob/master/src/Engines/Extensions/engine.views.ts#L142


    previewCamera = scene.activeCamera;

    scene.activeCamera = camera;

}



// Set sizes

if (canvas.clientWidth && canvas.clientHeight) {

    canvas.width = canvas.clientWidth;

    canvas.height = canvas.clientHeight;

    parent.width = canvas.clientWidth;

    parent.height = canvas.clientHeight;

    this.resize();

}



if (!parent.width || !parent.height) {

    return false;

}



// Render the frame

this._renderFrame();



// Copy to target

Can you repro the issues you are seeing?

Liam_Parker · July 9, 2020, 1:42am

Hey Delta,

You’re right with regards to the auto clearing. I re-added autoClear = false and preserveFrameBuffer = true and spector.js is reporting no clearing taking place. It looks like the frame buffer is actually persisting correctly. However the canvas on the page appears blank when we don’t scene.render(). It’s as if the frame buffer isn’t being copied to the canvas on the page if scene.render() is not called. I’m not 100% sure how that works at the base level though so purely speculation.

I’ve included a screenshot of the issue in safari, on the right you’ll see frame buffer canvas 4 is correctly rendered not blank. But whenever we skip scene.render() in the loop function the canvas displayed to the user goes blank. This wasn’t the case when I was using the standard flow for rendering, passing the canvas directly to engine. Should I manually be copying the frame buffer to my view canvas when not calling scene.render()?

I’d certainly be happy to contribute to this stuff moving forward, it’s quite important to us right now and this is a library we use extensively. In the meantime I’m still trying to get a code demo running for some reason I always get the following error

Failed to execute 'drawImage' on 'CanvasRenderingContext2D': The image argument is a canvas element with a width or height of 0.

Even after checking both the view canvas and working canvas are non zero dimensions before calling render()… I’ll post here the second I get it running…

As for the low resolution, I’ll wait to get a code demo running. But it appears the frame buffer is rendering at 360 x 306, which is the size of the view canvas exactly. But on my S8 I would expect the frame buffer to be 360 * 3 x 306 * 3 as the S8 has a device pixel ratio of 3, is it correct for me to think this? Again I’ll try and demonstrate this in a sandbox.

Liam_Parker · July 9, 2020, 5:08am

Hey,

I’ve discovered that the following code is executing every loop of the render function and is copying a blank image to the target view canvas if scene.render() was not called within the user render function.

github.com

BabylonJS/Babylon.js/blob/master/src/Engines/Extensions/engine.views.ts#L153


    }



    if (!parent.width || !parent.height) {

        return false;

    }



    // Render the frame

    this._renderFrame();



    // Copy to target

    context.drawImage(parent, 0, 0);



    // Restore

    if (previewCamera && scene) {

        scene.activeCamera = previewCamera;

    }

}



this.activeView = null;



return true;

Here’s a repro of the issue as minimal as I could make it. There are two canvses, the left canvas is the view and the right canvas is the working canvas. You can interact with the working canvas here to see the movement effect the view on the left.

I’d be happy to make a contribution here if you like. I don’t understand why parent is blank if scene.render() was not called though. If parent is the frame buffer it’s correctly drawn even while the view canvas is blank… There must be something I’m missing here.

Regards

Deltakosh · July 9, 2020, 4:42pm

Ha this is really interesting! yeah if scene.render() is not called we still copy the image to the canvas

to your question about blank being empty, I wonder if this is linked to that

github.com

BabylonJS/Babylon.js/blob/master/src/Engines/Extensions/engine.views.ts#L137


        continue;

    }



    this.activeView = view;



    previewCamera = scene.activeCamera;

    scene.activeCamera = camera;

}



// Set sizes

if (canvas.clientWidth && canvas.clientHeight) {

    canvas.width = canvas.clientWidth;

    canvas.height = canvas.clientHeight;

    parent.width = canvas.clientWidth;

    parent.height = canvas.clientHeight;

    this.resize();

}



if (!parent.width || !parent.height) {

    return false;

}

The content should remain as you are preserving the buffer.

Liam_Parker · July 10, 2020, 4:05pm

Seems like that’s the case, how did you know? Setting the parent width and height causes the bug. If I remove line 140-141 the bug stops occurring, of course we have bigger issues at that point, the sizing goes all off. Interestingly enough, I didn’t notice anything weird about the values being set, they seem valid and correct.

I’ll give this another look using some spare time on the weekend. Keen to know if you have any further insight.

Deltakosh · July 10, 2020, 6:00pm

the fix could be to resize ONLY if there is a change (that could save us a bit of CPU btw)

Liam_Parker · July 13, 2020, 1:53am

What’s the process for the preview releases? Is it automated or does someone press a button somewhere?

Thanks for the insight Delta, it was interesting to make a contribution to the project (albeit a small one) and get a better understanding of the inner working of views.

Deltakosh · July 13, 2020, 2:00pm

This is a great one.

Preview are merged once a day and the preview.babylonjs.com is updated once a day as well