Flickering/Issues with Multi-View

Questions
1

Hello,

I am creating an editor using babylon.js and I’m having issues with multi-views. Basically, when I click objects in my scene I want to be able to preview that object in a separate panel. I therefore register a view with the engine when my panel component is mounted (vue js component wrapping an html canvas), and de-register that view before the component is destroyed.

I followed the instructions in the official docs but the issue I am facing is that the view flickers. It seems to me that it is initialized with garbage information (as I sometimes can even see parts of my desktop rendered in that view). The effect is more apparent using firefox but it also happens on chrome. (see the attached image: the green colour is the background colour of the canvas element, the pink colour and glitches are the issue I mentioned → should be transparent).

image

Any help would be greatly appreciated! Thanks in advance!

2

It will be complicated to help unfortunately without a simple repro

can you somehow repro it in the playground? or in jsfiddle?

3

Hello David, thanks for the reply. I’ll try to reproduce in the playground and will post the link later.

1 Like
4

So, after a lot of head scratching I managed to isolate the issue to the clear colour. Somehow setting a transparent clear colour to the scene does not work when using multi canvases/views.

Is that behaviour known/expected? I tried to search for similar cases online but didn’t find anything relevant.

I created a jsfiddle where you can see the difference in behaviour between a normal canvas and a view:
https://jsfiddle.net/patrick47/9dfLczms/

Does anyone have an idea why this happens? Thanks in advance!

5

We are simply copying the content of the engine canvas to the view canvas by doing:

context.drawImage(parent, 0, 0);

So, maybe there are some limitations when alpha is involved and the parent canvas is a webgl canvas?

I checked that the parent canvas is ok even in multi view mode (meaning, the content of the canvas is ok with no flickering if you display it in the page). So, the problem really occurs at copy time.

I tried to test something like this for the copy:

const data = parent.getContext("2d").getImageData(0, 0, parent.width, parent.height);
context.putImageData(data, 0, 0);

but it fails because you are not allowed to get a 2d context on parent because there’s already a webgl context that has been retrieved for it.

Maybe @sebavan or @Deltakosh will have some inputs about this one.

1 Like
6

It looks like a low level browser issue to me as seeing the desktop should be impossible.

This is unfortunately not smthg we will be able to fix from our side.

You should register the issue on Chromium and Mozilla ?

1 Like
7

Hi, Thanks for your answers @Evgeni_Popov and @sebavan!

Could it maybe be an issue of uninitialized memory where the desktop or other glitches showing up in the canvas would be caused by random/old data in the buffer? Because the cube itself looks alright when drawn in the view, just the background looks random/glitchy.

I’ll have a look if I can find clues in the source code, taking into account the info you provided @Evgeni_Popov. If it’s a bug I’ll report like you suggested @sebavan

Thanks again for your help! :slight_smile:

8

yup the random memory sounds likely and should never happen in JS as it might be a sec issue in some cases.

9

Hi again,

I set up a local babylon dev environment on my machine and added:

context.clearRect(0, 0, 1000, 1000);

before the line that @Evgeni_Popov mentioned:

context.drawImage(parent, 0, 0);

just to see if that would change anything.

Here is the result:

image
image870×855 50.8 KB

  • The left half of the image where the bounds of the clearRect end now looks better (although the edge of the cube looks pink and green, but could that be an unrelated issue altogether?)
  • The other half which has not been cleared is still flickering (as expected).

I don’t know enough about the internals of babylon or the browser to know whether a clear is the right way to go here, but what’s your input @sebavan, @Evgeni_Popov ?

10

It seems to work (in all the PGs below I have set (through css) the background color of the canvas displayed to green (“lime”) - also, all the “non multi view” PGs are ok, obviously):

So, if it’s ok for @Deltakosh / @sebavan I think we can add a context.clearRect(0, 0, parent.width, parent.height); call just before context.drawImage(parent, 0, 0);: wanna do a PR?

11

Hey @Evgeni_Popov, thanks!
Yeah I could give it a shot this weekend. Would be a good occasion to try and contribute something back :slight_smile: (although very small)
I read a bit about the performance impact of clearRect however and it seems that it could be expensive on some devices. Maybe I can add a check for alpha (or add a flag?) in order to do the clearRect only when absolutely necessary. Then in the multi view docs page, we could warn that there might be a performance dip if using a transparent scene with a multi view setup. What do you think?

12

Seems good to me!

Let’s wait for the PR, as I don’t really know where this flag should be added (maybe we can add it as a parameter to the registerView call and store it in the EngineView class?).

13

OK Cool. :+1: In that case I’ll figure something out and create a PR this weekend so that there is something concrete to review as you suggested.

14

PR: