Strange behavior with readPixels and RawTexture CreateRGBATexture on different browser


I found a strange behavior for Safari dealing with readPixels and RawTexture CreateRGBATexture.

In our project, we need to capture the videoTexture current frame and use it to do some cross fade effect.

Our code works fine on chrome but fails on Safari.

What we are trying to do is as below.

  1. Use texture.readPixels to get the texture’s bufferView data.

  2. Create a RawTexture with those data using CreateRGBATexture function.

  3. Paste it to another material.

In chrome it works all fine, but in Safari it does not work.

I created a PG as below to demonstrate it.

Use Chrome, Safari, Firefox to open it and rotate the cube will see the different result.

The inner cube should have the same texture as the outside cube.

Chrome works while Safari fails.

Chrome version: 89.0.4389.82
Safari version: 14.0.1

Does anyone know how to fix it?

Really appreciate for your help. Thanks.

Adding @Cedric

On Chrome and Edge, I get a black inner cube :confused:

In Babylon v5 there is a breaking change with readPixels, which is now asynchronous and returns a Promise:

If your PG does work in Chrome and in some other browsers, it should mean you are using v4.2?

1 Like

Thanks for your reply. It did work.

However, I’ve found another problem while capturing the current video texture data.

In VideoTexture initialization, it created a createDynamicTexture with videoWidth and videoHeight.

Then the texture size will not change again.

Once video’s resolution changes, readPixels function will still get the initialized byteLength which is not the correct one.

If the resolution decreases, we will get a smaller texture. It can be scaled back to normal size.

But if the resolution increases, we will get a cropped texture with most of data lost and can not be recovered.

Do you have any idea to solve this kind of problem?

I think you should setup a repro in the Playground to understand what you want to do (why/when the video texture resolution changes?). If you need to change the video, maybe you should create a new video texture?

OK, I got it. I need some time to create the exact same situation in PG. We are currently using hls as the video source so that it can automatically scale up and down according to network bandwidth. However, only Safari support native hls. Other browsers need to use the hls library.