Safari HEVC/H.265 Video with transparency has black background

Bugs
, , ,
1

Hey,

I think I found a bug but im not quite sure.

I am trying to display a video which has transparency. For every Browser except Safari WebM works totally fine. Safari does not support WebM transparency and displays the video with a black background. Safari does support the H.265 Codec which also supports transparency. I have a H.265 Quicktime file with transparency and I am trying to display it as videotexture on a plane. Unfortunately I get a black background. When I use the same video as the source for a default HTML Video Element and display it, the transparency works, so I assume Babylonjs doesnt recognize the transparency.

Here is the PG which reproduces this. Every Browser except Safari will of course throw the “not supported source was found” error when executing. I tested it on Safari 17.1 on a Mac mini 2023.

image
image789Ă—665 29.2 KB

Here also a Screenshot from the PG. On the top right is the plain html Video and the bottom left displays the video as videotexture on a Plane.

Is this expected behaviour?

2

Try to set the opacity texture:

I can’t check if this works myself because your PG does not work on my iPhone SE.

3

@Evgeni_Popov Yes, I already tried using the video as opacity texture and it didnt work. What I also did was “videoMaterial.useAlphaFromDiffuseTexture = true;” which also didnt work.

4

Can anyone here reproduce this with Safari? This is actually a really big problem for me because there is no really other good solution to display transparent videos in Safari.

5

maybe try and set the material alpha mode ?

6

@MeshSlayer Its working now, but the plane is still slightly visible. Its hard to see but the plane is a bit lighter. The non transparent part of the video also becomes transparent… I think using a blend mode is not the right way, the video itself has an alpha channel and I should be able to use that. https://playground.babylonjs.com/#ZMCFYA#685

image
image620Ă—635 16.4 KB

I Placed a cube behind the video here.

image
image639Ă—613 18.3 KB

7

What if you use “combine” instead of “add” for the alpha mode?

8

I think a custom shader that cuts off the black pixels is needed here.

I tried setting videoTexture.getAlphaFromRGB to true, it’s working but the images is somehow faded out.

9

Combine results in a black background.

image
image1074Ă—926 86.2 KB

10

A Custom Shader could be used as a workarround but it leaves borders arround the object and I cant really get rid of them. I did a summary post where I listed all posibilities to use a transparent videos compatible with all browsers.

The real question is, is this a babylonjs bug because the video has an alpha channel…
In my mind setting videoTexture.hasAlpha = true should just do it, just like when using a webm as video source.

When I use videoTexture.getAlphaFromRGB and videoMaterial.opacityTexture = videoTexture it looks like the alpha channel is used but as you said the video is faded out. Here the PG that reproduces this:

image
image808Ă—698 14.9 KB

11

It seems it stopped working starting at some iOS version:

The last post is from May 2023, though, I don’t know if it’s still valid.

2 Likes
12

What exactly stopped working? They just state “Transparency stopped working in the latest iOS 16.4.1, how can I solve it?” Transparency is working when using a default video tag as I mentioned in my main post…

13

It stopped working in a Threejs snippet, so when used in a WebGL context (as in Babylon).

14

Wait, so babylon has no control how the video is displayed? So this is a WebGL thing?

1 Like
15

We use the HTML video element and copy the data from the canvas to the texture each frame. The problem could be that the video decoding performed by the browser in the canvas does not contain the alpha channel (wild guess here).

1 Like
16

@Simon_Weck you could open an issue on the webkit bug tracker ?