Transparent Video

Questions
, , ,
1

Hey there,

this is not a question specifically related to Babylonjs but I feel like this might be interesting for people here. I need a transparent Video in a 3D Scene with playback control. WebM has the option for transparency and this works totaly fine in every browser except Safari. Safari displays the video with a black background. Here is a PG which reproduces it on Safari: https://playground.babylonjs.com/#LT4WHM#2. I tested it with Safari Version 17.1.

On Can I use Safari Version 16.0 and higher are listed with full support for WebM (WebM video format | Can I use... Support tables for HTML5, CSS3, etc). It doesnt state that transparency is not supported. I am not sure about this, but I think when Safary only partly supported WebM they listed that transparency is not supported. I think that was back in 2022.

I cant really find any official source that WebM in Safari doesnt Support transparency nor that it does. So my question is does Safari support webm transparency? If no, will it support transparency in the future?

As a workarround there are these options I see at the moment:

  1. Fallback Video for Safari as HEVC/H.265 video format with transparency (HEVC/H.265 video format | Can I use... Support tables for HTML5, CSS3, etc)
  2. Custom Shader that discards pixel (e.g. Greenscreen. Working PGs here: https://playground.babylonjs.com/#GH6TZV#29, https://playground.babylonjs.com/#GH6TZV#2) This solution leaves green/black borders. Maybe someone knows a solution how to get rid of these borders?
  3. Use a second video as opacity texture. Both videos have to be synchronized to make this work. Working PG here: https://playground.babylonjs.com/#Q4SSSV#1
  4. Looping through PNGs/Use one big Texture with all frames and offset the uv poisition. I didnt test this one but the file size will be hughe and the performance will be really bad.

There is also the AV1 video format which competes with the HEVC/H.265 format from apple. At the moment Safari only partly supports this format which might change in the future (AV1 video format | Can I use... Support tables for HTML5, CSS3, etc). I dont know much about this format and I also dont know if this format supports transparency. Maybe someone knows more about this format?

This post basically summarizes everything I found out about transparent videos so far: Overlaying Video With Transparency While Wrangling Cross-Browser Support | CSS-Tricks - CSS-Tricks

If I am missing something or if you guys have more ideas regarding this topic I would love to hear them.
Thank you for your help!

2

So, safari doesn’t support the transparency channel when using webm. It’s a known issue, and has been documented is many (MANY) angry blog posts :slight_smile:

I believe your solution, encoding the video to a different format on top of webm is the right solution.

Speaking about Babylon directly, we don’t support multiple sources of video to a single texture. So you will either need to create two textures (not the best solution) or decide which URL to load based on the environment you are at.

2 Likes