VideoTexture support for Snapshot rendering

Feature requests
1

Currently, VideoTexture does not work under FAST Snapshot Rendering.

Correct me if I’m wrong, but I suspect this is because Babylon internally uses importExternalTexture.

If copyExternalImageToTexture were used instead, wouldn’t it be possible to make it work under SR, even though the performance might be lower than importExternalTexture?

If that is the case, I think it would be great to have an “SR compatibility mode” option in VideoTexture that internally switches to using copyExternalImageToTexture.

Alternatively, my understanding is that it is also possible to exclude specific draw calls from the render bundle.
Ref: WebGPU Render Bundle best practices | Toji.dev

Whichever method is used, enabling VideoTexture under SR would expand the range of creative expression.

I would highly appreciate it if you could consider this.

Thank you for this great framework!

2

Cc @Evgeni_Popov

3

Hi @syuilo, thanks for the detailed report and suggestions — you were on the right track!

A quick clarification on the internals first: for a standard material, VideoTexture on WebGPU doesn’t actually bind a GPUExternalTexture directly. Each frame it copies the current video frame into a regular internal texture (the one the material binds), so the draw call itself is already render-bundle friendly. The reason it froze under FAST snapshot rendering was different: that per-frame copy is triggered from the material bind path, and FAST snapshot skips re-evaluating/binding meshes (it replays recorded command bundles), so the copy simply never ran.

I pushed a fix that:

  • drives the per-frame video update from a scene before-render hook while FAST snapshot rendering is active, so the video keeps playing, and
  • generates the video mipmaps on a dedicated command encoder while snapshot rendering is enabled (the old path reused the main render encoder, which corrupted the recorded bundle for mipmapped video textures — the common case).

So no new “SR compatibility mode” / copyExternalImageToTexture is needed: the copy was already there, it just wasn’t being driven each frame.

PR: Fix VideoTexture not updating under WebGPU FAST snapshot rendering by Popov72 · Pull Request #18591 · BabylonJS/Babylon.js · GitHub

Once it’s merged you’ll be able to test it here (pick the WebGPU engine) — there’s a button to toggle FAST snapshot on/off so you can see the video keep playing: https://playground.babylonjs.com/?webgpu#NSMB74#2

Thanks again for the great report, and for the kind words about the framework!

2 Likes
4

Thank you for the detailed explanation and implementation!
This was necessary to enable viewing of live streams in 3D space.
I’ll try it out in my project right away.