OffscreenCanvas VideoTextures

Questions
1

I think you can already guess the challenge I’m facing from the thread title.

As I understand it we can’t use VideoTexture from a WebWorker context because WebWorkers don’t have access to the DOM and can’t create video elements.

In looking for workarounds I thought I might stream the video frames to the worker individually as images, and update a DynamicTexture instead on the worker side; however, the task is not trivial, and I have no clue if that’s really efficient; and since my app is quite video intensive and can have upwards of 30-40 VideoTextures running simultaneously, I wanted to check with the community for thoughts before I go down the rabbit hole (in case there is a smarter strategy)

Would be grateful for any thoughts/ideas on how to best approach this :slight_smile:

2

This would unfortunately involves more copy and I wonder if in the end you would not lose the rest of your gains :frowning:

1 Like
3

I managed to get a prototype pipeline as follows:

  1. Video element draws to the canvas element using canvas.drawImage()
  2. Generate an imageURL from canvas element using canvas.toDataURL()
  3. Send imageURL to the worker, and from the worker do a texture.updateURL()

This technically worked, but the performance is absolutely atrocious and basically unusable :frowning:

4

You might want to take a look at GitHub - padenot/ringbuf.js: Wait-free thread-safe single-consumer single-producer ring buffer using SharedArrayBuffer and the concept of ring buffers or circular buffers in general (If SharedArrayBuffer is option for you). This was not an exact fit for my case but this library I researched/tinkered with and sounds like it might be for yours.

1 Like
5

Thank you, this one may be a bit too advanced for me but I’ll research on my spare time see how far I get.

Appreciate it!