VideoTexture memory leak in Firefox

Bugs
1

My project uses VideoTexture to stream from WebRTC and the local webcam into a babylon scene. I’ve had some reports of it crashing in Firefox after using huge amounts of memory.

I’ve managed to reproduce using the following playground example: https://playground.babylonjs.com/#ZMCFYA#386

Steps to reproduce:

  1. Open https://playground.babylonjs.com/#ZMCFYA#386 in Firefox (I tested on MacOS, however I’ve had reports that this is also an issue on Windows/Linux)
  2. Allow webcam
  3. Open the Activity Monitor on your OS and look at memory usage for firefox
  4. In my experience, the memory usage starts around 600mb and then climbs continuously until the process crashes (on my M1 Pro Mac I got up to 60GB memory usage for firefox with some heavy swapping)

In my tests, it only seems to be an issue when using WebRTC streams or direct getUserMedia sources, not when streaming from a video element with src.

I suspect this is an issue with Firefox and WebRTC rather than Babylon JS specifically, however it would be awesome if we could find a workaround or a simple reproduce case for a Firefox bug report.

Thanks!

2

You might update this one 1764257 - texImage2D leaks memory with RGBA format from getUserMedia source I guess it might be the same ?

2 Likes
3

Yes, that looks like it! Do you know if there is a way to make VideoTexture use RGB instead of RGBA texture mode? That might be a quick fix for now.

4

The code is here if you want to give it a try Babylon.js/engine.videoTexture.ts at master · sebavan/Babylon.js · GitHub

Let s see if that works how to add a workaround in standard.

1 Like
5

Awesome thanks @sebavan for the pointer! I just tested this out with the local source, and sure enough changing to this._gl.RGB instead of this._gl.RGBA on that line stops the leak!

Is this something we can potentially get into the main version? Is there a situation when you want the alpha channel on a video texture?

6

I’ve managed to fix this in my project by monkey patching ThinEngine.prototype.updateVideoTexture

So no rush on fixing this in main.

Just in case anyone else experiences this before a fix lands in main, here’s what i did to patch locally: main.ts · GitHub

2 Likes
7

I will check how to integrate the workaround in today thanks for the feedback !!!

1 Like
9
3 Likes