Greetings ya’ll,
I had been struggling with this for a long time, and now that I fixed it I want to get it into the BabylonJS codebase, but the Github said to post here first, so here I go…
VideoTexture.CreateFromWebCam creates a video texture from the device’s camera. However, I found that the video stream would either freeze, or start up completely black on iOS very frequently. It was a huge show-stopper. This includes all iPhones and iPads (technically iPadOS now, but whatever.)
If you don’t know how camera video streams work, they basically have to be assigned to a <video> element.
VideoTexture.CreateFromWebCam eventually calls VideoTexture.CreateFromStreamAsync, and this function actually creates the <video> element to host the video stream.
The solution to the iOS problem is this:
I discovered that appending the <video> element to the DOM fixes the problem. VideoTexture.CreateFromStreamAsync creates the element but never actually appends it to the DOM, for obvious reasons. I think we should change this so that this video functionality isn’t fundamentally broken on iPhone/iPad.
If you check out this thread which gave me the clue for this fix, another person says that when you add the <video> to the DOM, you can’t hide it with display: none; or else the bug will come back. But in my app, I simply have another element covering up the <video> so it’s no big deal.
I don’t know the correct way to add this to BabylonJS because I’m not sure how opinionated BJS should be about where exactly this video element gets popped into the DOM. Additionally, this video element must be removed when the BJS engine is disposed or the scene is disposed.
Anyways, there is nearly nothing out there on the internet about this bug that I could find so I’m hoping this post helps someone else.
By the way, because this fix is not actually in BJS yet, I fixed it in my app by actually copying the following functions out of node_modules/@babylonjs/core/Materials/Textures/videoTexture.js and implementing them as standalone functions in my codebase;
- CreateFromWebCam
- CreateFromWebCamAsync
- CreateFromStreamAsync
I tweaked CreateFromStreamAsync to add the <video> element to the DOM where I wanted it. I also had to remove references to _this and Logger which weren’t necessary anyways. So it’s a hack but the functions live pretty well outside of the VideoTexture class and overall it was pretty easy.
So, please let me know how we can fix this for the rest of the world.
