Can i use an HLS video stream as a Video Dome source?

, ,

I’m trying to use an HLS encoded video stream as the source for my 180 stereo video dome.

The Video in the playground isn’t stereo and the video dome is not set up to be, this is merely about loading the video in general

Expected Behavior:
The video should load onto the video element

Result:
The playlist does not load and the video does not play

Playground

There are some libraries that can manipulate the video element and create their own player to stream the video but since i can’t replace the video dome element with a player because i need the stereo functionality this is not a possible solution so i guess i’m requesting a feature if this is not supported currently? It would be really nice if this could be handled by babylon in the future to eliminate loading times of large vr videos.

The abstract answer is - we support any video that is supported in your browser. If your browser can play it in a video HTML element, Babylon should support it. If it does work in an HTML element, but doesn’t work in Bbylon, I will be very happy to check what happened.
If there is an external player that can decode the video, we can see if we can integrate it, but that would require changes in the video texture and the engine itself - and to be honest will not happen any time soon. The best would be if the custom player can stream to an Video HTML element, this way you can just use babylon as is.

Here is the example of HLS Video Dome - https://playground.babylonjs.com/#7279JP#2

Another example - video texture from DASH video format - https://playground.babylonjs.com/#9FSDC7#234

3 Likes

This works flawlessly in vr and browser in mono, but when using a stereo video it doesn’t work until the video is completely loaded, any ideas why? all i changed was using a stereo video and changing the video mode videoDome.videoMode = BABYLON.VideoDome.MODE_SIDEBYSIDE; :confused:

if you don’t change the video mode, does it play without waiting for the video to fully load?

Yes that’s what’s confusing me :sweat_smile:

I’ve replicated it in the Playground

I’ve set the videoMode to side by side and i’m loading a new video when pressing the “P” button
this works fine outside of vr but when entering vr it doesn’t work anymore, any ideas?

2 Likes

if I understated correctly, the issue is when changing the video source in the HLS class, right? it seems like there will be a need of replacing the underlying video texture, but i’ll debug that and see what can be done.

I can only assume so, but it works when changing the video when not in XR so i really don’t know.

oh, doesn’t work for me when outside of XR as well. When I click P i stop seeing the video

Hm, even after clearing my cache the video starts streaming when clicking P outside of XR

Not on any browser I tested with. The video texture goes black and no video is playing

The line mediaElement.play() was missing, my bad.
after some further testing i can’t reproduce the issue, neither on the playground nor using the default babylon template, idk what’s wrong with my code i’ll try to remove all asynchronous loading if possible.
But also my served playlist doesn’t work where the public test stream does work so somehow i’m serving my playlist wrongfully and if i use the public test stream in my original code it also doesn’t stream which means my code is faulty too. My only hope is to remove asynchronicity if that does’t work i’ll try to create the project from the beginning

Just to check, it couldn’t possibly be that my video is stereo meaning the width and height causing issues? (2880x1440) My stream does Work outside of XR and in VLC Media player but when entering vr and loading my stream it doesn’t work so i guess it has to be smth about my stream

Sorry for wasting your guys time!

1 Like

unless there is some limitation for video playback in XR that I am not aware of (not babylon, but WebXR itself), I can’t really think of a reason why the video will not play in VR mode. The flag is used as part of the rendering process, not as part of loading the video - we either split per eye or display a single image for both eyes. It also can’t really be the video’s dimensions. But I am open for everything!
If you can provide a simple reproduction that I can test with - video loaded, enter XR, video stops - I will be able to provide a more detailed answer

1 Like

Thank you, just to clarify the issue is not that the video stops playing when entering vr it actually continues playing, the issue is when in XR and trying to start a different stream the video doesn’t start playing until all parts are available, but i’ll look into it on my side, since it works with the test streams on the playground it’s either my code or my server

can you try initializing the video Dome after entering XR? does it work then?

After a week of testing i managed to replicate my issue in the Playground

How to replicate:
(It’s important to clear the cache when trying what works and what doesn’t)

:heavy_check_mark: works
Open Playground > press “P” key > video loads as expected
Open Playground > press “P” key > video loads > enter XR > video continues loading and playing properly

:question: doesn’t work
Open Playground > enter XR > press “P” key > video seems to load in network tab but doesn’t render

sorry for the ping @RaananW but since it’s been a week i’m afraid of being forgotten :sweat_smile:

1 Like