How to stream pc monitor into 3d world space?

I want to create an web app that the user can stream multiple pc screen, something like immersed on oculus quest 2, but I don’t know how to stream monitor into 3d world, can anyone help?

You would need to use webRTC to capture the screens and a videoTexture to display the stream in babylon.


@sebavan Thanks, I will try it, another question, does this work with vr controller? I want to control mouse cursor with meta quest 2 controller

You should be able to handle the controllers following the XR documentation.

@sebavan Thanks for reply, I still can’t figure out how to interact with stream screen in 3d world space, can you help me?

I am definitely XR noob but you should be able to inspire yourself from:

as well as WebXR Controllers Support | Babylon.js Documentation

cc @RaananW for when he ll be back from vacation

1 Like

Those are the two pages i would recommend as well. VR controllers are fully supported, and the API is fully documented there.

About streaming screens to a web experience - not that simple. I know @gabrieljbaker of (the amazing) Frame VR is using a service for this, maybe he can share its name.