has everyone ever tried out whether Oculus Mixed Reality Capture (MRC) works together with a babylonjs application?
MRC basically allows a video capture of a person wearing a VR headset in front of a greenscreen to be composed with a virtual image captured from the 3D scene from the virtual position that matches the camera in the real world. This gives the impression of a video capture from within the VR application but with the real video capture of the person.
I read from the docs of MRC that an application needs to support this. From a technical point of view, I get that basically the application needs a second camera to render the application from the point of view of the real camera capturing the person. This camera is send as a video stream to another computer that also gets the video stream from the camera with the greenscreen and composes one common video stream.
Any thoughts, any ideas about the topic?
Here is the original docs from Meta:
- Oculus Creators Portal | Mixed Reality Content Tool
- And here, someone has done it with three.js: [Experiment] Mixed Reality Capture with a browser (WebXR +Three.js) - YouTube
I have an Oculus Quest available but so far, did not bother to set MRC up in my appartment, but willing to do so by next week.