How can I create a video by use BabylonReactNative?

Questions
1

just like the title

I am using “BabylonReactNative” to create a animated 3d model. I noticed that the web version has the feasibility of creating videos (How to render videos frame by frame to produce smooth video recording via VideoRecorder API - #3 by Anupam_Das) . So I want to how can I made this by using BabylonReactNative ?

2

Adding @sebavan to this thread. I’m not Babylon has a user facing API that can easily do this.

3

Babylon.js does have VideoRecorder | Babylon.js Documentation (babylonjs.com), but this does not work in BabylonReactNative today as it depends on WebAPIs that are not polyfilled in BabylonNative.

Ideally BabylonNative would either implement the required WebAPIs, or implement a higher level VideoRecorder abstraction, which would then be available in BabylonReactNative as well. I’m not aware of any planned work for this, but @bghgary can confirm.

It is however possible to write your own video encoder as a React Native module and use it in conjunction with BabylonReactNative. Your video encoder module would need to use React Native’s JSI to be able to efficiently pass ArrayBuffers (of per video frame RGB image data) from JS to native, and the video encoder implementation could either be cross plat (e.g. using ffmpeg or something) or there could be multiple per-platform implementations using system provided video encoding APIs.

Once that is in place, you can read pixel data from individual frames being rendered by Babylon. I think there are two ways you can do this today:

  1. Render to a RenderTargetTexture, and then call readPixels on that texture. That would give you back an ArrayBuffer of rgb data that could be passed to a video encoder.
  2. Use the BabylonReactNative specific CaptureSession class. This is an experimental API that most likely will be removed in the future (when BabylonNative supports video capture directly), but in the meantime you could use this class to also get raw rgb data that could be passed to a video encoder.
1 Like