How to render videos frame by frame to produce smooth video recording via VideoRecorder API

Questions
1

Im noticing framerate hitches in the resulting video when recording to .webm via the VideoRecorder API. Is there a way to make the video recorder frame-rate independent (eg: recording one frame at a time ) so the video is smooth and does not rely on the framerate?

I realize rendering frame by frame is one option then using ffmpeg but this seems to require lots lots of space and additional plumbing via FFMPEG… wondering if there is a more straight forward way to control the VideoRecorder frame-by-frame?

-Anupam

2

You can save as GIF and then do with all these frames what you’d like to :slight_smile:

3

are you suggesting saving the gifs frame by frame to disk like this: Render Scenes To .png Files | Babylon.js Documentation

then using something like ffmpeg to stitch them together?

we are trying to avoid having to save the images to disk and use another service like FFMPEG…etc.
Trying to do it all in-browser, in-memory.

4

There is the function Save to GIF in Inspector.
You may use it as a source to have all frames in memory without saving to disk.

image
image1091×679 27.9 KB

1 Like
5

if you want the best quality and smooth framerate I would export as a png sequence and convert to mp4 using something like ffmpeg . For me, i ended up creating a babylon native backend service that does this. IMO probably best to do this on your backend.

-Anupam

2 Likes
6

WebCodec would be another alternative in supporting environments.

7

@Anupam_Das what library did you use to persist every frame to a PNG file?

8

you can add one png at a time via ffmpeg. So on each render step you feed it one image at a time and then finally output a movie when all frames are processed.

1 Like
9

This could help you.

1 Like