NullEngine and server-side rendering

Questions
1

Hello there :smiling_face:

I think I’m trying to do something impossible with BabylonJs but I want to be sure before switching to another technology since I love this library :heart:

My goal

Reusing a bunch a code built for Babylon to generate screenshots of Scene on the server-side (serve by Bun but that’s irrelevant for my problem).

What I tried so far

I ran into the NullEngine on Babylon.js docs but I cannot ascertain if you could or couldn’t create screenshots. The example in the doc is running perfectly but every time I try to make a screenshot with CreateScreenshotUsingRenderTargetAsync, I have a null pointer exception deep inside the babylon library. Same goes with the getDepthMap() of the DepthRenderer.

The thing is, I don’t care if all textures are black as mentioned in the doc above because I only need a screenshot for the depth (this I can do :arrow_down: in the browser with BabylonJs).

I found some topics in this forum about server-side rendering but they are 4 year-old therefore pretty deprecated.

Example of what I need :

5617c3de-32c2-48d4-9c9f-f64b92e66831
5617c3de-32c2-48d4-9c9f-f64b92e668311197×813 11.2 KB

Other ways ?

And lastly, I stumbled on the signature of the constructor of the Engine class
https://doc.babylonjs.com/typedoc/classes/BABYLON.Engine#constructor.new_Engine

It seems a canvas html element is not mandatory. What the heck is a WebGL2RenderingContext (no API documentation) ? Or the OffscreenCanvas ?

If someone could clear this up, I’ll drop my experimentations and switch to another solution, like exporting the scene and using another engine.

Puppeteer

Finally, there is the Puppeteer solution but it’s cheating and too memory intensive

But… why ?

Yes I know, my request seems very strange but thoses screenshots will be created on the fly by a python script running in Flask and implementing a ControlNet node for the InvokeAI workflow to create an animation, I have strong requirements unfortunately

Anyway, thank in advance

2

They are called Web API :slight_smile:

1 Like
3

My bad, I didn’t check MDN, I thought it was a babylon type.

And I found an implementation of WebGL1 for nodejs :open_mouth:

I’ll try ASAP with the Engine class. Sounds like black magic but who knows ?

Thank you :heart:

1 Like
4

Without a web context you don’t have many choices you should reconsider Puppeteer, it’s not cheating it’s pragmatism :sweat_smile::grin:… (Memory has never been an issue in my projects using Puppeteer)

1 Like
5

This could help too:
Rendering Scenes On A Remote Server | Babylon.js Documentation (babylonjs.com)

1 Like
6

Thank you for your reply but as I wrote, using Puppeteer is not a Babylon solution, it’s a chrome dev tools solution. Besides, I realize my approach of this problem was very silly. I’ll explain in the post below :arrow_down:

7

Some quick updates for those who’d stumble on the same problem (very unlikely but who knows ?)

First, I tried headless-gl with Babylon but running this lib inside a headless server is a “pain in the donkey”. You need to install X and running a X virtual buffer. In other words : black magic.

But I realize this morning (CET time) my approach was completely dumb. I wanted to be smart and fancy and ignore the obvious (over-engineering is one of my many sins) :rofl:

My approach was to wrap and move Babylon and my already existing webapp to the server-side in some container using exotic half-baked solutions, and then calling this customized API from within the webapp InvokeAI ( Home - InvokeAI Documentation )

But instead, it’s far simpler to call the InvokeAI API from within my Babylon webapp :laughing:
And I don’t need some heavy and convoluted dark magic. It’s straightforward (though calling the InvokeAI API is not an easy task when you have multiple nodes).

Anyway, thank you everybody, I’m trying my new approach NOW.
Sometimes only by talking about a problem will solve it.

3 Likes