Running Babylon.js on a nodejs server with headless-gl

I’m trying to get Babylon.js running on a nodejs server with headless-gl and was wondering if anyone has had any success doing something like this. What I’m ultimately trying to do is load a .gltf file, then run the render loop for a specified time while capturing the frames from the GL context. Since I’m running on a nodejs server, I don’t have access to the web browser APIs so I’m using the following to simulate a browser like environment:

  • jsdom for the DOM implementation
  • headless-gl for the WebGL implementation
  • node-canvas for Canvas support in jsdom

I have been bumping into a series of problems during the creation of the babylon.js engine and the load of the resources in the scene. I am hacking and slashing to get around some of these problems and have made some progress, but I’m wondering if anyone has success doing something similar. Are there settings that I can set on Babylon.js that remove the need for a browser like context (it doesn’t look like this is the case based on some of the Babylon.js code I have seen)? Are there other node modules that would work better for simulating the browser like environment?

Any help or feedback is appreciated.

1 Like

Hi Brentonator! Sorry for no replies. How is your progress on this? I wish you would have gotten some replies. (sniff) Maybe after this topic-bump… we will get more info.

We succeed in rendering server side with GitHub - GoogleChrome/puppeteer: Headless Chrome Node API

1 Like

Maybe have a look at the NullEngine() (Server Side - Babylon.js Documentation). Would that cover your use case? I have been successfully running a headless babylonjs on nodejs with it.

NullEngine Seems can’t output the image of the render result but it is needed for many cases on the server side.

But that is the whole point of the NullEngine. Running BabylonJS without any WebGL and rendering, thus of course also no image.
What is your usecase here? It sounds very much like the NullEngine is not the right tool here.

Sorry for so late replying, I just want to take some screenshots of the 3D model on the server-side.

HI there @vimcaw:

I’m trying to get some server side renders from BJS scenes, using NodeJS and Puppeteer, and facing the problem of how to know when the scene is completely created and shown, in order to call the Puppeteers’ screenshot API.

Have you got any tips on it?

1 Like

Hi there @sharp:

Any tips about how to know when the BJS’ scene to screenshot via Puppeteers is completely created and shown?

Thanks for your time.

Hi @paleRider you can trigger a custom-event puppeteer/custom-event.js at main · puppeteer/puppeteer · GitHub in your scene.executeWhenReady for example or see https://babylonjs.medium.com/understanding-asynchronous-code-in-babylon-js-cb534634beb for more details

1 Like

Thanks for your advice @sharp.

The case is that I can’t have it working, I’m already tried and checked several ways, but It refuses to work.

In my (static) HTML I have a JS object (Singleton) with the BJS functionality that renders the scene in a canvas fulfilling the whole page. The render is shown OK, and I fire the custom “sceneReady” event from scene.executeWhenReady as you state:

(Excerpt from com.vortice3d.bjs.js, called from render.html)

//2
_scene = new BABYLON.Scene(_engine);
_scene.clearColor = BABYLON.Color4.FromHexString('#00000000');
//
_scene.executeWhenReady(function () {
    _engine.runRenderLoop(function () {
        _scene.render();
     });
     //
     window.dispatchEvent(new Event("sceneReady"));
     console.log(_TAG+"sceneReady Dispatched!");
});

On the other side, my NodeJS module, using Puppeteer, is the classic for taking screenshots and for implementing custom-event listening:

var puppeteer = require('puppeteer');
var _TAG = "com.vortice3d.screenshot\t";
var _DEFAULT_VIEWPORT = {
    width: 1920,
    height: 1080,
    deviceScaleFactor: 1,
  };

async function takeScreenshot(params) {
    //1
    var browser = await puppeteer.launch({
        defaultViewport: _DEFAULT_VIEWPORT,
        headless: false
    });

    //2
    var page = await browser.newPage();
    //enable "console.log" inside "page.evaluate"
    page.on('console', function (consoleObj) {
        console.log(_TAG+consoleObj.text());
    });

    await page.exposeFunction('onCustomEvent', function(evt){
        console.log(`${evt.type} fired`, evt.detail || '');
    });

    await page.evaluateOnNewDocument(function () {
        document.addEventListener("sceneReady", function (evt) {
            window.onCustomEvent({
                type: "sceneReady",
                detail: evt.detail
            });
        });
    });

    var result = await page.goto(params.url, {
        /*waitUntil: "domcontentloaded"*/
        /*waitUntil: 'networkidle0'*/
    });
    console.log(_TAG + "Result: " + result.status());
    
    await page.waitForTimeout(2000);
    var canvas = await page.$('canvas');
    await canvas.screenshot({
        omitBackground: true,
        path: params.path
    });

    //await page.close();
    await browser.close();
}

module.exports = takeScreenshot;

With this I can get my screenshots with the BJS scene completely rendered, but only thanks to the addition of the await page.waitForTimeout(2000), that is not the desired way, as the exposed custom event handler (onCustomEvent) is never called. Timeover delay is a “magic number” and so the correct way is to go with the custom event instead.

Any tip with this?

Thanks for your time.

P.S:
Of course, in the case onCustomEvent is fired, the screenshot functionality would have to be moved there.
“Dispatched!” effectively is shown in my console, proving dispathEvent is called.
Using document.dispatchEvent(new Event(“sceneReady”)); doesn’t make the trick.

Hi @paleRider, sorry I can’t study your code right now but if you want a simple and basic solution to make screenshot you can expose a function like this :

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();

    await page.exposeFunction('screenshot', async (e) => {
        await page.screenshot({ path: 'example.png' });
        await browser.close();
    });

    await page.goto('http://localhost/');
})();

and in your HTML :

_scene.executeWhenReady(function () {
    _engine.runRenderLoop(function () {
        _scene.render();
     });
     screenshot();
});
1 Like

Twice thanks indeed, @sharp:

Reviewing your code I’ve realized where was my error with the customEvent.

The case is that it must be:

 await page.exposeFunction('onCustomEvent', async function(evt){
        console.log(`${evt.type} fired`, evt.detail || '');
    });

instead of:

 await page.exposeFunction('onCustomEvent', function(evt){
        console.log(`${evt.type} fired`, evt.detail || '');
    });

That is, the callback must be stated as async.

Best regards.

1 Like

@sharp @vimcaw @jodo @Brentonator
Have you also found a way to manipulate model (like changing textures and all) before taking a screenshot?
I keep failing to do so.
Where exactly do I have to put

//2
_scene = new BABYLON.Scene(_engine);
_scene.clearColor = BABYLON.Color4.FromHexString('#00000000');
//
_scene.executeWhenReady(function () {
    _engine.runRenderLoop(function () {
        _scene.render();
     });
     //
     window.dispatchEvent(new Event("sceneReady"));
     console.log(_TAG+"sceneReady Dispatched!");
});

these?

were we able to resolve this issue? how want to use puppeteer to print out of Babylon scene?

@RaananW was doing this within our automated tests, maybe he has some inputs?

This is how we do it in our docs:

Documentation/lib/buildUtils/tools.ts at master · BabylonJS/Documentation (github.com)

And in my template you can find a way to run vis-tests using playwright:
babylonjs-webpack-es6/tests/validation.spec.ts at master · RaananW/babylonjs-webpack-es6 (github.com)

Eventually, playwright/puppeteer are just browser orchestrators. Anything you can run in a browser you can run there, and then ask for a screenshot.

1 Like