Multi-monitor, multi-window, Electron, NW.js, Tauri, Proton Native, NeutralinoJS, BB-Native?

Questions
, , ,
1

I have been following BabylonJS for quite a few years now, and all this time I’ve been building a massive NodeJS runtime CLI application development framework that actually implements its own runtime scripting language and command interpreter. Now I find myself wondering if I can bring this kind of application development framework and empower it with a GUI front end and a BabylonJS rendering environment. I am just now starting to experiment with a few of sandbox projects to see what it will take to bring my vision to life.

One part of my vision is multi-window, multi-monitor rendering goodness. I tried this out with ElectronJS, VueJS and BabylonJS, and I was able to get a basic scene setup relatively easy I found out later as I started to get deeper into the implementation that it wasn’t truly a multi-monitor, multi-window, rendering goodness, but really just a simulator of rendering goodness by smoke and mirrors updating object positions through an IPC - Inter-Process Communication layer. Really each window had it’s own BabylonJS engine object, it’s own BabylonJS scene object, and the fact that it looked like things were moving across from one monitor to another was all just faked, and as I started adding more game objects and updates to the scene everything got super laggy and glitchy, which meant building advanced throttling technologies and message stream buffers to make it all work…and I only have 4 monitors on 1 computer…which I hope to increase to 8 monitors someday. So that would mean 2-times as many updates running across 8 different windows. This is an unacceptable solution and is not scalable. So I tried the same thing with NW.js, given that it is supposed to support a shared WebGL execution space. But then I found out that NW.js does not actually support ESM/ES6/ES7 import statements, even though they say they do. So then I researched into Tauri, Proton Native, NeutralinoJS and I found more of the same.

Then I remembered hearing about BabylonNative and so I looked into that and my eyes glazed over with formulas and API layers and I got lost in the ocean of documentation. So now here I am swimming in an endless ocean of tech. Is there anybody that can point me to a simple ESM/ES6/ES7 JavaScript based hello-world template application that uses BabylonNative to render to multiple monitors with multiple windows and a single native application JavaScript execution context? If I can just get a simple sandbox hello world app like this stood up, then I can start to tinker with it and explore different mods and enhancements to the code.

Or should I try and build my own home-grown competitor to ElectronJS using SDL2, and V8 and C-layout??

I did download and compile the BabylonNative, and I started reading through the JsRuntime.md and appRuntime.md…but again I got lost and my eyes and mind started swimming again. I am more of a hands-on type of guy, I want to get in and get started actually doing then I can learn and see how things work and how to do it myself. So this brings me back to my above question, does anybody have a simple example repo that I can fork and start to play with that uses modern ESM JavaScript and BabylonNative for Windows, simple Hello World??

Thanks
~Seth

2

Hey, and welcome!

The thing is, you’ll need some plumbing to achieve what you want.

Here’s why: Each monitor has a different memory address in the GPU, meaning each one gets a separate WebGL or WebGPU context. This results in different rendering engines for Babylon.js (or any other 3D engine).

If you want to render the same scene across multiple monitors, you can use render targets and update textures as images (like an <img> element in HTML).

The basic idea is to have a main rendering scene containing everything you need, with a dedicated camera for each monitor. Each camera will render to a specific render target (RTT), and it’s up to you to send that RTT content to a system that will display the images on the other monitor.

But be warned, dear explorer: This approach comes with a cost. Since it requires crossing multiple memory boundaries, the RTT data will need to be stored as either a base64 string or a Float32Array before it can be used by the target images.

1 Like
3

I had a try to this yesterday because I’m working with BabylonJS + Electron right now…

I don’t now why you need multiple screens to render one single scene (I mean, instead of one big screen) but I was wondering : what about using a single window expanded to multiple monitors ? For example on my side I can achieve a no framed window expanded across 2 monitors. Single engine, single context, etc … But it works only because 2 screens are same size and aligned. Any position shift or resolution difference would break the whole stuff.

1 Like
4

You can use Multiple Canvases

Then create a new window `window.open()’

and move the canvas into it.
Windows can be moved to other monitors, but when created, they always appear on the “parent” monitor (where the main browser tab was open). To save window positions, I used a Chrome extension, but I can no longer remember its name.

    const createWindow = (index) => {
        const win = window.open('', 'window' + index, `width=785, height=299`);
        win.document.body.appendChild(document.querySelector('#renderCanvas' + index));
    }

image
image1652×798 56.4 KB