Node-RED and BabylonJS

Hi There!

I don’t know whether anyone else has posed this question but a quick search didn’t provide any results, so here goes!

Node-RED is a visual programming environment that creates 2D flows that represent the flow of data through computational units. Its mainly intended for IoT applications but I find it too generalisitic to be only used for IoT (end personal opinion).

I have created a few flows and now I have created a flow that generates a 3D representation of the 2D flow → this.

What I would like to do is add an easter-egg whereby the user taps a bunch of random keys and a super-mario-like character appears who runs around in the 3D world! (The idea would be that the Node-RED flow would be used to create the code for the 3D world but at the same time, the flow also becomes the 3D world!)

My question is: how hard would this be? As one can see from the above example, my BabylonJS talents are sufficient to place colourful blocks linked with tubes but not much else :wink:

Or to reframe the question, is there a representation of a 3D world that can be loaded into an existing 3D game? I.e. something like a 3D game with levels and the flow would become a level in that game - leaving only the conversion from Node-RED flow to a representation of a level in the game.

Any ideas and suggestions most welcome :slight_smile:

Thank you!

Hi!

I think your idea is similar with my Blackprint integration in the past, but with PlayCanvas. You can check the video preview on there if you interested, or try it directly by launching the scene.

It doesn’t modify the editor from PlayCanvas, but it does load and embed Blackprint Mini Editor into the page via script. The Blackprint instance can be saved as .json file and it can be run without embedding the Blackprint Mini Editor on production. It also has feature to generate code, but it’s still far from finish as we must write custom/template code for every nodes before we can generate codes from it.

Not every nodes is included inside of Blackprint internally, but the nodes is more like a plugin/addons. I mean like… you need to load external module that have the nodes you need (for example: nodes for Babylon.js) and have it registered into Blackprint. After that you can use the Blackprint Editor to manage the connection between nodes and have it running at runtime with the help of the Blackprint Engine.

With that modularity it can provide more freedom for programmer to create their own nodes, replace a module with custom module to customize the nodes logic that already exist, and extends the functionality of Blackprint.

Have I made nodes for Babylon.js?

Well of course, but it’s still far from finish. Currently it’s only include nodes for creating Babylon.js’s canvas and render a sphere on it, not so a big thing indeed haha. I just made it for trying to integrate Babylon.js with Blackprint, and it was works!

As I’m still maintaining Blackprint Editor and the engine, I hope someone would continue these nodes and suggesting some new useful feature to help improve the project.

Is Blackprint already stable?

Not yet, I’m still looking for suggestion of what I can improve for Blackprint.
Every increment of v0.*.0 will have breaking changes, but the breaking changes is almost rare right now as the important feature already finished.
Without other’s suggestion I may need to take longer time to figure it out by myself and make this stable.

1 Like

Wow, you did that all by yourself? Well done! :slight_smile:

Thanks for sharing :+1:

Have you played around with Node-RED? It’s the same concept done differently … could be a good place for inspiration → GitHub - node-red/node-red: Low-code programming for event-driven applications

If you want to see what the Node-RED editor looks like, then try the in browser version. Node-RED normally needs a backend execution server, the in-browser version does minimal execution in the browser and is missing much of the what Node-RED really can do however at least one can experience the editor interface.

Also there is a collection of visual programmming applications over at the Node-RED forum. There are many flow based programming (FBP) environments mainly because each feeds its own niche - Node-RED is for (I)IoT, for example.

What is missing is a generalised form of FBP which can be applied across the board to all problems. For that, one needs to have visual comparison, visual version control, visual pull requests etc - all the tooling around text-based programming needs to be emulated in a visual context.

It is … but :wink: … what I aim at is modify the Node-RED flow in 3D - so kind of building a flow editor in 3D is my idea. From what I saw, you have integrated the flow editing into PlayCanvas so that objects in a scene can be manipulated visually using a flow. Which is similar to Blenders node programming interface.

I’d like to create a 3D flow editor that is based on Node-RED, i.e., changes are linked - moving a flow-node in 3D also moves it in 2D and vice versa. But its only one idea I have!

The idea might sound strange, after all the best way to program is to use text and a keyboard - everyone knows that except those that used punchcards, they believed that punchcards was the best way to program. :wink: So I see visual programming in 2D or 3D to be a natural progression from punchcards, via text to visual.

Thank you! atm. I did most of the code and designed the engine and the editor (but not including some image/icons).

Have you played around with Node-RED?

Yes, but I haven’t tried it too far yet. When I first time play around with it, I feels it missing feature like labeled input/output ports. The user interface is so simple and convenient to use, but I think it was my personal preference to have input/output ports that can share data between nodes. So I was thinking Node-RED is created for different purpose, so I’m not continue to play with it.

But anyway, their editor looks more complete and I think I can adapt some feature inspired from it. Thank you for suggesting it :+1:

Ahh I see, thanks for the clarification. I thought you want to make the flow (but not the editor) become one of the game logic itself. I mean like when you editing the node flow (in 2D node editor) in the game, at the same time it will result changing the 3D world too. But after you mention 3D flow “editor” now I understand what you want to make. Sorry for my misunderstand. :sweat_smile:

Well… because it’s for easter egg, some unique things may looks interesting to have in the game. In my experience I have only bring the 2D node editor itself into the game level, it’s just like rendering WebView into a cube’s texture in UE4 and floating editor like in PlayCanvas.

Having the 3D flow editor seems need more work, but if I want to make it… I may also need to use raycast to check if the player is pointing into that node, or using object collision to check if player have touched that nodes in the 3D world. If Node-RED didn’t have an API to modify the flow editor then it may be bit more challenging… But if you didn’t plan to have linked changes and only want to represent 2D nodes into 3D world, then it may be more simpler.

That was the beginning “wow, I can view a 2D flow in 3D” … then I began to think about making changes in 3D that are also reflected in 2D… Then came the idea of having a character run through the 3D rendered flow. It’s more an idea than it’s going to happen - unfortunately, I have time constraints.

And you’re quite right about collision detection and object boundaries etc - hence another reason for leaving this on the back-burner. My 3D skills don’t extend to doing all that.

One thing to remember about Node-RED is that it’s not a state machine, it’s a flow machine. What NR describes are the flow of messages through a bunch of connected nodes (that do something with the message). What this means is that nodes will always have a maximum of one input - there is always just one message going through the flow and if multiple messages are flowing, then they are distinct and unique - independent from one another. Thing of the wires/lines in Node-RED as pipe through something might or might not flow, at any time.

Something likes noisecraft is a state machine - each wire is always carrying a value. So nodes can have multiple inputs with each wire carrying a value. A specific time point (given by a clock), each node computes values that it assigns to it output wires. That’s then becomes the new current value of the wire. So the lines are wires in the sense of electric circuits. This is also how Blender nodes work.

Node-RED has a very well thought out UI - there are many features that are really useful but that don’t clutter the UI. But as I said, it’s a completely different beast to other visual programming environments.

Indeed, Node-RED was originally built to route message between MQTT message buses. Hence its basic assumptions that messages are being passed through its wires. The ideas behind Node-RED are in turn based on flow based programming which underlies concepts such as unix/linux “pipes”.

1 Like