Secrets of Babylon.js

Hi guys !

Ben here, alias CraigFeldspar.

I’ve been contributing to BJS for a while now, and I’d really like to know more about you guys, this amazing community.
More specifically, I would like to know : what are the main topics you are struggling with ? or you would like to know more about ?

Maybe you are curious about the depth of the engine and WebGL calls ? or you wish you would know more about maths behind 3D graphics ?
Maybe also you would like to adopt a more streamlined process from Blender to Babylon…

Let me know everything ! I’m trying to find new ways to help you out, and your insights are gold to me.

Looking forward to your answers


I’m not sure if others have had this issue but what I find challenging is the game part of BabylonJs. BJS doesn’t have a visual game editor unlike Unity, Unreal Blueprints, Godot, or GameMaker. And that’s cool, I’m not asking for one :). We do have tutorials for small arcade games that are pretty much an action-reaction engine.

A tutorial of how to use BablyonJS like MonoGame or SDL is used: the engine inside a game. I understand it’s a more holistic focus and because of that it involves more than just what BJS methods to call so it might not be realistic It would be more a “How to make a complex game tutorial” using BabylonJS so I can understand why we don’t have anything like it.

These are my personal struggles: Using an ECS( engine to manage game state, building complex ai (behaviourTrees, stacked fsm), managing a game state step inside of BJS’s render loop, managing input state across platforms … and having it all play together using BJS.

Or you could tell me I’ve been hitting nails with a wrench and BabylonJS was never meant for anything larger than simple arcade games and I’d understand with no hurt feelings =D

Thanks for all your hard work!


At some point I would like to add special effects to my game, but I wouldn’t really know where to start. I didn’t do much research at all so it’s not surprising :). I’m also wondering, if I would hire someone to do it, would a person with only Unity experience be able and willing to adopt Babylonjs. I know very little of the whole workflow around making special effects.


Demos/tips on matching the lighting in Babylon to the lighting in Blender would be awesome! :pray: For example if I make something I like in Blender with the default studio lighting, how to make it look more similar in Babylon? Also just generally curious about your workflow from Blender to Babylon, but especially about the lighting. :slight_smile:


Yo guys !

First of all, I’m really impressed by the quality of the projects you shared here… great stuff !
What I’m getting from your answers is that you all start to have bigger scale projects, and you are limited with the workflow and making all parts work together - assets, entities, state, effects, light baking…
I strongly believe that Babylon.js is the way to go for these bigger scale projects - having myself worked for years on 1 single bjs project, I know your struggles and new challenges that come with the serious “production stuff”.
I will think about that and try to come up with content that can help advanced bjs projects. I will DM you so we can expand on that if need be.

Keep in touch ! And thank you guys for your feedback.

Anyone reading that, do NOT hesitate to post below on this topic, I’ll keep the discussion open so we can also look into your issues.


My biggest problem is knowing how to reference models in the vertex buffer (i think thats the right term) on the gpu in shaders. Like if i load some gltf, and i want to use some shader to manipulate it , i have no idea how to set that up in a game or something. So many shader examples use just 1 model, its difficult to understand how to apply shader code to a single mesh in a larger scene. Shading a primative mesh looks cool, but isnt very helpful for me.


For me this sounds also like something that could be solved with a better workflow. If your modeling tool allowed to export some metadata tied to the mesh/material, we could identify that in BJS to apply your custom material. This way you could keep a clean GLTF workflow and use custom shader stuff


This interests me a lot! The math stuff is widely available on the web (IMHO the same applies to the Blender to Babylon process - it’s basically Blender to GLB) but when starting to code more serious things I can barely find related informations. For example I was curious how can I render a fullscreen quad, apply shader to it, etc. and don’t have to use a Plane with a Material. I look at the BabylonJS source code very often to learn how things are done but it’s the hard way and I need to figure a lot of things myself. It reminds me the days back in the 90’s when I got an Amiga 500 and had minimum information about the custom chipset and how to control it. I’ve figured it myself but it took me weeks to do basic things. However the golden side of this approach is that I can remember the copper and blitter addresses until now :smiley:

Thank you very much!


I will probably have something for you in a few weeks, stay tuned :wink:

1 Like

Patiently waiting… Thanks a lot!

Great question. In general Id say covering more advanced topics is needed.
I work on product customization software, so our world is all about states both in the renderer/scene state and the data that drives it coming from outside of babylon. So tutorials on proper state management, observables and best practices would be helpful.

Also related to that perhaps is some thoughts on architectural approaches to building with babylon. For example in unity, there is ECS and component based design and a number of articles on the subject. Perhaps an article on how Babylon itself is architected and some philosophies behind it may help inform how to build on top of it.

We also have to tackle the problem of how to integrate our existing solutions into babylon, not only from a purely rendering standpoint, but also from tooling. More articles on approaches to customize and extend the inspector. As well as customize/extend the babylon editor to suit our own needs. More docs or tuts on this would be helpful as this area is fairly minimal.

Finally, more tutorials on custom shader authoring with raw HLSL not node shader editor. A great example would be to port a bare bones HLSL shader to work in babylon. (eg: take the cool ocean displacement shader from GPU gems and port to babylon)

This is what I live by (I’m very pedantic about getting the architectural design right). Started writing the first tutorial on production optimization of Babylon 3D views. If there is any specific subject you’d like to know about, post the questions here. They will make the next subject to write about.

For state management of non-game apps, a unidirectional data flow has proven itself again and again. I built from Photoshops-like Map editing app, to trading bots with the same architecture. Babylon is no exception, this 3D configurator (under development) uses:

Redux -> React -> Babylon ─┐

data flow in declarative style. It loads/saves/updates any 3D project with pure json data and CDN files, just like you would with classic product pages.

This way it is completely decoupled, and yet fully integrated with Babylon. In other words, it’s platform agnostic, be it on the web, native iOS, Android, or desktop environments.

And it produces blazing fast applications that scale almost automatically. For example, most 3D planner apps I’ve tried on the web are quite laggy in responsiveness (there is a noticeable lag from your click to UI update). With proper architecture, you have butter smooth experience like Apple software.

This subject is too deep for a single post reply. It deserves a full fledge series of articles by itself. But that is the bird’s eye view.

1 Like

I implemented a state management in a project using Vuex. I use loose coupling betwen the State, the UI (Quasar/Vue) and the BabylonJS scene using messaging.

UI → State → SceneDirector (responsible for puppeting the entities on the scene) → MessageQueue → BJS (and the whole process backwards) The State is changed by external inputs like thermal, prerssue, co2, sensors etc. as well.

I wrote about a simplified version (no Vuex involved) of the implementation here. Maybe worth to check:

1 Like

This is the reason why you should consider the unidirectional data flow - it eliminates these kinds of problems automatically, and makes your application easy to understand and debug.


Thank you!
Actually I never had problems with Vue reactivity because I know what I am doing in my code :sunglasses: or one can use Object.freeze to disable it on certain objects. Other solutions are available as well.

The messaging part is needed in my project to control the scene by MQTT messages (remote control) and I can easily change the Vue based ui to any other ui framework (we planned this when started prototyping) with this setup. All I have to do is call the correct methods in the SceneDirector class from the new ui. So it’s kinda cool but actually yeah it’s an overkill for small apps.

1 Like