Golden Paths for Babylon.js!

[ecoin]

I was also planning to create a 3D widget as a generic script solution that can be inserted to any website/app.

The idea is that the average web user (without CG knowledge) can go to an app, create their product in 3D using existing catalog of models (or uploading their own texture, mesh…), copy paste a script url to their site, and voila - a 3D product preview on their site.

The app part is almost ready for production release, only the script part I wasn’t able to get started. Please add me to that convo, if you do start the project, because I have exp developing many Wordpress, Express and some Django sites before.

[labris]

@ecoin

average web user (without CG knowledge) can go to an app, create their product in 3D using existing catalog of models (or uploading their own texture, mesh…), copy paste a script url to their site, and voila - a 3D product preview on their site.

I started to make a simplified version (only model loading) with Babylon Viewer some time ago but since then had no time to finish
I believe it could be a really useful addon for Babylon Viewer because it is really complicated for average web user to change Viewer settings with the code.
The idea is simple (some other 3D viewers have this implementation already):

1. Put a model URL (or load a model).
2. Tune ligths, shadows, background, some other values.
3. Get the snippet to paste anywhere.

[ecoin]

Here is a quick demo of how users can design their own models from existing catalog of meshes and materials:

[ecoin]

@bvaisman

• Direct application : build dynamically a house with a roof as explained in the doc Developing Build a House from Plans | Babylon.js Documentation . You draw the path for the ground, you expand the walls, you cut the doors and the windows, you build the ceiling, you draw the lines which represents the roof parts and you expand dynamically the roof.

After :

• being able to extrude directly from a 2d mesh. I think it’s now too difficult to build the shape of the polygon from a known complex profile before extruding it along a path.

I’ve already developed all above ideas (except the roof concept is in todo list) with dynamic Windows/Doors cutouts. Can make the final mesh exportable as file if there is demand for it (the app will be free for all to use).

Here is a snippet of how drawing wall/custom shapes work (you can then apply materials to each face separately, or hide certain sections):

[bvaisman]

Ok, @ecoin , could you tell me if it is a free solution available somewhere ?

[ecoin]

The initial alpha version will be published in a few weeks, I will make a post in this forum when it’s out.

[bvaisman]

Ok, fine, let us know when it will be available.

[Panuchka]

I would like to touch the “ECS is hard” aspect on this, since I’ve had a pretty cool developer experience using it for my game project. I suspect that for a regular web page or for a product you are creating it is pretty straightforward to use some kind of state management system (redux/mobx/vuex…) since it integrates well with the web technologies in general.

However, in games for example, I really like the ECS pattern because of the composability. It is great to be able to compose entities out of components and have individual systems to handle those components and their interactions. ECS is not a silver bullet for all the games though. For example, very story driven games might have a hard time with ECS, since ECS is more about throwing things into a world and having them interact with each other than having a tightly scripted game environment.

So anyway, I think my point is this; there are different golden paths for different situations I would personally like to see some game specific golden paths that do not depend on the architecture that has been picked. These include:

• Node material editor: Have a larger pool of examples for shaders, have more learning materials for shader creation and MOST OF ALL; a guide on adding the NME shaders to a pre-existing models. Now the process of adding shaders has been for models that have no materials. For example, how to add a shader material to a glb model that has been exported from Blender? How to use a NME without the ‘snippet server’, since in a big project you want to have the shader stored locally.
• Easier integration of physics engine; With the typescript world, it is hard to add a modern physics engine to babylon. All the current engines are not that well maintained. Maybe look into alternatives; Getting started | Rapier
• Add some physics game examples with Babylon; for example a character controller
• TODO: More when they pop to my mind

Also if there is any ECS work, I would suggest using something else than ECSY. It is not well maintained. Use these instead:

• GitHub - 3mcd/javelin: TypeScript ECS and multiplayer game development toolkit
• GitHub - NateTheGreatt/bitECS: Functional, minimal, data-oriented, ultra-high performance ECS library written in JavaScript

[labris]

@Panuchka

How to use a NME without the ‘snippet server’, since in a big project you want to have the shader stored locally.

You can directly setup a Node Material from a file saved from the Node Material Editor.
Here is the code to use:

nodeMaterial.loadAsync("file-url.json").then(() => {
    nodeMaterial.build(true);
});

[ecoin]

@Panuchka

I suspect that for a regular web page or for a product you are creating it is pretty straightforward to use some kind of state management system (redux/mobx/vuex…) since it integrates well with the web technologies in general.

Surprisingly, it was not easy at all to integrate Babylon with modern web technologies, because Babylon in general follows imperative programming.

Initially, I tested with @brianzinn 's excellent work react-babylonjs. However it wasn’t flexible enough for my use case - I wanted clean performant code written in declarative style for maximum composability and reuse, while having all user actions undoable, with separate undo channels, sharing materials, being able to render thousands/millions of objects on a regular computer, etc.

The final architecture I have is a hybrid between OOP and functional programming - conceptually, it’s similar to ECS in someway (the part where things interact with each other).

Where something needs custom behavior, I encapsulate it into a reusable React component and expose its method for other components to call.

From the draw walls example posted above, here is how the measurements are drawn when you use the pen or ruler tool:

<Tape closed={finished} points={points} scene={this.scene}/>

where points is an array of vector2 floats.

So I agree, that ECS is indeed a great choice for certain types of games/apps, my point was that it is usually hard to reason about (from my limited exp looking into Unity’s ECS, and having been spoiled by luxury tools from frontend space, like redux-saga, apollo, etc.)

[syntheticmagus]

This is fantastic stuff! Thanks again to all for weighing in on this!

The votes are now in, and I think we have a plan: the next golden path will be about developing a product viewer/configurator and integrating it into an existing website made with Express.js (with a possible bonus section integrating with WordPress, timing-dependent). @ecoin, I’ll probably reach out to you either in PMs or in a separate topic to get your thoughts since you have experience with both WordPress and Express.js. Thanks again for the excellent feedback and awesome discussion, all! I look forward to following up with the next golden path soon!

[ecoin]

Hey Justin!

In case you, or anyone here, need a production ready setup of Babylon for product viewer/configurator on the web, I’ve written a guide with project starter template using the latest ES6/7/8 with Next.js.

It should work with either Typescript or Javascript out of the box without any further configuration.

You can find it, and ask any questions, in this forum post.

[Burhantech]

Hi everyone, I was curious about the Babylon’s object tracker and wanted to ask you; would it be possible to scan image (or a QR code) and see the AR model on the image?

Would that be possible on web (no app)? If yes, how would it work?

For example; With my phone I scan an image of a bird. A pop up brings me to web (safari, chrome, opera…) and I see the AR model on the image of the bird.

Looking forward to your answers @jeremy-coleman @syntheticmagus

Best
Burhan

[jeremy-coleman]

This should get u started

GitHub - JING1201/webxr-summon-dragon-demo

There is another xr demo with a dino from one of the ms presentations, that will be useful also.

Object tracking for specific objects will require some offline training unless its a common thing that other people have already trained

[Burhantech]

Thank you for your answer. For the object tracking part. How can I show the AR model on an image?

Forexample: I show a bird model on a bird image.

Looking forward to your answer.

Best

[jeremy-coleman]

Detect → pose → map posed mesh to picture

Or you could skip posing and just use object detection, which should return you a box drawn around the picture. You could then take some random bird gltf and put its feet on the box. It depends how sophisticated you want to be. You could classify the bird species and have a model for each , or you could just take the picture and do something generative. A birdwatcher might not like that, but noone else would know.

This seemed to be well written, a quick google search gave many other promising results.
https://marcbadger.github.io/avian-mesh/

Theres also a kaggle dataset of 370 species and metadata if you want to train something yourself.