How to stop NodeMaterial from generating same effects when not needed and save compilation time

Hello, we are trying to load tens of small PBR meshes a second during runtime that loading ahead of time is not an option. One large bottleneck we are experiencing is during material compilation. The number of meshes that can processed smoothly a second at a time suffer because of it. Doing some research and it seems to be generating the same effect for each mesh.

image

It’s not required in our case, the best we can tell. What’s the best way to pick a pre-existing effect for a mesh to use and skip this compilation step ? We are cloning and freezing a node material and just swapping textures.

1 Like

Ah never-mind… I was digging through the code and found can set _buildId on the NodeMaterial to force it.

The API docs say customShaderNameResolve NodeMaterial | Babylon.js Documentation is there but I don’t think its implemented in NodeMaterial at all.

hey can you share your use case in the PG? You should never have to update the buildId as it is a private

I will try to put one together. We were looking into NodeMaterial though and here Babylon.js/nodeMaterial.ts at master · BabylonJS/Babylon.js · GitHub is where _buildId is used. It’s auto incremented up top this._buildId = NodeMaterial._BuildIdGenerator++ we are cloning a NodeMaterial a hundred or so times for the pool and its incrementing that number and making custom effects for each.

ok makes sense (why cloning if you need the same effect?)

Because we want multiple objects in our scene to use the same NodeMaterial but different textures, without having to recompile for each. Perhaps this is not the best way to go about it ?

Nope it makes completely sense
I should then expose the buildId for you so you do not have to mess with an underscore property

I will make that change for next nightly

3 Likes

Setting the buildId manually seems to solve part of the problem, but we’re unclear how textures get linked to the effect atm. What we see right now is that if we have a single shared effect across multiple nodeMaterials, it looks like they all end up sharing textures - in our situation we’re loading these meshes one after the other, and it looks like whichever mesh loaded last ends up applying its textures to all of the others (or that’s a guess at any rate, all the meshes flicker between different textures all the time, occasionally being correct).

My assumption was that the material stores references to textures, and binds them to the samplers in the cached effect when sending the mesh to the GPU, so it would make sense to have multiple materials with their own textures pointing to a single effect. But maybe that’s not the case? Or did we just screw something up on our end? :grinning_face_with_smiling_eyes:

Nope it should work as you said.
Here is a simple example: Babylon.js Playground (babylonjs-playground.com)

Two materials (but same effect) with different textures
You can see that the only updates happening are regarding textures and geometries:

If you can repro your issue in the Playground I should be able to help further

1 Like

I’m not sure what the fancy viewer you’re using to debug is, so I’m going to rely on the console to count how many effects I’ve got :slight_smile:

Your scene checks out:


If I go to engine._compiledEffects there is just one entry.

This is our situation by default, when we create two nodeMaterials from the same node graph and set different textures on them: https://www.babylonjs-playground.com/#SZR2Q4#1

As you can see, default behaviour is this creates two separate effects.

Will try and figure out how to duplicate the cloning + buildId setup in a PG, but this is the crux of it.

I’m using this baby: SpectorJS - Explore and Troubleshoot your WebGL scenes with ease (babylonjs.com)

I have reproduce your issue there: Local Reflection Texture Test | Babylon.js Playground

Let me dig into it and find a solution :slight_smile:

1 Like

Alright people!!

The next nightly (will be live in less than a hour) will let you do this:

 BABYLON.NodeMaterial.ParseFromSnippetAsync("#PX0637", scene).then(nme_mat => {
        var texture_block = nme_mat.getTextureBlocks()[0];
        texture_block.texture = new BABYLON.Texture("/textures/grass.png");
        box.material = nme_mat;

        var clone = nme_mat.clone("mat2", true);
        box2.material = clone;

        texture_block = clone.getTextureBlocks()[0];
        texture_block.texture = new BABYLON.Texture("/textures/crate.png");
    });

The only difference is the 2nd parameter for the clone function. When set to true, it will let you reuse the same effect (until you call build again of course).

So this PG will work (when the PR will be live): Local Reflection Texture Test | Babylon.js Playground

Link to the PR: Nightly by deltakosh · Pull Request #10230 · BabylonJS/Babylon.js (github.com)

Enjoy!

4 Likes

Wow, that’s efficiency! Thanks, we’ll let you know how that goes :smiley:

1 Like

Yes please:)

1 Like

Working like a dream folks :slight_smile:

Lovely!

Picture is worth a thousand words, not bad performance on the swaps for a 2018 MacBook Pro !

Thank you again

1 Like

Beautiful!