How to Override Materials

Questions
21

Thank you, particularly for pointing out the #ifdef that I didn’t know about, but perhaps I should have been clearer that this is a toy example. I used 1k spheres because it was simpler to add a lot of geometry, but in a real scene I’ll have 1k different objects with 1k different materials. So there’s no gain in using instances if I still have to replace the material on every frame for every unique object, as it will bottleneck just the same.

As I see, the only way out would be to clone (and not instance) the objects and apply the secondary shader material only once to the clone, although this would double the memory used by meshes (not a big problem, in practice) and I’ll have to apply any changes to the objects (translations, rotations, animations etc) in synchrony. Am I right?

I’ll condense all this information in the how to along examples for all cases, for using RTT as textures and multiple passes, as well as the necessary optimizations. The fork is here and I’ll push a PR once it’s finished. GitHub - Corollarium/BabylonJS_Documentation at RTT_howto

22

You could also use your own shaders instead of StandarMaterial.

StandarMaterial.isReadyForSubMesh is slow because it has to do a lot of work to know if the effect should be recreated or not.

Depending on your needs, creating your own shaders may be an alternate solution.

23

I’ll be loading gltf meshes exported from modelers. From what I checked they use PBR materials, and it’s unfortunately not viable to write custom shaders for all of them. I guess this is a common use case, so cloning seems more general.

Moving on, are animations not applied to the RTT model? Check the example: https://www.babylonjs-playground.com/#BYYJ4A#9

24

As you provide your own shaders, you are now responsible for making sure you have all the code required to handle the features you are using (like instances, mesh with skeleton, …).

Fortunately, there’s not much to do to support rigged meshes, as everything has already been done by Babylon implementers. See the changes in the magicVertexShader code:

https://www.babylonjs-playground.com/#BYYJ4A#10

25

Thanks, got it and wrote the tutorial. I found the list of shader includes too.

But there’s still something odd. Comment and uncomment the renderTarget.renderList.push(ground); line in your own example. If the ground is rendered on the RTT, the robot is not. Is that a bug or is there something still missing?

26

Ok, this one was tricky…

We set the same material (shaderMaterial variable in the PG) to both the ground and the robot in renderTarget.onBeforeRender.

The function (isReady) that is checking if the current effect is ok / should be regenerated for the ShaderMaterial class starts with:

        if (!this.checkReadyOnEveryCall) {
            if (this._renderId === scene.getRenderId()) {
                if (this._checkCache(mesh, useInstances)) {
                    return true;
                }
            }
        }

Let’s say the first mesh (the ground) in the render list has already been rendered, and so an effect has already been generated for shaderMaterial by a previous call to isReady.

checkReadyOnEveryCall is false by default, the 2nd if is true and so if this._checkCache returns true we return from the function and don’t regenerate the effect.

_checkCache is:

    private _checkCache(mesh?: AbstractMesh, useInstances?: boolean): boolean {
        if (!mesh) {
            return true;
        }

        if (this._effect && (this._effect.defines.indexOf("#define INSTANCES") !== -1) !== useInstances) {
            return false;
        }

        return true;
    }

This function returns false only if the previous effect has been compiled with an “instance” setting different from the current one. As in the PG we don’t use instances, the function returns true.

To sum up, isReady returns true early for all meshes except the first one, meaning all meshes after the first one will be rendered with the same effect (shader settings) than the first mesh. As the ground (first mesh rendered) does not use bones, the robot is rendered with a shader that does not support bones and so is not rendered correctly. Putting the ground mesh after the robot won’t work either of course, as then the ground will be rendered with a shader configured to use bones (it’s quite funny, you can test, the ground is dancing with the robot…)

The easy fix is to set shaderMaterial.checkReadyOnEveryCall = true. This way we bypass the simple tests at the start of isReady and we run the full tests based on the shader / mesh settings:

https://www.babylonjs-playground.com/#BYYJ4A#15

Another fix would be to create another instance of the shader material to be used for the robot only. Actually, it would be a better fix performance wise because the robot is really made of multiple meshes, and setting checkReadyOnEveryCall to true will trigger a full check for each of the mesh:

https://www.babylonjs-playground.com/#BYYJ4A#14

We didn’t have the problem in the previous PG because we did use instances! So the effect was correctly regenerated when rendering first the ground (no instance) and then a sphere (instanced).

27

It works properly, but there seems to be a large bottleneck yet, even with the instanced shader version. I see drops below 60fps even with this very trivial scene, apparently due to isReadyForSubmesh calls on https://www.babylonjs-playground.com/#BYYJ4A#14 which calls _prepareEffect. Here’s a Chrome profile if you want to check. Profile-BYYJ4A-14.zip (1.0 MB)

To get back to the original problem: what is an approach that works for most scenes that has good performance? Let’s consider a scene that has static objects, animated objects and instances as a definition of “most scenes”. As I understood from this thread it seems that cloning objects and setting their materials only once would be the best answer, as the expensive material checking calls are avoided.

I made another PoC, with clones: https://www.babylonjs-playground.com/#BYYJ4A#18. It seems that reloading the object is the best choice for cloning right now (How to clone a GLB model and play seperate animation on each clone?).

There are still two problems with this approach:

  • I can’t get the main scene to render only the original objects. The RTT renders correctly. setEnabled on scene.onBeforeRender seems not to do what I think it would.
  • It doesn’t seem trivial to synchronize transformations and animations on the original object and the clone. Apparently in my demo the animations are in sync by pure chance that the loader is fast enough, and they’d need a deterministic lockstep. I suppose I could make a façade object that clones the methods of the original Mesh, forwarding any calls to the methods on both meshes. It would work, but it’s ugly and I don’t think it’s very trivial to do.

The new playground doesn’t have instances, but I am assuming that any instances can also be duplicated from the cloned base objects.

BTW, if we reach some form of consensus on something that would be interesting to add to BJS to help with multiple passes I could submit a PR at a later point, or perhaps add a module to another repo. The howto I’m writing is almost finished, but since I keeping finding new problems to bother you guys… :smiley: thanks @Deltakosh and @Evgeni_Popov for all the help.

28

Warning: (too) long post!

Regarding your PG, it does not work because onBeforeRender doesn’t exist on Scene, you should use onBeforeRenderObservable.

However, it still won’t work because you disabled the cloned meshes for the rendering (onBeforeRenderObservable is called just before calling render for the RTT), meaning the RTT will be empty because it does not take into account meshes that are disabled. So you should do as you did in the first place by using renderTarget.onBeforeRenderObservable to enable the clones / renderTarget.onAfterRenderObservable to disable them (why did you change to use scene.onXXX over renderTarget.onXXX?).

In fact, that won’t still work because enabling the clones in renderTarget.onBeforeRenderObservable is too late in the processing, the collect of meshes to display by the RTT is done before. You should use onBeforeBindObservable and onAfterUnbindObservable instead.

Here’s the PG:

https://www.babylonjs-playground.com/#BYYJ4A#19 (I disabled the depth render because it is not used and it unnecessarily cluttered Spector).

There’s still a problem as you can see, which is that the robot clone (in the RTT) does not animate. It seems that when a mesh is disabled while hitting Scene.render() (which is what happens in the PG, the clones are enabled only during the RTT rendering), it won’t be animated. I don’t know the workaround for this, but it must be possible to animate the clones “by hand”.

However, I didn’t try hard to find the answer because I don’t think it’s the right solution, cloning all meshes only for RTT rendering seems overkill to me. Also, as you said, keeping in-sync animated meshes may need some thinking, and is also a performance hit: the engine has now double animations to process. You may work around this problem by reusing the animations from the original mesh for the “cloned” one, but it’s yet some more work to do (if at all possible?).

If you need performance, I think you should instead:

  • clone the shader material and set a cloned material on each mesh before RTT rendering / reset the original material back after rendering as we did previously
  • freeze the materials as much as possible: call .freeze() on the material

Last point is the one trying to avoid running a full check in isReady / isReadyForSubMesh each time. Try to do it on as many materials as possible.

At the start of StandardMaterial.isReadyForSubMesh you have:

    public isReadyForSubMesh(mesh: AbstractMesh, subMesh: SubMesh, useInstances: boolean = false): boolean {
        if (subMesh.effect && this.isFrozen) {
            if (this._wasPreviouslyReady) {
                return true;
            }
        }

So, when the material is frozen and the effect has already been generated previously, it returns early and does not perform any check. Of course, use it wisely: if anything that should trigger a material recompile ever change (lights, some defines, …) unfreeze the material and refreeze later when the material (effect) is updated.

You have the same early return in the PBR material, however you don’t have it for ShaderMaterial and I don’t know why: @Deltakosh Would it be a problem to add those 5 lines at the beginning of ShaderMaterial.isReady? It could be a small performance improvement for people wanting to fine manage their custom materials.

Here’s the PG with .freeze called on all materials:

https://www.babylonjs-playground.com/#BYYJ4A#20

However, you won’t see much (any) improvement…

That’s because we are flipping between materials all the time, and when the material changes, the previous effect used for a mesh is nullified and a new effect must be created with the flipped material (subMesh.effect is null in the above code and so this.isFrozen is of no effect)…

What we would need for maximum efficiency is that when flipping material, we also set the effect to use so that it is not recreated. I don’t think it is currently possible, when you do mesh.material = ... the engine nullifies the effect for all sub meshes of the mesh (which will trigger a recreation of the effect later on in isReadyForSubMesh).

[…] Ok, it’s possible as we can get / set the effect used by a submesh ourselves. So, save the effects created for the original / RTT passes and reuse them later just after flipping materials:

https://www.babylonjs-playground.com/#BYYJ4A#21

As expected, we now see (in Chrome performance tab and I also checked directly on my local copy of Babylon repo) that a full run of isReadyForSubMesh occurs only a single time for each of the materials (except for ShaderMaterial, as explained above, but it does not take a lot of time anyway).

As you can see, however, it requires some work and that we know what we are doing. But performances can be greatly enhanced when playing at low level with effects.

Note: I needed to access the defines associated with the effect, but as it is a private variable not visible from the outside I had to read it with submesh._materialDefines. I don’t know if exposing this variable would be something sensible or not (@Deltakosh ?)

WARNING: I’m no expert of the inner working of the engine and so take all I said / did with a big grain of salt…

It’s 6 AM, time to go to sleep (I struggled to make this work, until I realized I needed to set the materialDefines when calling setEffect)…

1 Like
29

I have no problem with exposing it :slight_smile:

30

PR here:

1 Like
31

Thanks for the masterclass and the working example. Sorry to steal your sleep :smiley: Based on your example I created one with instances and animation and everything seems to be perfect now: https://www.babylonjs-playground.com/#BYYJ4A#24

There’s one hitch: apparently if you reuse a material for another object it won’t work correctly. You can try it with sphereBase.material = grass0;. I understand that since mesh.material.isFrozen will be true for the 2nd object, it won’t have its subMeshEffects saved. It’s a minor case and could be handled by a separate flag if necessary.

Performance is acceptable and the bottleneck now seems to in isSynchronized() called by computeWorldMatrix(). I couldn’t understand how this could be a bottleneck looking at BJS source, but there were commits on 3.3.0 exactly to optimize this function, so I may be missing something. Anyway, it’s hitting 60fps on the demo.

Thanks a lot, I’ll be posting the howto PR soon.

32

The final PG with optimizations, cleaned up code. Babylon.js Playground

It’s a bit slower than I expected, if someone has ideas for further obvious optimizations let me know. This one will go to the documentation.

33

PR with the how to here:

2 Likes
34

Thanks team!!

35

Good job!

I will have a look tomorrow to the PGs and see if I can optimize something (robots disappear from screen before being entirely culled / stop being animated, that’s seems strange to me).

1 Like
36

Also, I’m getting this error with 4.1.0-beta4 (and the 4.1.0-beta3 loaders): TypeError: setting getter-only property "material" This is in the onBeforeRender callback.

37

Indeed, it’s the reason why it does not work. Instead of testing mesh.material.isFrozen we could test mesh.isFrozen and set mesh.isFrozen = true inside the if: https://www.babylonjs-playground.com/#BYYJ4A#25

With this change, it does work even when reusing grass0 for the sphere (well, sort of, it seems there’s a race condition somewhere… So better to stick with a cloned material).

I don’t see isSynchronized() being the bottleneck on my computer. It’s rather Engine.updateDynamicVertexBuffer, which is used to update the matrices for the clones, so it’s more or less expected.

That’s because the source robot is being culled, so no problem here.

38

I don’t have this error in the PG nor on my computer.

39

Maybe that’s because you added a second renderLoop / scene.render function in the PG. The PG environment already sets up a render loop, so no need to create one. I have removed it and used scene.onBeforeRenderObservable.add to update the time uniform instead:

It’s still Engine.updateDynamicVertexBuffer that takes the most time, so I think it’s ok. After that, it is Scene._evaluateActiveMeshes, so we could freeze some meshes to improve the performance (scene.freezeActiveMeshes()), but for a demo I think there’s no need to do that.

Nice caustic effect by the way!

40

Got if. I also needed ('isReady' in mesh) besides mesh.isReady(true) or in some cases the scene would crash – perhaps another race. Everything is fast in the PG demos now.

This only happens on my own app, when I use a gltf model with instances. It didn’t happen on 4.0.x. I couldn’t reproduce it on the PG with the models there, so it may be something specific to this file. I’ll try to make a PG to reproduce it.

Thanks, but I now noticed I’m using the wrong variable vUV instead of coord on the shader. The caustic should use the world xz coordinate as its uv texture. Looks better now: https://www.babylonjs-playground.com/#S1W87B#3

I created a new PR with the changes you proposed to the code and the updated PG.

https://github.com/BabylonJS/Documentation/pull/1802

1 Like