How to apply material plugins to DepthRenderer

[kzhsw]

Version: 7.42.0
Engine: WebGL2
Browser: Chrome

Desc:
I’m trying to implement the WEBGL_multi_draw extension in babylon.js. Similar to BatchedMesh of three.js, a custom vertex shader (and a few samplers) needs to be applied to the rendering shader.
I found that material plugins can be used to customize rendering of PBRMaterial and StandardMaterial, but there is a lot of places in babylon.js that depends on custom shaders.
There is a customShaderOptions for ShadowGenerator that allows users to define custom shader/uniforms/samplers, and a isReadyCustomDefines callback in which defines can also be customized.
But in case of DepthRenderer everything is packed in a isReady method, users have to override the whole method to customize it. And unlike ShadowGenerator, there are a lot of DepthRenderers created implicitly, for DepthReducer and some post processes.
So, the question is, how to apply something like a material plugins to DepthRenderer, and possibily other classes that uses a custom shader, like OutlineRenderer?

[Evgeni_Popov]

You can use the setMaterialForRendering function of the depth renderer to use a specific material. If you use a standard/PBR material, you can add plugins to them.

Here’s an example of a material with a plugin used to render PCS with the depth renderer:

DepthRenderer with PointsCloudSystem

Welcome aboard! You will need the material plugin described in this other thread to make PCS write to the depth buffer as the default depth material can’t handle it. You can simplify it a bit if you don’t need the size attenuation feature:

[kzhsw]

What should I do if the DepthRenderer is created implicitly? I do not even know that a DepthRenderer is created in this case.

[Evgeni_Popov]

You can get a depth renderer for a given camera by doing scene.enableDepthRenderer(camera). If a depth renderer does not exist for this camera, it will create one. If you don’t want to create a depth renderer if it does not exist, you can directly access scene._depthRenderer[id] with id=camera.id.

However, you don’t have to set a specific depth material if you don’t update the vertex positions in your plugin material: the default depth material created by the depth renderer should work.

[kzhsw]

Looks reasonable, I’ll check if this could work.

However, you don’t have to set a specific depth material if you don’t update the vertex positions in your plugin material

The postion is changed, the world matrix for transforming positions is replaced in my case, just like what three.js do.

Edit:
There seems to be a case that DepthReducer use another id

Edit: using a standard material, but the output is nothing like a depth map

[Evgeni_Popov]

I forgot this one, the depth reducer is used wrt cascaded shadow generator, when autoCalcDepthBounds is enabled.

@kzhsw

Edit: using a standard material, but the output is nothing like a depth map

You will have to generate the depth metric. See how it is done in the link I gave above (depth renderer with PCS).

[kzhsw]

Is it possible to have some custom callbacks in isReady for custom shader name, defines, uniforms, and samplers per subMesh/material to allow users to use some custom shaders, like ShadowGenerator.
The same might also be needed for GeometryBufferRenderer

[Evgeni_Popov]

There are a number of hooks available to a material plugin where you can inject code into the material, see:

https://doc.babylonjs.com/features/featuresDeepDive/materials/using/materialPlugins/#implementing-a-complete-and-well-designed-plugin

There’s no other way to inject code into a material, though (injecting code into materials is why we introduced material plugins in the first place).

[kzhsw]

Well, I mean, it seems using a material is not as performant as the custom effect in DepthRenderer, so is there a way to customize the creation of custom effect, maybe like ShadowGenerator, a _isReadyCustomDefines callback for defines, and a customShaderOptions for unifroms and samplers

[Evgeni_Popov]

No, there’s no way to customize a custom effect, as this is an implementation detail that the user shouldn’t be aware of (we could in future do away with the creation of these custom effects in layers, depth renders, etc., and use plain materials instead).

However, using a material shouldn’t be any less effective than these custom effects: have you measured any substantial differences between the two cases?