Welcome aboard!
Soft transparent shadows are not supported with node materials. However, it is possible to make them work with a bit of work:
https://playground.babylonjs.com/#CJEJD1#6
NME: https://nme.babylonjs.com/#Q85QUS#10
You need to generate yourself a Bayer dither pattern which is used to reject a fragment based on the current alpha value.
For this, I have created a custom block (new in 5.0) with the Bayer Dither code. Here’s the json file for this custom block:
{
"name": "BayerDither",
"comments": "Generates a Bayer Dither pattern",
"target": "Neutral",
"inParameters": [
{
"name": "input",
"type": "Vector2"
},
{
"name": "dummy",
"type": "Float"
}
],
"outParameters": [
{
"name": "output",
"type": "Float"
}
],
"functionName": "BayerDither8",
"code": [
"float BayerDither2(vec2 _P) {",
" return mod(2.0 * _P.y + _P.x + 1.0, 4.0);",
"}",
"float BayerDither4(vec2 _P) {",
" vec2 P1 = mod(_P, 2.0);",
" vec2 P2 = floor(0.5 * mod(_P, 4.0));",
" return 4.0 * BayerDither2(P1) + BayerDither2(P2);",
"}",
"void BayerDither8(vec2 _P, float dummy, out float res) {",
" vec2 P1 = mod(_P, 2.0);",
" vec2 P2 = floor(0.5 * mod(_P, 4.0));",
" vec2 P4 = floor(0.25 * mod(_P, 8.0));",
"#ifdef SM_SOFTTRANSPARENTSHADOW",
" res = 4.0 * (4.0 * BayerDither2(P1) + BayerDither2(P2)) + BayerDither2(P4);",
"#else",
" res = 0.0;",
"#endif",
"}"
]
}
Note that there is currently a bug (will be fixed by NME: Fix missing comma in function call (CustomBlock) by Popov72 · Pull Request #11598 · BabylonJS/Babylon.js · GitHub) when there is a single input parameter for a custom block. So I have added a second dummy parameter as a hack to overcome the bug. Once the PR is merged this parameter can be removed.
This block is used like this:
It’s simply the transcription of the GLSL code used by the soft transparent shadow code:
if ((bayerDither8(floor(mod(gl_FragCoord.xy, 8.0)))) / 64.0 >= alpha) discard;
You need to encapsulate this node material inside a ShadowDepthWrapper
to use it as the custom code to generate the shadow map (see Shadows | Babylon.js Documentation) - line 35 in the PG.
There’s still a problem, though: the Dither Bayer pattern is also used to display the mesh and not only the shadow:
You need to disable the discard code when it is not used to generate the shadow map. The easiest way I found is that BayerDither returns 0 when SM_SOFTTRANSPARENTSHADOW
is not defined (see the code in the json above): SM_SOFTTRANSPARENTSHADOW
is defined only when the shader is used to generate the shadow map.