StandardMaterial: needDepthPrePass writes whole-quad depth when transparencyMode = MATERIAL_ALPHATEST

Bugs
1

Hi all. We use alpha-tested leaf cards with needDepthPrePass to cut foliage overdraw, and found that on StandardMaterial the depth prepass skips the alpha test when the material uses transparencyMode = Material.MATERIAL_ALPHATEST. The prepass writes the depth of the entire quad, so the transparent part of the card hides whatever is drawn behind it afterwards.

Playground: https://playground.babylonjs.com/#G5C5KQ#1 Toggle needDepthPrePass to see the hole appear and disappear. It reproduces on both WebGL2 and WebGPU.

Cause. With a non-null transparencyMode, StandardMaterial defines ALPHATEST_AFTERALLALPHACOMPUTATIONS, which moves the alpha test to the end of default.fragment. But #include<depthPrePass> sits near the top and returns early in the DEPTHPREPASS variant, so in that variant the test never runs. (Line numbers from the built 9.17.1 shaders: GLSL default.fragment 120 / 134 / 256–257, WGSL 111 / 124 / 246–247.) PBR is not affected, because its alpha test runs inside the albedo/opacity block before the prepass exit. StandardMaterial on the legacy path (diffuseTexture.hasAlpha without transparencyMode) is not affected either, because it tests early.

Possible fix. In the DEPTHPREPASS variant, run the alpha test before the early return. Minimal version, just before #include<depthPrePass>:

#if defined(DEPTHPREPASS) && defined(ALPHATEST) && defined(ALPHATEST_AFTERALLALPHACOMPUTATIONS)
    if (alpha < alphaCutOff) discard;
#endif

Caveat: at that point alpha includes the diffuse-texture alpha but not contributions applied later (for example an opacity texture), so a complete fix would compute alpha fully before the prepass exit. We’re working around it with a small material plugin that injects the test above at CUSTOM_FRAGMENT_UPDATE_ALPHA, and would be glad to open a PR if you tell us which shape you prefer.

2

Thanks for the detailed report and Playground! Your diagnosis is correct: when transparencyMode is set, StandardMaterial postpones the alpha test until all alpha contributions have been computed. The depth-prepass variant exits before reaching that test, so it writes depth for the full primitive instead of only the pixels that survive the alpha test.

The current WGSL source implements the early exit a little differently, by excluding the remaining shader code at preprocessing time, but the result is the same on both backends.

Your proposed test before depthPrePass fixes the diffuse-alpha case demonstrated by the Playground. As you noted, however, it would not account for the opacity texture, vertex or instance alpha, or opacity Fresnel. That could leave the depth and color passes with different coverage.

Please feel free to open a PR. I think the complete fix should factor the StandardMaterial alpha computation and test into a shared shader block that is evaluated before depthPrePass in the depth-only variant, while remaining at its current location in the regular color variant. This would preserve the existing color-pass ordering and the depth-prepass early exit while ensuring that both passes use the same built-in alpha inputs.

A visualization test covering the reported diffuse-alpha case on WebGL2 and WebGPU would be useful. If practical, it would also be good to cover at least one additional alpha contributor, such as an opacity texture, to ensure that the two passes remain consistent.