Shader crash with OIT

Bugs
1

When turning OIT on, I get a shader code crash, that comes from a double definition highp vec4 glFragColor;

layout(location=0) out highp vec4 glFragData[SCENE_MRT_COUNT];
highp vec4 glFragColor;
layout(location=0) out vec2 depth;
layout(location=1) out vec4 frontColor;
layout(location=2) out vec4 backColor;
#define MAX_DEPTH 99999.0
highp vec4 glFragColor;

I don’t use custom materials. It works in the playground, but not in our huge node-based project. Any idea, what can cause this? This creates the same result on Babylon 5.31 and 5.71. Thanks!

2

OIT is not supported with node materials because they don’t support the pre-pass renderer. We have an issue opened for this:

3

This is a misunderstanding, sorry. We use node.js. So the project is node-based. No node materials in use.

4

Sorry! So, it should not fail.

It seems you have something which is using the pre-pass renderer in addition to OIT in the same shader, which does not seem possible…

An external link would help, one that would show a working scene and where we could enable OIT afterwards (through the browser console, for eg).

5

Thanks! What could be using the pre-pass renderer other than OIT?

To me, it looks like you add junks of code to the shader and a combination of two different properties combined creates the issue.

I cannot provide any external link, as this is such a huge project. If I turn off OIT, everything works, btw.

6

SSR, SSAO, sub-surface scattering PBR are using the pre-pass renderer by default.

Regarding SSR and SSAO, you can force using the geometry buffer renderer thanks to a parameter of their constructor. So, if you are using any of them, try forcing the geometry buffer renderer.

If you are using SS scattering, try turning it off.

7

We are using none of those. Would it help, if I post the whole shader you generate?

8

Yes, please do so.

10

BJS - [16:27:21]: #version 300 es
#define WEBGL2
#define MATERIALPLUGIN_1
#define DETAILDIRECTUV 0
#define DETAIL_NORMALBLENDMETHOD 0
#define DIFFUSEDIRECTUV 0
#define AMBIENTDIRECTUV 0
#define OPACITYDIRECTUV 0
#define EMISSIVEDIRECTUV 0
#define SPECULARDIRECTUV 0
#define BUMPDIRECTUV 0
#define SPECULARTERM
#define NORMAL
#define VERTEXCOLOR
#define NUM_BONE_INFLUENCERS 0
#define BonesPerMesh 0
#define LIGHTMAPDIRECTUV 0
#define NUM_MORPH_INFLUENCERS 0
#define ALPHABLEND
#define PREPASS
#define PREPASS_IRRADIANCE_INDEX -1
#define PREPASS_ALBEDO_SQRT_INDEX -1
#define PREPASS_DEPTH_INDEX -1
#define PREPASS_NORMAL_INDEX -1
#define PREPASS_POSITION_INDEX -1
#define PREPASS_VELOCITY_INDEX -1
#define PREPASS_REFLECTIVITY_INDEX -1
#define SCENE_MRT_COUNT 1
#define VIGNETTEBLENDMODEMULTIPLY
#define SAMPLER3DGREENDEPTH
#define SAMPLER3DBGRMAP
#define ORDER_INDEPENDENT_TRANSPARENCY
#define CAMERA_PERSPECTIVE
#define LIGHT0
#define DIRLIGHT0
#define LIGHT1
#define DIRLIGHT1

#define SHADER_NAME fragment:default
precision highp float;
uniform vec4 vEyePosition;
uniform vec4 vDiffuseColor;
uniform vec4 vSpecularColor;
uniform vec3 vEmissiveColor;
uniform vec3 vAmbientColor;
uniform float visibility;
uniform mat4 view;
#define ADDITIONAL_FRAGMENT_DECLARATION

layout(location=0) out highp vec4 glFragData[SCENE_MRT_COUNT];
highp vec4 glFragColor;

layout(location=0) out vec2 depth;
layout(location=1) out vec4 frontColor;
layout(location=2) out vec4 backColor;
#define MAX_DEPTH 99999.0
highp vec4 glFragColor;
uniform sampler2D oitDepthSampler;
uniform sampler2D oitFrontColorSampler;
#define CUSTOM_FRAGMENT_BEGIN
#define RECIPROCAL_PI2 0.15915494
in vec3 vPositionW;
in vec3 vNormalW;
in vec4 vColor;
const float PI=3.1415926535897932384626433832795;
const float HALF_MIN=5.96046448e-08;
const float LinearEncodePowerApprox=2.2;
const float GammaEncodePowerApprox=1.0/LinearEncodePowerApprox;
const vec3 LuminanceEncodeApprox=vec3(0.2126,0.7152,0.0722);
const float Epsilon=0.0000001;
#define saturate(x) clamp(x,0.0,1.0)
#define absEps(x) abs(x)+Epsilon
#define maxEps(x) max(x,Epsilon)
#define saturateEps(x) clamp(x,Epsilon,1.0)
mat3 transposeMat3(mat3 inMatrix) {
vec3 i0=inMatrix[0];
vec3 i1=inMatrix[1];
vec3 i2=inMatrix[2];
mat3 outMatrix=mat3(
vec3(i0.x,i1.x,i2.x),
vec3(i0.y,i1.y,i2.y),
vec3(i0.z,i1.z,i2.z)
);
return outMatrix;
}
mat3 inverseMat3(mat3 inMatrix) {
float a00=inMatrix[0][0],a01=inMatrix[0][1],a02=inMatrix[0][2];
float a10=inMatrix[1][0],a11=inMatrix[1][1],a12=inMatrix[1][2];
float a20=inMatrix[2][0],a21=inMatrix[2][1],a22=inMatrix[2][2];
float b01=a22a11-a12a21;
float b11=-a22a10+a12a20;
float b21=a21a10-a11a20;
float det=a00b01+a01b11+a02b21;
return mat3(b01,(-a22a01+a02a21),(a12a01-a02a11),
b11,(a22a00-a02a20),(-a12a00+a02a10),
b21,(-a21a00+a01a20),(a11a00-a01a10))/det;
}
float toLinearSpace(float color)
{
return pow(color,LinearEncodePowerApprox);
}
vec3 toLinearSpace(vec3 color)
{
return pow(color,vec3(LinearEncodePowerApprox));
}
vec4 toLinearSpace(vec4 color)
{
return vec4(pow(color.rgb,vec3(LinearEncodePowerApprox)),color.a);
}
float toGammaSpace(float color)
{
return pow(color,GammaEncodePowerApprox);
}
vec3 toGammaSpace(vec3 color)
{
return pow(color,vec3(GammaEncodePowerApprox));
}
vec4 toGammaSpace(vec4 color)
{
return vec4(pow(color.rgb,vec3(GammaEncodePowerApprox)),color.a);
}
float square(float value)
{
return valuevalue;
}
vec3 square(vec3 value)
{
return valuevalue;
}
float pow5(float value) {
float sq=valuevalue;
return sqsqvalue;
}
float getLuminance(vec3 color)
{
return clamp(dot(color,LuminanceEncodeApprox),0.,1.);
}
float getRand(vec2 seed) {
return fract(sin(dot(seed.xy ,vec2(12.9898,78.233)))43758.5453);
}
float dither(vec2 seed,float varianceAmount) {
float rand=getRand(seed);
float normVariance=varianceAmount/255.0;
float dither=mix(-normVariance,normVariance,rand);
return dither;
}
const float rgbdMaxRange=255.0;
vec4 toRGBD(vec3 color) {
float maxRGB=maxEps(max(color.r,max(color.g,color.b)));
float D =max(rgbdMaxRange/maxRGB,1.);
D =clamp(floor(D)/255.0,0.,1.);
vec3 rgb=color.rgbD;
rgb=toGammaSpace(rgb);
return vec4(clamp(rgb,0.,1.),D);
}
vec3 fromRGBD(vec4 rgbd) {
rgbd.rgb=toLinearSpace(rgbd.rgb);
return rgbd.rgb/rgbd.a;
}
vec3 parallaxCorrectNormal( vec3 vertexPos,vec3 origVec,vec3 cubeSize,vec3 cubePos ) {
vec3 invOrigVec=vec3(1.0,1.0,1.0)/origVec;
vec3 halfSize=cubeSize0.5;
vec3 intersecAtMaxPlane=(cubePos+halfSize-vertexPos)invOrigVec;
vec3 intersecAtMinPlane=(cubePos-halfSize-vertexPos)invOrigVec;
vec3 largestIntersec=max(intersecAtMaxPlane,intersecAtMinPlane);
float distance=min(min(largestIntersec.x,largestIntersec.y),largestIntersec.z);
vec3 intersectPositionWS=vertexPos+origVecdistance;
return intersectPositionWS-cubePos;
}
uniform vec4 vLightData0;
uniform vec4 vLightDiffuse0;
uniform vec4 vLightSpecular0;
uniform vec4 vLightData1;
uniform vec4 vLightDiffuse1;
uniform vec4 vLightSpecular1;
struct lightingInfo
{
vec3 diffuse;
vec3 specular;
};
lightingInfo computeLighting(vec3 viewDirectionW,vec3 vNormal,vec4 lightData,vec3 diffuseColor,vec3 specularColor,float range,float glossiness) {
lightingInfo result;
vec3 lightVectorW;
float attenuation=1.0;
if (lightData.w==0.)
{
vec3 direction=lightData.xyz-vPositionW;
attenuation=max(0.,1.0-length(direction)/range);
lightVectorW=normalize(direction);
}
else
{
lightVectorW=normalize(-lightData.xyz);
}
float ndl=max(0.,dot(vNormal,lightVectorW));
result.diffuse=ndldiffuseColorattenuation;
vec3 angleW=normalize(viewDirectionW+lightVectorW);
float specComp=max(0.,dot(vNormal,angleW));
specComp=pow(specComp,max(1.,glossiness));
result.specular=specCompspecularColorattenuation;
return result;
}
lightingInfo computeSpotLighting(vec3 viewDirectionW,vec3 vNormal,vec4 lightData,vec4 lightDirection,vec3 diffuseColor,vec3 specularColor,float range,float glossiness) {
lightingInfo result;
vec3 direction=lightData.xyz-vPositionW;
vec3 lightVectorW=normalize(direction);
float attenuation=max(0.,1.0-length(direction)/range);
float cosAngle=max(0.,dot(lightDirection.xyz,-lightVectorW));
if (cosAngle>=lightDirection.w)
{
cosAngle=max(0.,pow(cosAngle,lightData.w));
attenuation*=cosAngle;
float ndl=max(0.,dot(vNormal,lightVectorW));
result.diffuse=ndldiffuseColorattenuation;
vec3 angleW=normalize(viewDirectionW+lightVectorW);
float specComp=max(0.,dot(vNormal,angleW));
specComp=pow(specComp,max(1.,glossiness));
result.specular=specCompspecularColorattenuation;
return result;
}
result.diffuse=vec3(0.);
result.specular=vec3(0.);
return result;
}
lightingInfo computeHemisphericLighting(vec3 viewDirectionW,vec3 vNormal,vec4 lightData,vec3 diffuseColor,vec3 specularColor,vec3 groundColor,float glossiness) {
lightingInfo result;
float ndl=dot(vNormal,lightData.xyz)0.5+0.5;
result.diffuse=mix(groundColor,diffuseColor,ndl);
vec3 angleW=normalize(viewDirectionW+lightData.xyz);
float specComp=max(0.,dot(vNormal,angleW));
specComp=pow(specComp,max(1.,glossiness));
result.specular=specCompspecularColor;
return result;
}
#define inline
vec3 computeProjectionTextureDiffuseLighting(sampler2D projectionLightSampler,mat4 textureProjectionMatrix){
vec4 strq=textureProjectionMatrixvec4(vPositionW,1.0);
strq/=strq.w;
vec3 textureColor=texture(projectionLightSampler,strq.xy).rgb;
return textureColor;
}
#define CUSTOM_IMAGEPROCESSINGFUNCTIONS_DEFINITIONS
vec4 applyImageProcessing(vec4 result) {
#define CUSTOM_IMAGEPROCESSINGFUNCTIONS_UPDATERESULT_ATSTART
result.rgb=toGammaSpace(result.rgb);
result.rgb=saturate(result.rgb);
#define CUSTOM_IMAGEPROCESSINGFUNCTIONS_UPDATERESULT_ATEND
return result;
}
#define CUSTOM_FRAGMENT_DEFINITIONS
void main(void) {
#define CUSTOM_FRAGMENT_MAIN_BEGIN
vec3 viewDirectionW=normalize(vEyePosition.xyz-vPositionW);
vec4 baseColor=vec4(1.,1.,1.,1.);
vec3 diffuseColor=vDiffuseColor.rgb;
float alpha=vDiffuseColor.a;
vec3 normalW=normalize(vNormalW);
vec2 uvOffset=vec2(0.0,0.0);
baseColor.rgb=vColor.rgb;
#define CUSTOM_FRAGMENT_UPDATE_DIFFUSE
vec3 baseAmbientColor=vec3(1.,1.,1.);
#define CUSTOM_FRAGMENT_BEFORE_LIGHTS
float glossiness=vSpecularColor.a;
vec3 specularColor=vSpecularColor.rgb;
vec3 diffuseBase=vec3(0.,0.,0.);
lightingInfo info;
vec3 specularBase=vec3(0.,0.,0.);
float shadow=1.;
info=computeLighting(viewDirectionW,normalW,vLightData0,vLightDiffuse0.rgb,vLightSpecular0.rgb,vLightDiffuse0.a,glossiness);
shadow=1.;
diffuseBase+=info.diffuseshadow;
specularBase+=info.specularshadow;
info=computeLighting(viewDirectionW,normalW,vLightData1,vLightDiffuse1.rgb,vLightSpecular1.rgb,vLightDiffuse1.a,glossiness);
shadow=1.;
diffuseBase+=info.diffuseshadow;
specularBase+=info.specularshadow;
vec4 refractionColor=vec4(0.,0.,0.,1.);
vec4 reflectionColor=vec4(0.,0.,0.,1.);
vec3 emissiveColor=vEmissiveColor;
vec3 finalDiffuse=clamp(diffuseBasediffuseColor+emissiveColor+vAmbientColor,0.0,1.0)baseColor.rgb;
vec3 finalSpecular=specularBasespecularColor;
vec4 color=vec4(finalDiffusebaseAmbientColor+finalSpecular+reflectionColor.rgb+refractionColor.rgb,alpha);
#define CUSTOM_FRAGMENT_BEFORE_FOG
color.rgb=max(color.rgb,0.);
color.a*=visibility;
#define CUSTOM_FRAGMENT_BEFORE_FRAGCOLOR
float writeGeometryInfo=color.a>0.4 ? 1.0 : 0.0;
glFragData[0]=color;
glFragColor=color;
float fragDepth=gl_FragCoord.z;
ivec2 fragCoord=ivec2(gl_FragCoord.xy);
vec2 lastDepth=texelFetch(oitDepthSampler,fragCoord,0).rg;
vec4 lastFrontColor=texelFetch(oitFrontColorSampler,fragCoord,0);
depth.rg=vec2(-MAX_DEPTH);
frontColor=lastFrontColor;
backColor=vec4(0.0);
float nearestDepth=-lastDepth.x;
float furthestDepth=lastDepth.y;
float alphaMultiplier=1.0-lastFrontColor.a;
if (fragDepth<nearestDepth || fragDepth>furthestDepth) {
return;
}
if (fragDepth>nearestDepth && fragDepth<furthestDepth) {
depth.rg=vec2(-fragDepth,fragDepth);
return;
}
if (fragDepth==nearestDepth) {
frontColor.rgb+=color.rgbcolor.aalphaMultiplier;
frontColor.a=1.0-alphaMultiplier*(1.0-color.a);
} else {
backColor+=color;
}
#define CUSTOM_FRAGMENT_MAIN_END
}

11

The shader compiler crashes because of the double definition of highp vec4 glFragColor;

12

It seems you are using a CustomMaterial:

#define SHADER_NAME fragment:custom_2

The name “custom_” is generated by the CustomMaterial class.

Also, this shader uses the distEffectScale and jawDistance uniforms if that helps you find the material in your project. Are you doing any special things in this material, like redefining isReadyForSubMesh for example?

13

You were too fast. I exchanged that custom material with a standard material and replaced the post. You see now
#define SHADER_NAME fragment:default
without the custom uniforms.

isReadyForSubMesh is not used or overwritten.

14

So far, I only see these two non-standard lines after the material is given:

mesh.freezeWorldMatrix();
mesh.doNotSyncBoundingInfo = true;

15

Are some materials frozen? Are you setting a scene performance priority mode other than BackwardCompatible?

16

Freeze has been used, I turned it off → same crash

No BackwardCompatible and no scene performance priorities.

17

Thanks to your inputs, I was able to reproduce the bug!

This PR will fix the problem:

18

Cool! As we don’t use SSRRenderingPipeline - what can I do now to avoid this issue?

19

I used SSR only to enable the pre-pass renderer in the example. I don’t know what enables the pre-pass renderer in your case, but the fix should be the same (at least I hope!).

20

Ok, so I keep searching or wait for the PR. Thanks!

21

I updated the PG above with the fix:

So, try to copy/paste this code in your project, so that it overrides the default Material.alpha getter/setter:

Object.defineProperty(BABYLON.Material.prototype, "alpha", {
    get: function () {
        return this._alpha;
    },
    set: function (value) {
        if (this._alpha === value) {
            return;
        }

        const oldValue = this._alpha;
        this._alpha = value;

        // Only call dirty when there is a state change (no alpha / alpha)
        if (oldValue === 1 || value === 1) {
            this.markAsDirty(BABYLON.Material.MiscDirtyFlag + BABYLON.Material.PrePassDirtyFlag);
        }
    },
    enumerable: true,
    configurable: true,
});

Put a console.log inside the setter to make sure you are using the updated version!