shaderMaterial and 300es imports in MRT

[Mikael]

Hello, I’m creating a custom material to render to an MRT in which there is two textures and I’m creating a material that renders to the first (the main one, the other is for effects on post process).

Babylon.js Playground

the thing is I wanna put some lights and shadows on my custom shader as explained here:

Shadows on a ShaderMaterial?

I have had a look at ShadowOnlyMaterial and at SimpleMaterial. It seems as if the shaders there are assembled via a number of #include statements. I’m totally new to this and I’d like to understand what is being assembled there and what the end product (the whole shader source) looks like. Is there any kind of documentation on this? Or else, can you point me to the places in the source tree from where the shader snippets are included? Then I could try to understand what happens and write a prop…

And it breaks:

Babylon.js Playground

Of course my assumption is that I used 300es and obviously the imports are 100es style or something but the 300es is the only way I’ve found of targetting the first texture in the MRT, I’m not even sure it would be supported in 100es.

Is there a way to fix that?

Note: I passed the colors as uniforms through setVertex3 but it crashes so I hardcoded them in the shader.
It works well on my app but not on the PG, weird.

[Evgeni_Popov]

The ShaderMaterial is not really meant to be used with lights, see

#include directives in shader code output from Node Material Editor

The shader code generated by the NME can’t be used “as is”, you must at the very least bind the uniform and sampler variables with the right values. In your example, u_world, u_viewProjection, u_Diffusecolor, etc must be bound with a value, else they are empty by default. Even the standard matrices must be, because, for eg, u_world is not the standard name used by Babylon, it is world instead. All those things (and much others) are done for you by the NodeMaterial class so you don’t have to wor…

It is in the context of trying to use the code generated by “Generate code” in the NME, but the explanations regarding the ShaderMaterial not supporting lights is what matters.

You should try to use a node material instead, if you want to support the standard lights from Babylon.

Another possibility is to use a CustomMaterial, which is a wrapper around StandardMaterial and lets you inject some custom code at some specific parts of the shader (you also have PBRCustomMaterial as a wrapper for PBRMaterial).

[Mikael]

Thanks!

However I’ve noticed on the node editor there is no way to setup the output texture for the MRT.

[Evgeni_Popov]

Indeed, the node editor does not support multiple outputs. I guess your only option for the time being is to use a CustomMaterial.

[Mikael]

Ok, I’ll investigate that path hover looking on some examples in the PG it uses old glsl style and I’m not sure I can bring my

layout(location=0) out vec4 mainColor;
layout(location=1) out vec4 zoneColor;

in it. (as far as I understand it, maybe I’m wrong)

In other hand the only use of standard features for lighting would be just having the vector of the light direction to compare with the normal of the face (or its interpolation in case of smooth shading) and knowing if I’m in the shadow or not.
I’m not into multiple sources of lights or colored shadows.
I don’t know it simplifies the problem enough or if it remains a hassle.

[Mikael]

https://playground.babylonjs.com/#04JDPF#98
taking a look at this one, I have to provide at least the position of the light as it changes when it does, I’ll try to implement that.
Meanwhile I have to find the equivalent for shadows and that part might be tough.

If you have good advices on those path or think I’m in for a good dead end you’re welcome and of course thanks for the help!

[Mikael]

https://playground.babylonjs.com/#8RU8Q3#26
Ok light part is done (at the extend of the effect I wanna achieve).
Shadow part left.

[Evgeni_Popov]

You need to use a special syntax in the fragment shader to generate data in the different outputs in case of MRT. See how it is done in this PG:

https://playground.babylonjs.com/#8RU8Q3#25

Regarding shadows, here’s how you can do it for one light and not using any filtering method:

https://playground.babylonjs.com/#8RU8Q3#29

It’s possible to do it but definitely not easy.

[Mikael]

Nice thanks!

Still, trying it on my project I have:
404 http://localhost:9000/src/Shaders/ShadersInclude/lightVxUboDeclaration.fx
Which I usually have when my shader is not declared in the Effects so I checked in the node_modules to know what to import and there is lightUbo… but nothing about lightVxUboDeclaration and since I treeshake my project I’m not sure what to import to make it work

[Evgeni_Popov]

I think you can import import "@babylonjs/core/Shaders/ShadersInclude/lightVxUboDeclaration"?

[Mikael]

I tried but that file doesn’t exist no reference to lightVxUboDeclaration in the whole node_modules

[Evgeni_Popov]

Are you using v5.0? This file is new in 5.0 I think.

[Mikael]

4.2 but if it’s new to 5.0 4.2 shouldn’t reference it.

[Evgeni_Popov]

It is referenced by my PG:

#include<__decl__lightVxFragment>[0..1]

__decl__lightVxFragment is replaced by lightVxUboDeclaration at runtime for systems that support ubo.

[Mikael]

Removing Vx seem to work but maybe does nothing
#include<__decl__lightFragment>[0…1]

[Mikael]

ok it seem to work but I had to change it to show the thin instances but it seem to affect the orientation of the light per instance

precision highp float;

uniform vec3 diffuse;
uniform vec3 specular;

varying vec3 vDiffuse;
varying vec3 vSpecular;
varying vec3 vPosition;
varying vec3 vPositionW;

// Attributes
attribute vec3 position;

// Uniforms

uniform mat4 viewProjection;

#include<instancesDeclaration>
#include<__decl__lightFragment>[0]

void main(void) {
    #include<instancesVertex>   
    vDiffuse = diffuse;
    vSpecular = specular;
    vPosition = position;
    vec4 worldPos = world * vec4(position, 1.0);

    gl_Position = viewProjection * finalWorld * vec4(position, 1.0);

    vPositionW = worldPos.xyz;

    #include<shadowsVertex>[0]
}

of course I changed the gl_Position to make the thin instances work, that might be the source of the problem.

[Evgeni_Popov]

@Mikael

Removing Vx seem to work but maybe does nothing
#include<__decl__lightFragment>[0…1]

I don’t know, as you are using 4.2 I can’t really help, there has been quite some changes in the source code.

vec4 worldPos = world * vec4(position, 1.0);

It probably should be:

vec4 worldPos = finalWorld * vec4(position, 1.0);

[Mikael]

vec4 worldPos = finalWorld * vec4(position, 1.0);

=> Aaaand it works!

Thanks you’ve been a great help!

[Evgeni_Popov]

Glad you made it work!

You can improve the shadows by using poisson filtering:
https://playground.babylonjs.com/#8RU8Q3#30

Or PCF:
https://playground.babylonjs.com/#8RU8Q3#31

[Mikael]

At low resolution poisson helps but i’d rather raise the resolution or better narrow the frustum of the shadow map (still have to test that)
PCF doesn’t work, I suspect it to be v5.

You seem to sell me v5 a lot, since it’s a personal project not in critical prod so I might use it if you tell me it’s stable enough to be used.

I have also to take the angle from the light to the normal of the face to vary a bit the color.