Gltf mesh with shader material

Sonja · April 24, 2019, 12:24pm

hi guys,
I am working on the shader material, it works fine wenn I just use build-in mesh von Babylonjs. I need to use the shader material now on the imported GLTF mesh now. I get confused because of the right-hand coordinate system and left-hand coordinate system. What does the clockweiseOrientation von PBRMaterial really do? I think I need build the same property in my shader material. Does anybody have some idea?

In Playground I put the shader material with build-in mesh (sphere) and glb file, you can check the result. The TestMat is a build-in PBRMaterial, the glb mesh with same correct setting(the clockweiseOrientation property) works fine with PBRMaterial, but I want to have this property in my shader material. I tried to change the vertex coordinate, it did nothing.

https://www.babylonjs-playground.com/#4AJ16M#45
gltf mesh with shader material

Deltakosh · April 24, 2019, 2:51pm

Hey!
So because glTF is right handed you need to have shaderMaterial.sideOrientation = BABYLON.Material.ClockWiseSideOrientation; for your material to work with your glb object:
https://www.babylonjs-playground.com/#4AJ16M#46

So unfortunately you will not be able to apply exactly the same material to a regular mesh as it will appear inverted then

The trick will be to use observable to adapt the material:
https://www.babylonjs-playground.com/#4AJ16M#47

Sonja · April 24, 2019, 11:10pm

Hi! Thanks a lot!

I thought, that the ClockWiseSideOrientation property is a part of the material, and I need to convert it in the vertex shader, but it is actually a rendering setting for all material, for PBRMaterial and Shader Material too!! I got confused because the results between regular mesh and imported glb mesh were always different. In my case, I don’t need to use the shader material with the regular mesh, just need to be sure, that the shader material works fine with the gltf mesh.

Deltakosh · April 25, 2019, 2:47pm

Perfect

Sonja · April 26, 2019, 9:58am

Hi,

@Deltakosh, I just find another issue,… wenn I just put the ShaderMaterial on the animated GLB file, the animation is somehow stopped. Do we have some settings here about the ShaderMaterial and the animation?

here is the Link in playground:https://www.babylonjs-playground.com/#4AJ16M#53
The animated GLB file works fine with PBRMaterial(in link is the TestMat), but wenn I use the ShaderMaterial, the mesh doesn’t move anymore.

Thanks,
Sonja

John_Smith · April 26, 2019, 12:28pm

Off topic question — how do you upload .glb files to the babylon js playground?

Sonja · April 26, 2019, 12:37pm

hi John,
@John_Smith, this is a available Mesh in BabylonJS Playground : Available Meshes to Import - Babylon.js Documentation
also some textures are available here: Available Textures in the Playground - Babylon.js Documentation
They are good resources for builing the questions.

I work in my project with my own glb file, but the issue is the same, the animation doesn’t work.

Deltakosh · April 26, 2019, 12:41pm

for animations (bones in this case) you have to add more code in your material to deal with the skeleton animation.

fortunately Babylon will provide you with the code for the shader:

github.com

BabylonJS/Babylon.js/blob/master/materialsLibrary/src/simple/simple.vertex.fx#L55






#include<clipPlaneVertexDeclaration>



#include<fogVertexDeclaration>

#include<__decl__lightFragment>[0..maxSimultaneousLights]



void main(void) {



#include<instancesVertex>

#include<bonesVertex>



	gl_Position = viewProjection * finalWorld * vec4(position, 1.0);



	vec4 worldPos = finalWorld * vec4(position, 1.0);

	vPositionW = vec3(worldPos);



#ifdef NORMAL

	vNormalW = normalize(vec3(finalWorld * vec4(normal, 0.0)));

#endif

Deltakosh · April 26, 2019, 12:42pm

If you do not make it work, please create a playground with your current code:)

Sonja · May 16, 2019, 2:19pm

Hi,
The simpleMaterial is a so good example for learning about Babylonjs builtin Materials. I didn’t realize that the shaders in Babylonjs are so complicated until I see this simple-Material example:sweat_smile:
I tried to begin with it, but got problem at the very first step. Does it work like the native glsl shader?
I see this [0..maxSimultaneousLights] repeating a lot of times in the shader, I think it is a loop for all of the lights. Builtin materials have maxSimultaneousLights property, for shaderMaterial I didn’t find maxSimultaneousLights property. Shoud I define it myself as a uniform? I see some definitions of maxSimultaneousLights in the material .ts file, but I don’t know how it works. When I just copy the shader code from simpleMaterial, and use them as a custom shaderMaterial, it doesn’t work, the error is Cannot read property 'maxSimultaneousLights' of undefined like the followed playground. https://www.babylonjs-playground.com/#S8DWQL#2
I don’t know if the shader include shader code already in shader store or not. When I just run it in my local computer with the full shader include code, it gets the same error.

Deltakosh · May 16, 2019, 2:34pm

No worries! Shader are hard and that’s why we have Babylon.js
here is how you can fix it:
https://www.babylonjs-playground.com/#S8DWQL#3

the [0…xxx] construct is a way to tell to babylonjs to generate X times a specific include (one for each light here for instance)

The shaderMaterial does not have this property (only supported by regular materials). So in your case, you can just repalce it by 4 for instance

Deltakosh · May 16, 2019, 2:35pm

Also please make sure to declare all uniforms in the ShaderMaterial declaration (I see that the bones or instances ones are not present)

Sonja · May 17, 2019, 12:41pm

Hi!
The bones are moving! But the lights still don’t work.
https://www.babylonjs-playground.com/#S8DWQL#4
I’m still little bit confused about the uniforms.

some of them I can get from the scene automatically, that I don’t need to set them myself: "world", "worldViewProjection", "view","viewProjection"
some are properties of textures, which I can also get: "vDiffuseInfos", "diffuseMatrix"
for the skeleton, "mBones" works after the declaring

I get confused, because I don’t know which uniform I need to set. What about the connection of the light in the scene? How can I send the light into the shader? Is through this one "vLightsType"? "vEyePosition" takes the camera position automatically? And about the sampler2D, should I also declare it as uniform?

Sorry about so many questions, but I think the default uniforms should be somewhere…where I can’t find…

Thanks!

Deltakosh · May 17, 2019, 5:04pm

No worries, this is a pretty advanced topic you are working on
This is the function that will help you probably:

github.com

BabylonJS/Babylon.js/blob/master/src/Materials/materialHelper.ts#L423


}



/**

 * Prepares the uniforms and samplers list to be used in the effect. This can automatically remove from the list uniforms

 * that won t be acctive due to defines being turned off.

 * @param uniformsListOrOptions The uniform names to prepare or an EffectCreationOptions containing the liist and extra information

 * @param samplersList The samplers list

 * @param defines The defines helping in the list generation

 * @param maxSimultaneousLights The maximum number of simultanous light allowed in the effect

 */

public static PrepareUniformsAndSamplersList(uniformsListOrOptions: string[] | EffectCreationOptions, samplersList?: string[], defines?: any, maxSimultaneousLights = 4): void {

    let uniformsList: string[];

    let uniformBuffersList: Nullable<string[]> = null;



    if ((<EffectCreationOptions>uniformsListOrOptions).uniformsNames) {

        var options = <EffectCreationOptions>uniformsListOrOptions;

        uniformsList = options.uniformsNames;

        uniformBuffersList = options.uniformBuffersNames;

        samplersList = options.samplers;

        defines = options.defines;

        maxSimultaneousLights = options.maxSimultaneousLights;

Sonja · May 20, 2019, 2:31pm

Hi,
I checked all of the uniforms, actually BABYLONJS populate the scene-uniforms( the matrices), bone-uniforms, and the eyePosition-automatically for the shaderMaterial. What about the light uniforms?

What the .ts file does, is connecting the shader to the engine. Does it mean that we don’t have this connecting for lights in shaderMaterial? I see something like this:

uniform Light0
 {
vec4 vLightData;
vec4 vLightDiffuse;
vec3 vLightSpecular;

 #ifdef SPOTLIGHT0
vec4 vLightDirection;
#endif

#ifdef HEMILIGHT0
vec3 vLightGround;
#endif

vec4 shadowsInfo;
vec2 depthValues;
} light0;

I think, they are from #include<__decl__lightFragment>[0..maxSimultaneousLights], which I’m not sure how to use it for shaderMaterial. Just for testing, I put these info directly in shader, something happens. https://www.babylonjs-playground.com/#S8DWQL#5

vec4 vLightDiffuse = vec4(1.,1.,0., 1.);

vec3 vLightSpecular = vec3(1., 1., 0.);
vec3 diffuseColor = vDiffuseColor.rgb;
vec4 shadowsInfo = vec4(0.5, 0.5, 0.5,1.);
vec2 depthValues = vec2(1., 1.);
float glossiness = 0.;
lightingInfo info = computeLighting(viewDirectionW, normalW, vLightData, vLightDiffuse.rgb, vLightSpecular, vLightDiffuse.a, glossiness);
float shadow = 1.;
diffuseBase += info.diffuse * shadow;

Could I connect the lights from the scene somehow to the shader? When I need to do it manuelly like this way, I will have big problem with multi lights.

Thanks!

Deltakosh · May 20, 2019, 2:43pm

Lights will be auto populated but you need to use the importer for that as well:

github.com

BabylonJS/Babylon.js/blob/master/materialsLibrary/src/simple/simple.fragment.fx#L84




	// Lighting

	vec3 diffuseBase = vec3(0., 0., 0.);

    lightingInfo info;

	float shadow = 1.;

    float glossiness = 0.;



#ifdef SPECULARTERM

	vec3 specularBase = vec3(0., 0., 0.);

#endif    

#include<lightFragment>[0..maxSimultaneousLights]





#ifdef VERTEXALPHA

	alpha *= vColor.a;

#endif



	vec3 finalDiffuse = clamp(diffuseBase * diffuseColor, 0.0, 1.0) * baseColor.rgb;



	// Composition

	vec4 color = vec4(finalDiffuse, alpha);

This import will populate the diffuseBase vector for you:

github.com

BabylonJS/Babylon.js/blob/master/materialsLibrary/src/simple/simple.fragment.fx#L76


#endif



	// Normal

#ifdef NORMAL

	vec3 normalW = normalize(vNormalW);

#else

	vec3 normalW = vec3(1.0, 1.0, 1.0);

#endif



	// Lighting

	vec3 diffuseBase = vec3(0., 0., 0.);

    lightingInfo info;

	float shadow = 1.;

    float glossiness = 0.;



#ifdef SPECULARTERM

	vec3 specularBase = vec3(0., 0., 0.);

#endif    

#include<lightFragment>[0..maxSimultaneousLights]

to help you get the list of uniforms, you can call this function:

github.com

BabylonJS/Babylon.js/blob/d6ab54c60505b919295cf897f34613f6542e0c28/src/Materials/materialHelper.ts#L285


/**

 * Prepares the defines related to the light information passed in parameter

 * @param scene The scene we are intending to draw

 * @param mesh The mesh the effect is compiling for

 * @param defines The defines to update

 * @param specularSupported Specifies whether specular is supported or not (override lights data)

 * @param maxSimultaneousLights Specfies how manuy lights can be added to the effect at max

 * @param disableLighting Specifies whether the lighting is disabled (override scene and light)

 * @returns true if normals will be required for the rest of the effect

 */

public static PrepareDefinesForLights(scene: Scene, mesh: AbstractMesh, defines: any, specularSupported: boolean, maxSimultaneousLights = 4, disableLighting = false): boolean {

    if (!defines._areLightsDirty) {

        return defines._needNormals;

    }



    var lightIndex = 0;

    var needNormals = false;

    var needRebuild = false;

    var lightmapMode = false;

    var shadowEnabled = false;

    var specularEnabled = false;

Like here:

github.com

BabylonJS/Babylon.js/blob/master/materialsLibrary/src/simple/simpleMaterial.ts#L125


                defines.DIFFUSE = true;

            }

        }

    }

}



// Misc.

MaterialHelper.PrepareDefinesForMisc(mesh, scene, false, this.pointsCloud, this.fogEnabled, this._shouldTurnAlphaTestOn(mesh), defines);



// Lights

defines._needNormals = MaterialHelper.PrepareDefinesForLights(scene, mesh, defines, false, this._maxSimultaneousLights, this._disableLighting);



// Values that need to be evaluated on every frame

MaterialHelper.PrepareDefinesForFrameBoundValues(scene, engine, defines, useInstances ? true : false);



// Attribs

MaterialHelper.PrepareDefinesForAttributes(mesh, defines, true, true);



// Get correct effect

if (defines.isDirty) {

    defines.markAsProcessed();

Sonja · May 21, 2019, 2:39pm

Hey!
I think I’m getting closer…
With help from MaterialHelper and MaterialDefines, I can send the infos from Babylon to the shaders (theoretically, does it work like this?). But something is still missing, the light doesn’t work without rendering error.
https://www.babylonjs-playground.com/#S8DWQL#6
I add this to the ShaderMaterial, do I still miss something?

  var myrMaterialDefines = (function (_super) {
        __extends(myrMaterialDefines, _super);
        function myrMaterialDefines() {
            _super.call(this);
            this.BUMP = true;
            this.REFLECTION = true;
            this.CLIPPLANE = false;
            this.ALPHATEST = false;
            this.POINTSIZE = false;
            this.FOG = false;
            this.NORMAL = true;
            this.UV1 = true;
            this.UV2 = true;
            this.VERTEXCOLOR = false;
            this.VERTEXALPHA = false;
            this.NUM_BONE_INFLUENCERS = 4;
            this.BonesPerMesh = 8;
            this.INSTANCES = true;
            this.SPECULARTERM = true;
            this.rebuild();
        }
        return myrMaterialDefines;
    })(BABYLON.MaterialDefines);
    //BABYLON.MaterialDefines.markAsLightDirty;
    this._defines = new myrMaterialDefines();
    if (scene.lightsEnabled && !this.disableLighting) {
        BABYLON.MaterialHelper.PrepareDefinesForLights(scene, mesh, this._defines,true, 4);
    }

Thanks!

Deltakosh · May 21, 2019, 2:42pm

You did not update the uniforms list with all the uniforms required by lights

Sonja · May 23, 2019, 3:41pm

emmm… what do you mean by updating the uniforms? You mean I need to use PrepareUniformsAndSamplersList to generate all of the uniforms again and set the effect like this(I didn’t get the point…)? https://www.babylonjs-playground.com/#S8DWQL#8

I checked the shaderMaterial.ts and the simpleMaterial.ts again in github, actually nothing is written about lights and shadows in the shaderMaterial.ts. So what I’m doing here is to add the lights defines and uniforms and somehow update them…But what I’m doing there in the PG is redefining(overwriting) all of the Defines and Uniforms, and it still works…

Deltakosh · May 23, 2019, 4:15pm

The uniform list must be known when creating the shader. Same for defines.

But the point is: why not creating your own material ? like the SimpleMaterial for instance in the material library?

The shaderMaterial is meant to be used with your own shader code. If you want to reuse babylon.js one, maybe it is better to create a new Material that will inherit from Babylon material (like the SimpleMaterial).

Topic		Replies	Views
Gltf import and pbr materials Bugs	5	771	January 22, 2019
Face ordering problems when replacing material on GLTF-imported mesh Questions	8	1171	March 12, 2019
Material SideOrientation/Normal flipped during glb export Bugs	19	1987	May 22, 2019
Set sideOrientation to Clockwise is not working in script, but works in inspector Bugs	5	706	September 29, 2022
Override Material in gltf loader Questions	2	1388	November 19, 2019

Gltf mesh with shader material

Related topics