Animation support on custom shaders

Questions
1

How can I have custom vertex & fragment shader with skeletal animation support?

2

They’re not well documented, unfortunately, but CustomMaterial and PBRCustomMaterial allow you to inject shader code into different parts of the StandardMaterial or PBRMaterial mega-shaders. They’re laid out in this thread. Take a look; they might be a good solution, depending on how much you’re doing in your custom shader.

1 Like
3

If you are using a ShaderMaterial, it’s already taken care for you.

1 Like
4

Yes. That’s what I am using. do these custom shaders inherit features from Standard/PBR shader? If yes, then how can I get animation working in custom shaders.

5

Thanks a lot. I will give it a try.

6

hey you can try custom material as @SpaceToast say and if you need help for import your shader in CustomMaterial let i help you

7

ShaderMaterial materials are materials you write from scratch, so they don’t inherit anything from other materials.

However, it’s very easy to support bones and instances, see this example (from the How to use RenderTargetTexture and run multiple passes doc):

https://www.babylonjs-playground.com/#S1W87B#5

You simply need to add some includes in your shader to support bones / instances. However, ShaderMaterial materials don’t support lights, see #include directives in shader code output from Node Material Editor

CustomMaterial and PBRCustomMaterial do support everything as they are really StandardMaterial and PBRMaterial in which you can inject code at some specific locations.

8

Thanks for this, it was helpful, but could you by any chance advise on how to implement a custom shader on a gltf model which is both skinned and has morph targets ? Really stuck and need some help

I did

#include<morphTargetsVertexGlobalDeclaration>
#include<morphTargetsVertexDeclaration>[0..maxSimultaneousMorphTargets]

and

#include<morphTargetsVertex>[0..maxSimultaneousMorphTargets]

inside main()

But its not working

Here is the ode just incase

  // Attributes
    attribute vec3 position;
    attribute vec3 normal;
    attribute vec2 uv;
    
    // Uniforms
    uniform mat4 worldViewProjection;
    uniform mat4 view;
    uniform mat4 projection;
    
    
    // Varying
    varying vec2 vUV;
    varying vec3 vNormal;
    varying vec3 vPosition;

    #include<bonesDeclaration>
    #include<instancesDeclaration>
    #include<morphTargetsVertexGlobalDeclaration>
    #include<morphTargetsVertexDeclaration>[0..maxSimultaneousMorphTargets]
    
void main() {
  vec3 positionUpdated = position;

  // include shaders to calculate instances and bones
  #include<instancesVertex>
  #include<bonesVertex>
  #include<morphTargetsVertex>[0..maxSimultaneousMorphTargets]

  // return normal, uv and position
  vec4 worldPos = finalWorld * vec4(positionUpdated, 1.0);
  vNormal = normalize(vec3(finalWorld * vec4(normal, 0.0)));
  vPosition = worldPos.xyz;
  vUV = uv;

  gl_Position = projection * view * worldPos;
}`;

Thanks!

9

The #include<morphTargetsVertex> in main should come first, before the include for instancesVertex. Also, if you morph the normals, you should declare a variable vec3 normalUpdated = normal and use normalUpdated afterwards.

10

Sadly this didn’t work :frowning:

Could you please check this playground ? setShaderMat var at the top controls shader vs built-in mat, with built-in pbr shader morphing works fine, but with a custom shader it stops.

The model itself contains both bone (rotateAction) and morphing (keyAction), but for some reason I can’t get the bone action to start in the playground whereas it works fine for me in my own app.
I am using BABYLON.AssetsManager locally and I can’t do that in the playground, I assume that is the cause for this, but can’t fix it.
Locally, the bone action continues working even with custom shader applied, whereas morphing stops just like in the playground example.

I would really appreciate it if you could help me figure this out.

Happy new year and Thanks again!

12

Morph targets were not supported by the ShaderMaterial class, this PR fixes that:

Note that your bone animation does not work because it is not in the file:
image

The only one animation is “keyAction” which is the morph target animation.

And Happy New Year to you too!

1 Like
14

Thank you, will track this !

If you download the model from the address linked in playground and put this model inside babylon sandbox you will see the rotateAction animation (which is bone action)

15

It’s because you need to use Append and not ImportMesh: only mesh data are loaded when using ImportMesh and the skeleton/bone animations are different pieces of data than the mesh data.

2 Likes
16

Confirm PR as working, thanks @Evgeni_Popov.

1 Like
17

Erm, sorry to bother again.
Made a slight modification to the scene which breaks the thing, could you please take a look ?

Basically, one cube now has bone anim (rotateAction) only and the other has morph anim (keyAction) only.
For some reason that causes the morphing to stop working again whereas it works with pbrmat.

Sorry :confused:

18

There’s an optimization in ShaderMaterial that makes it not updated when handling the 2nd cube, so the shader code from the 1st cube is used instead.

You can either:

  • set checkReadyOnEveryCall = true on the material to force the update for each cube
  • use another instance for the 2nd cube (clone the material)
1 Like
19

Oh, interesting that the optimization has the same effect as the original issue.
Thanks!