How to do lighting with ShaderMaterial?

Hello,

I had a mesh with a slightly complex and parameterized geometry. Originally the positions were implemented in JS and moved to the GPU whenever they changed.

I thought it might be a good idea to port the geometry/position calculation from JS to a WebGL vertex shader. In the Babylon docs I have found that ShaderMaterials support this. It took me a little while to get used to the idea that the geometry is part of the material, but then I came up with something like this:

In the original JS-based implementation I

  • let Babylon compute the normals of my mesh,
  • used the baseTexture as my standard material’s diffuseTexture,
  • configured some specularColor, and
  • provided a few lights

and was happy with BabylonJS taking care of the lighting.

Now my ShaderMaterial behaves more like an emissiveTexture. I could, of course, try to implement the lighting (specular + diffusive behavior) by myself, but I’d still prefer to leave this to Babylon. (It wouldn’t be hard to provide the normals, but I do not want to replicate the rest of the lighting functionality.)

So my question is: Is there a way to let Babylon take care of the lighting after computing the positions in a WebGL vertex shader? (It need not be based on ShaderMaterial. I used this just because this was the only way I found in the docs to provide a vertex shader to Babylon.)

Regards,
Heribert

I would suggest 2 options:

  • Node Material: Babylon.js docs so you even have a visual editor :slight_smile:
  • Material Plugin: Babylon.js docs which offers a way to inject your own code within an existing standard / PBR material.

Thank you for the hints!

  • I had a glance at the Node Material but got the impression that switching to that would be
    too much of a leap, in particular since I am meanwhile somewhat familiar with WebGL
    and I already had some hand-written WebGL code for the needed functionality.
  • So I wrote a Material Plugin and it works :tada::
    https://www.babylonjs-playground.com/#WAEV2S#1
    Compare this to the approach without proper lighting linked in my initial post.
    (I had to figure out the maths for the normals. These are actually the “correct” normals.
    Probably approximate normals computed from the triangulation would be good enough.
    Maybe that approximation could be left to some existing Babylon code as well.)
  • To get the Material Plugin to work, I had to browse quite some BabylonJS source code on Github.
    There I also came across the code for PBR materials and got the impression that PBR materials
    might be adaptable to my needs even without the need for a plugin.
    But that is something still to be investigated. I will report on my results.
3 Likes

Ok, as promised here are the results of my investigations regarding PBR materials, just in case someone with a similar problem stumbles over this thread:

  • Not only PBR materials are adaptable, but also StandardMaterial can be adapted.
    I am concentrating on the latter here.
  • There is CustomMaterial, a subclass of StandardMaterial.
    • Advantage over MaterialPlugin: At least for my use case CustomMaterials feel much more light-weight.
      (I don’t want to tweak all materials but only need an additional feature for a particular material.)
    • Disadvantage: I needed to install another package (@babylonjs/materials)
    • Disadvantage: I hardly found any documentation. I searched for playground examples and RTFS.
  • And I actually figured out how to subclass StandardMaterial directly for my needs.
    • Disadvantage: Even more RTFS (and I might be relying on implementation details
      that might change in future BabylonJS releases).
    • Disadvantage: Code is a bit longer than the CustomMaterial-based solution.
    • Advantage: No need for the extra package @babylonjs/materials

I used a simpler example than in my earlier posts. Assume you have a mesh that roughly resembles the unit sphere. The material gets a uniform parameter “bulge”. When it is 0, vertices are at the mesh positions. When it is 1, vertices are normalized and thus on the unit sphere. Values in between are used for interpolation.

Here the derivation from CustomMaterials:

import {CustomMaterial} from "@babylonjs/materials";

...

class SphereMaterial extends CustomMaterial {
  constructor(name: string, scene: Scene) {
    super(name, scene);

    this.AddUniform("bulge", "float", 0.9)
    .Vertex_Before_PositionUpdated(`
      vec3 normalizedPos = normalize(position);
      positionUpdated = mix(position, normalizedPos, bulge);
    `)
    .Vertex_Before_NormalUpdated(`
      normalUpdated = mix(normal, normalizedPos, bulge);
    `);
  }

  set bulge(value: number) {
    // No idea why we have to do this incantation of  "onBindObservable".
    // But I found this in a playground example and it seems to work.
    this.onBindObservable.add(() => {
      this.getEffect().setFloat("bulge", value);
    });
  }
}

And here an implementation directly derived from StandardMaterial:


const sphereMaterialShaderName = "SphereMaterial";

Effect.ShadersStore[sphereMaterialShaderName + "VertexShader"] =
  Effect.ShadersStore["defaultVertexShader"]
  .replace("#define CUSTOM_VERTEX_DEFINITIONS", `
uniform float bulge;

$&`)
  .replace("#define CUSTOM_VERTEX_UPDATE_POSITION", `
vec3 normalizedPos = normalize(position);
positionUpdated = mix(position, normalizedPos, bulge);

$&`)
  .replace("#define CUSTOM_VERTEX_UPDATE_NORMAL", `
normalUpdated = mix(normal, normalizedPos, bulge);

$&`);

Effect.ShadersStore[sphereMaterialShaderName + "PixelShader"] = 
  Effect.ShadersStore["defaultPixelShader"];

class SphereMaterial extends StandardMaterial {
  constructor(name: string, scene: Scene) {
    super(name, scene);

    this.customShaderNameResolve = (_shaderName: string, uniforms: string[]): string => {
      uniforms.push("bulge");
      return sphereMaterialShaderName;
    }
 }

  set bulge(value: number) {
    // No idea why we have to do this incantation of  "onBindObservable".
    // But I found this in a playground example and it seems to work.
    this.onBindObservable.add(() => {
      this.getEffect().setFloat("bulge", value);
    });
  }
}

Both of these work. But here are nevertheless some questions to the experts:
@sebavan, was there a reason you did not mention CustomMaterial or did you just not think of it? And is the usage of this.customShaderNameResolve in the second example considered accessing an implementation detail? (After all, “customShaderNameResolve” does not start with an underscore!)

It is a community project from the amazing @nasimiasl but I only mentioned the one we maintain within the Babylon Team itself :slight_smile:

Normally, you can set it in your overload in the third parameter when calling super: Babylon.js/packages/dev/core/src/Meshes/linesMesh.ts at master · BabylonJS/Babylon.js · GitHub

1 Like

Thanks, @sebavan for the clarification regarding CustomMaterial.

Regarding customShaderNameResolve I must admit that I am confused and do not understand your answer.

I mean you do not need a custom shader name resolver here.

new ShaderMaterial("colorShader", this.getScene(), "color", options, false);

here the third param is the shader name and the options contains the list of uniforms. So in your case it could be:

class SphereMaterial extends StandardMaterial {
  constructor(name: string, scene: Scene) {
    super(name, scene, sphereMaterialShaderName, { 
...
});

Thanks for your reply. But the constructor of StandardMaterial only takes 2 parameters: a name and a scene. So I cannot call super(...) with more arguments.

Oh, now I think I understand where the confusion probably came from: mixing up the similar-sounding classes StandardMaterial and ShaderMaterial.

Initially I used an approach based on ShaderMaterial because that was what I had found in the docs for providing my own vertex shader. After your hints and some searching through the source code I (successfully) implemented several solutions not using ShaderMaterial:

  1. a material plugin
  2. subclassing CustomMaterial
  3. subclassing StandardMaterial directly

My question was about solution 3: “Is it OK to set this.customShaderNameResolve? Or is this an implementation detail I should not rely on?” I should have emphasized that this is about StandardMaterial, not ShaderMaterial, in particular since the latter still occurs in the thread title.

(Using ShaderMaterial would bring us back to the beginning of this thread and my playground example in the initial post. I wanted to replace my simplistic fragment shader with existing Babylon functionality properly handling diffusive and specular surfaces.)

1 Like

That now makes total sense !!! :slight_smile:

You can use it despite being at implementation detail. It should be totally fine.

1 Like

So finally everything is clarified. Thanks for your patience! :smiley:

And yours, the older I am the slower I am and I started like

Sloths GIFs - Find & Share on GIPHY

So you can imagine after a few years :slight_smile:

Can’t agree more lol