[Lite] Custom wgsl shaders for both regular and thin instanced meshes

Questions
,
1

Looking to port some code from regular babylonjs to babylon lite. In regular babylonjs, the way I understood to add instance matrix support into a wgsl shader is with the following:
#include<instancesDeclaration>
@vertex fn main(input:VertexInputs) -> FragmentInputs {
#include instancesVertex
let worldPos = finalWorld * vec4f(vertexInputs.position, 1.0);

finalWorld then represents the world matrix, or the world * instance matrix.

This allows the same material to be applied to both regular meshes, as well as instanced meshes.

Looking at babylon lite, I don’t see any equivalent to this functionality. I read through the documentation on github, and I see there is mentions of automatically injecting world0, world1, etc. for thin instance meshes, but then the shader does not compile for non-thin instanced meshes.

Is there a way to use the same shader for both types of meshes, or is it now required to maintain 2 different shader sources for this?

2

@Evgeni_Popov might be able to help on Monday

3

You are right that directly referencing world0 through world3 makes the WGSL specific to thin-instanced meshes.

I opened PR #674, which adds an opt-in helper allowing the same vertex shader to support both regular and thin-instanced meshes:

const material = createShaderMaterial({
    vertexSource,
    fragmentSource,
    attributes: ["position"],
    uniforms: ["world", "viewProjection"],
});

enableShaderMaterialInstanceWorld(material);

The vertex shader can then use:

let finalWorld = getFinalWorld(input);
let worldPos = finalWorld * vec4f(input.position, 1.0);

For a regular mesh, getFinalWorld(input) returns the mesh world matrix. For a thin-instanced mesh, it returns the mesh world matrix multiplied by the instance matrix.

The feature is opt-in, so scenes that do not enable it have no bundle-size increase. Focused bundle measurements for both a regular scene and a ShaderMaterial scene showed a 0-byte delta against public master.

Until the PR is merged and released, two shader variants are still required.

4

This would be fantastic. The only question I would have with this; can the same shader material object returned from the createShaderMaterial function work on both regular and thin-instanced meshes simultaneously? Will I need to create 2 shadermaterials, and only enable the instance world on one of the variants? I suspect the latter, which is different to how regular babylon worked. Fine if that’s the case, just want to confirm my suspicions.

If possible, reusing the same shadermaterial that is returned for both would be preferable. Similar to createStandardMaterial and apply that material to all meshes.

5

Yes, the same ShaderMaterial object can be used simultaneously by regular and thin-instanced meshes. You only create and enable it once:

const material = createShaderMaterial({
    vertexSource,
    fragmentSource,
    attributes: ["position"],
    uniforms: ["world", "viewProjection"],
});

enableShaderMaterialInstanceWorld(material);

You can then assign material to both kinds of mesh. Babylon Lite creates the appropriate GPU pipeline variant internally:

  • Regular mesh: getFinalWorld(input) returns shaderSystem.world.
  • Thin-instanced mesh: it returns shaderSystem.world * instanceWorld.

The material, shader source, uniforms, textures, and other state remain shared—there is no need to create two ShaderMaterial objects.

The only caveat is that the common shader must not directly reference thin-instance-only fields such as input.world0 or input.instanceColor. Use getFinalWorld(input) for the world transform; instanceColor still requires an instanced-specific shader path if you need it.

6

The only caveat is that the common shader must not directly reference thin-instance-only fields such as input.world0 or input.instanceColor. Use getFinalWorld(input) for the world transform; instanceColor still requires an instanced-specific shader path if you need it.

Does this mean if I want to have access to a per instance color, there will need to be a similar opt in feature to getFinalWorld for getFinalColor or something? I hadn’t looked much into the color yet because I was so focused on the position, but I assume the input.color property doesn’t contain the color for thin instances? Previously in regular babylon, I was used to just using input.color directly, rather than requiring resolving it between regular / instanced meshes.

7

Yes, that is correct: in Babylon Lite, input.color and input.instanceColor are two different attributes.

input.color is the mesh’s ordinary per-vertex color, requested through:

attributes: ["position", "color"]

It does not contain the color assigned with setThinInstanceColors(). That data is instance-rate data and is exposed separately as input.instanceColor only in a thin-instanced pipeline variant that has an instance-color buffer.

Therefore, with the API currently merged in PR #674, a shader that directly references input.instanceColor cannot also be used by a regular mesh.

An analogous opt-in color helper would be the appropriate solution. It should expose one stable function to the shared WGSL and specialize its implementation for each pipeline variant. The useful semantics would likely be an effective color:

  • ordinary vertex color when present;
  • white when there is no color source;
  • thin-instance color when present;
  • vertex color multiplied by thin-instance color when both are present.

That matches the usual Babylon color-composition behavior and would let one shader source use something like getFinalColor(input) without making instanceColor part of every ShaderMaterial or increasing bundles that do not opt in.

This color helper is not part of PR #674 yet; that PR only added getFinalWorld(input).

[EDIT] New PR to add getFinalColor:

8

Wanted to start off saying the position update works fantastic. It’s working exactly as expected.

The color one, still having some issues with. Before these changes, the color was appearing black for thin instanced meshes when using the same shader. All thin instance meshes were also going to world origin, so I was able to see all of them were black.
The way the code is set up for the color MR would be multiplying the input.color by the input.instanceColor, which seems fine but ultimately multiplies the desired color by black, as the shader also includes the color property for non thin instance meshes. Because of that, even when trying to use the getFinalColor it appears black for all thin instanced meshes. I could be doing something wrong with actually setting the colors on the material, but that’s what I observe as of now.

Edit: Got around this by just filling the vertex color with white for instanced meshes.

9

You are not doing anything wrong — this is a bug in the current fallback behavior.

When the material declares:

attributes: ["position", "color"]

the generated thin-instance implementation correctly uses:

input.color * input.instanceColor

However, if that thin-instanced mesh has no vertex-color buffer, Babylon Lite currently binds a zero-filled fallback for input.color. Multiplying the valid instance color by (0, 0, 0, 0) makes the result black.

Your workaround of supplying white vertex colors is therefore exactly the correct neutral value: multiplying by (1, 1, 1, 1) leaves the instance color unchanged.

I have prepared a fix so that, for a ShaderMaterial using enableShaderMaterialFinalColor(), a missing vertex-color buffer uses a white fallback instead:

  • vertex color only: returns the vertex color;
  • instance color only: returns the instance color;
  • both: multiplies them;
  • neither: returns white.

This remains opt-in and does not add bundle cost to ShaderMaterial scenes that do not use the helper.

Until that fix is merged and released, please keep the white vertex-color workaround.