# Customizable characters optimization

**URL:** <https://forum.babylonjs.com/t/customizable-characters-optimization/39378>\
**Category:** Questions\
**Created:** [March 27, 2023, 12:12pm UTC](https://forum.babylonjs.com/t/customizable-characters-optimization/39378 "2023-03-27T12:12:21Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alexander\_Sosnovskiy](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/alexander_sosnovskiy/32/30165_2.png) [@Alexander\_Sosnovskiy](https://forum.babylonjs.com/u/Alexander_Sosnovskiy)\
**Post date:** [March 27, 2023, 12:12pm UTC](https://forum.babylonjs.com/t/customizable-characters-optimization/39378/1 "2023-03-27T12:12:21Z")

</div>

Hello! I need to render about 50 characters with animations.

My first attempt was using InstanedMesh + VAT. It works great for a single variant(just eyes+body)

Then I wanted to add Hats, Pants and so on.

My plan was using single vertices data and then via vertex color manipulate each instance(transparent - disable, any other - enable). But is doesn’t work AFAIK [Using .glb model with instantiation and vertex colors - #7 by Evgeni\_Popov](https://forum.babylonjs.com/t/using-glb-model-with-instantiation-and-vertex-colors/22968/7)

Any ideas how to achieve my goal?

All my NPCs use same animations and skeleton + geometry. All I need is just enable/disable/colorize some vertices.

---

<div class="post-metadata">

**Author:** ![Evgeni\_Popov](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/evgeni_popov/32/4432_2.png) [@Evgeni\_Popov](https://forum.babylonjs.com/u/Evgeni_Popov)\
**Post date:** [March 27, 2023, 1:03pm UTC](https://forum.babylonjs.com/t/customizable-characters-optimization/39378/2 "2023-03-27T13:03:22Z")

</div>

If hat, pant, … are separate meshes, then you could use the instance color to colorize each one separately. If you have a single mesh for all the eyes+body+hat+pant parts and you want to colorize some parts differently than others, then you will have to resort to what the PG you linked is doing.

---

<div class="post-metadata">

**Author:** ![Alexander\_Sosnovskiy](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/alexander_sosnovskiy/32/30165_2.png) [@Alexander\_Sosnovskiy](https://forum.babylonjs.com/u/Alexander_Sosnovskiy)\
**Post date:** [March 27, 2023, 3:07pm UTC](https://forum.babylonjs.com/t/customizable-characters-optimization/39378/3 "2023-03-27T15:07:14Z")

</div>

Thanks.

Yes, I use one mesh(merged 25 meshes) to decrease draw calls.

If I understand correctly, I need a big texture to store VERT\_COUNT\*INST\_COUNT pixels, right?

---

<div class="post-metadata">

**Author:** ![Evgeni\_Popov](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/evgeni_popov/32/4432_2.png) [@Evgeni\_Popov](https://forum.babylonjs.com/u/Evgeni_Popov)\
**Post date:** [March 27, 2023, 3:53pm UTC](https://forum.babylonjs.com/t/customizable-characters-optimization/39378/4 "2023-03-27T15:53:48Z")

</div>

Yes if you want to be able to give a different color to each vertex. But if you only want to give a different color to a range of vertices, you can compact things a lot.

For eg, say you want to colorize only the eyes and the pants. If the vertex ids for the eyes are in the [v0,v1] range and for the pants it is the [v2,v3] range, then in your vertex shader you can check the current vertex\_id against those two ranges and select the color for the eyes from `texture(vec2(0, instanceId))` and the color for the pants from `texture(vec2(1, instanceId))`. In this case, you only need a texture with width=number of different parts and height=number of instances (you could also have the colors for more than one instance in the X dimension, but it’s a little more calculation to get the right color).

---

<div class="post-metadata">

**Author:** ![Alexander\_Sosnovskiy](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/alexander_sosnovskiy/32/30165_2.png) [@Alexander\_Sosnovskiy](https://forum.babylonjs.com/u/Alexander_Sosnovskiy)\
**Post date:** [March 27, 2023, 3:55pm UTC](https://forum.babylonjs.com/t/customizable-characters-optimization/39378/5 "2023-03-27T15:55:32Z")

</div>

You are the best! Thanks! I’ll try and comeback with the playground)

So I only need to manually provide vertices ranges, right?

---

<div class="post-metadata">

**Author:** ![Alexander\_Sosnovskiy](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/alexander_sosnovskiy/32/30165_2.png) [@Alexander\_Sosnovskiy](https://forum.babylonjs.com/u/Alexander_Sosnovskiy)\
**Post date:** [March 28, 2023, 2:33pm UTC](https://forum.babylonjs.com/t/customizable-characters-optimization/39378/6 "2023-03-28T14:33:57Z")

</div>

One issue which occurs for me - if your instances count is dynamically changed, you can’t use `gl_InstanceID` because it will change.

I decided to use color.r channel for each instance to provide “ID”(the row) in the texture.

So, my code looks like:

```auto
//meshes - array of meshes to be merged. In my case it is a loot's part like Pant, Helm, Hat etc.
//merged - mesh which you want to instantiate
//pixelData - Uint8Array

          const mat = new BABYLON.CustomMaterial('customMaterial', scene);
          const numInstances = 64; //MAX INSTANCES
          const numVerticesPerInstance = merged.getTotalVertices();
          const width = meshes.length;
          const height = numInstances;

          const vertColors: number[] = [];
          for (let i = 0; i < numVerticesPerInstance; i++) {
            vertColors.push(0, 0, 0, 0);
          }

          merged.registerInstancedBuffer(BABYLON.VertexBuffer.ColorKind, 4);
//r channel is 0 because it is a first "object".
//For new instancedMesh we should manually set it to incremental value like 1, 2, etc.
          merged.instancedBuffers.color = new BABYLON.Color4(0, 0, 0, 1);
          merged.setVerticesData(BABYLON.VertexBuffer.ColorKind, vertColors);

          const texVertColors = BABYLON.RawTexture.CreateRGBATexture(
            pixelData,
            width,
            height,
            scene,
            false,
            false,
            BABYLON.Constants.TEXTURE_NEAREST_SAMPLINGMODE,
            BABYLON.Constants.TEXTURETYPE_UNSIGNED_BYTE
          );

          mat.AddUniform('texVertColors', 'sampler2D', texVertColors);

          let vInd = 0;
          const rangesStr = `${meshes.reduce((acc, mesh, i) => {
            if (i === 0) {
              vInd += mesh.getTotalVertices();
              return acc;
            }

            let str = `(step(${vInd}.0,vId)-step(${vInd + mesh.getTotalVertices()}.0,vId))*${i}.0`;

            if (i < meshes.length - 1) {
              str += '+';
            }
            vInd += mesh.getTotalVertices();

            return acc + str;
          }, '')}`;

          mat.Vertex_MainEnd(`
          float vId = float(gl_VertexID);
          float partId = ${rangesStr};
          float instanceId = color.r;
          vColor = texelFetch(texVertColors, ivec2(partId, instanceId), 0);
          `);
// here, in ivec2 we use partId and instanceId (from color.r)

```

---

<div class="post-metadata">

**Author:** ![Alexander\_Sosnovskiy](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/alexander_sosnovskiy/32/30165_2.png) [@Alexander\_Sosnovskiy](https://forum.babylonjs.com/u/Alexander_Sosnovskiy)\
**Post date:** [March 28, 2023, 10:22pm UTC](https://forum.babylonjs.com/t/customizable-characters-optimization/39378/7 "2023-03-28T22:22:32Z")

</div>

I have one issue with such approach.

Lets see:

This is forward side, seems ok, except pants.  
 ![Screenshot 2023-03-29 at 01.09.03](https://us1.discourse-cdn.com/flex024/uploads/babylonjs/original/3X/d/c/dcfd157c7a2699fbd48ac5b700f532baa2c2d54e.png)

This is backward side, and we see mustache, but we shouldn’t)  
 ![Screenshot 2023-03-29 at 01.08.58](https://us1.discourse-cdn.com/flex024/uploads/babylonjs/original/3X/6/0/6064bcf7acec73dd08ec6ba1c783a9a55564bd3e.png)

Why?

I use alpha-testing/blending like this:

```auto
const mat = new BABYLON.CustomMaterial('CharacterMaterial', scene);
 (mat as BABYLON.StandardMaterial).transparencyMode = 3;
mat.needAlphaTesting = () => true;
mat.specularColor = BABYLON.Color3.White().scaleInPlace(0.05);

merged.material = mat;
merged.useVertexColors = true;
merged.hasVertexAlpha = true;

```

- the code above for shader’s part.

I think I don’t understand how it works together, however I had try to read like 3-4 times about depth buffers, alpha testing and so on.

All my parts are opaque meshes, so I thought I can just use opaque material, but …

```auto
const mat = new BABYLON.CustomMaterial('CharacterMaterial', scene);
(mat as BABYLON.StandardMaterial).transparencyMode = 0; // <-- set to opaque
// mat.needAlphaTesting = () => true; // disable this
mat.specularColor = BABYLON.Color3.White().scaleInPlace(0.05);

merged.material = mat;
// merged.useVertexColors = true; // disable this
// merged.hasVertexAlpha = true; // and this

```

**Result:**  
 ![Screenshot 2023-03-29 at 01.16.56](https://us1.discourse-cdn.com/flex024/uploads/babylonjs/original/3X/1/3/134291aaa7dc4299fc10d26cd245a042f353d251.png)

Ok. How it works?(my _assumption_)

- we set `vColor` for each vertex based on texture. For some of them we use alpha=0, because we want to disable it.
- but, as we don’t use alpha testing, we get the black vertex(because for disabling I use `Color4(0,0,0,0)`.

Is it right?

If so, may I solve it with opaque material?

If not, how to fix issue from the beginning?)

---

<div class="post-metadata">

**Author:** ![Alexander\_Sosnovskiy](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/alexander_sosnovskiy/32/30165_2.png) [@Alexander\_Sosnovskiy](https://forum.babylonjs.com/u/Alexander_Sosnovskiy)\
**Post date:** [March 28, 2023, 10:45pm UTC](https://forum.babylonjs.com/t/customizable-characters-optimization/39378/8 "2023-03-28T22:45:27Z")

</div>

Ok, I think I’m man of the night, because 2am and I found a good solution - reset y pos for vertex based on alpha:

```auto
          mat.Vertex_MainEnd(`
          float vId = float(gl_VertexID);
          float partId = ${rangesStr};
          float instanceId = color.r;
          vColor = texelFetch(texVertColors, ivec2(partId, instanceId), 0);
          gl_Position.y = vColor.w * gl_Position.y;
          `);

```

@Evgeni_Popov Is it optimal and safe way to “discard” vertex from screen?

---

<div class="post-metadata">

**Author:** ![Evgeni\_Popov](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/evgeni_popov/32/4432_2.png) [@Evgeni\_Popov](https://forum.babylonjs.com/u/Evgeni_Popov)\
**Post date:** [March 29, 2023, 9:28am UTC](https://forum.babylonjs.com/t/customizable-characters-optimization/39378/9 "2023-03-29T09:28:24Z")

</div>

I’m not sure to understand everything, a repro would help!

However, having transparent objects is always a problem to deal with ([Transparent Rendering | Babylon.js Documentation](https://doc.babylonjs.com/features/featuresDeepDive/materials/advanced/transparent_rendering)).

As I can see it, your character does not need transparency, so you should avoid having transparent materials: you see the mustache in the backward side because it is drawn last, or at least after the head. Because transparent objects don’t write to the depth buffer, the mustache won’t be occluded by the head.

---

<div class="post-metadata">

**Author:** ![Alexander\_Sosnovskiy](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/alexander_sosnovskiy/32/30165_2.png) [@Alexander\_Sosnovskiy](https://forum.babylonjs.com/u/Alexander_Sosnovskiy)\
**Post date:** [March 29, 2023, 9:58am UTC](https://forum.babylonjs.com/t/customizable-characters-optimization/39378/10 "2023-03-29T09:58:18Z")

</div>

Thank you! What you think about `gl_Position.y = vColor.w * gl_Position.y;` hack?)

---

<div class="post-metadata">

**Author:** ![Evgeni\_Popov](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/evgeni_popov/32/4432_2.png) [@Evgeni\_Popov](https://forum.babylonjs.com/u/Evgeni_Popov)\
**Post date:** [March 29, 2023, 10:27am UTC](https://forum.babylonjs.com/t/customizable-characters-optimization/39378/11 "2023-03-29T10:27:17Z")

</div>

You should not have to use these hacks if all your meshes are opaque and your materials are not transparent.

> we set `vColor` for each vertex based on texture. For some of them we use alpha=0, because we want to disable it.

Why do you need to do this? What do you mean by “we want to disable it”?

---

<div class="post-metadata">

**Author:** ![Alexander\_Sosnovskiy](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/alexander_sosnovskiy/32/30165_2.png) [@Alexander\_Sosnovskiy](https://forum.babylonjs.com/u/Alexander_Sosnovskiy)\
**Post date:** [March 29, 2023, 11:03am UTC](https://forum.babylonjs.com/t/customizable-characters-optimization/39378/12 "2023-03-29T11:03:28Z")

</div>

> [@Alexander\_Sosnovskiy](#):
>
> My plan was using single vertices data and then via vertex color manipulate each instance(transparent - disable, any other - enable)

I have one mesh and for each vertex I set a color(all my “clothes” + body + eyes is a single mesh)

The vertex doesn’t really transparent, because I use alpha channel for manipulating like this `gl_Position.y = vColor.w * gl_Position.y;`

---

<div class="post-metadata">

**Author:** ![Evgeni\_Popov](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/evgeni_popov/32/4432_2.png) [@Evgeni\_Popov](https://forum.babylonjs.com/u/Evgeni_Popov)\
**Post date:** [March 29, 2023, 11:15am UTC](https://forum.babylonjs.com/t/customizable-characters-optimization/39378/13 "2023-03-29T11:15:59Z")

</div>

Ok, so having not transparent materials should work and `gl_Position.y = vColor.w * gl_Position.y;` looks ok to me.