Hello! I need to render about 50 characters with animations.
My first attempt was using InstanedMesh + VAT. It works great for a single variant(just eyes+body)
Then I wanted to add Hats, Pants and so on.
My plan was using single vertices data and then via vertex color manipulate each instance(transparent - disable, any other - enable). But is doesn’t work AFAIK Using .glb model with instantiation and vertex colors - #7 by Evgeni_Popov
Any ideas how to achieve my goal?
All my NPCs use same animations and skeleton + geometry. All I need is just enable/disable/colorize some vertices.
If hat, pant, … are separate meshes, then you could use the instance color to colorize each one separately. If you have a single mesh for all the eyes+body+hat+pant parts and you want to colorize some parts differently than others, then you will have to resort to what the PG you linked is doing.
Thanks.
Yes, I use one mesh(merged 25 meshes) to decrease draw calls.
If I understand correctly, I need a big texture to store VERT_COUNT*INST_COUNT pixels, right?
Yes if you want to be able to give a different color to each vertex. But if you only want to give a different color to a range of vertices, you can compact things a lot.
For eg, say you want to colorize only the eyes and the pants. If the vertex ids for the eyes are in the [v0,v1] range and for the pants it is the [v2,v3] range, then in your vertex shader you can check the current vertex_id against those two ranges and select the color for the eyes from texture(vec2(0, instanceId)) and the color for the pants from texture(vec2(1, instanceId)). In this case, you only need a texture with width=number of different parts and height=number of instances (you could also have the colors for more than one instance in the X dimension, but it’s a little more calculation to get the right color).
1 Like
You are the best! Thanks! I’ll try and comeback with the playground)
So I only need to manually provide vertices ranges, right?
One issue which occurs for me - if your instances count is dynamically changed, you can’t use gl_InstanceID because it will change.
I decided to use color.r channel for each instance to provide “ID”(the row) in the texture.
So, my code looks like:
//meshes - array of meshes to be merged. In my case it is a loot's part like Pant, Helm, Hat etc.
//merged - mesh which you want to instantiate
//pixelData - Uint8Array
const mat = new BABYLON.CustomMaterial('customMaterial', scene);
const numInstances = 64; //MAX INSTANCES
const numVerticesPerInstance = merged.getTotalVertices();
const width = meshes.length;
const height = numInstances;
const vertColors: number[] = [];
for (let i = 0; i < numVerticesPerInstance; i++) {
vertColors.push(0, 0, 0, 0);
}
merged.registerInstancedBuffer(BABYLON.VertexBuffer.ColorKind, 4);
//r channel is 0 because it is a first "object".
//For new instancedMesh we should manually set it to incremental value like 1, 2, etc.
merged.instancedBuffers.color = new BABYLON.Color4(0, 0, 0, 1);
merged.setVerticesData(BABYLON.VertexBuffer.ColorKind, vertColors);
const texVertColors = BABYLON.RawTexture.CreateRGBATexture(
pixelData,
width,
height,
scene,
false,
false,
BABYLON.Constants.TEXTURE_NEAREST_SAMPLINGMODE,
BABYLON.Constants.TEXTURETYPE_UNSIGNED_BYTE
);
mat.AddUniform('texVertColors', 'sampler2D', texVertColors);
let vInd = 0;
const rangesStr = `${meshes.reduce((acc, mesh, i) => {
if (i === 0) {
vInd += mesh.getTotalVertices();
return acc;
}
let str = `(step(${vInd}.0,vId)-step(${vInd + mesh.getTotalVertices()}.0,vId))*${i}.0`;
if (i < meshes.length - 1) {
str += '+';
}
vInd += mesh.getTotalVertices();
return acc + str;
}, '')}`;
mat.Vertex_MainEnd(`
float vId = float(gl_VertexID);
float partId = ${rangesStr};
float instanceId = color.r;
vColor = texelFetch(texVertColors, ivec2(partId, instanceId), 0);
`);
// here, in ivec2 we use partId and instanceId (from color.r)
I have one issue with such approach.
Lets see:
This is forward side, seems ok, except pants.
This is backward side, and we see mustache, but we shouldn’t)
Why?
I use alpha-testing/blending like this:
const mat = new BABYLON.CustomMaterial('CharacterMaterial', scene);
(mat as BABYLON.StandardMaterial).transparencyMode = 3;
mat.needAlphaTesting = () => true;
mat.specularColor = BABYLON.Color3.White().scaleInPlace(0.05);
merged.material = mat;
merged.useVertexColors = true;
merged.hasVertexAlpha = true;
- the code above for shader’s part.
I think I don’t understand how it works together, however I had try to read like 3-4 times about depth buffers, alpha testing and so on.
All my parts are opaque meshes, so I thought I can just use opaque material, but …
const mat = new BABYLON.CustomMaterial('CharacterMaterial', scene);
(mat as BABYLON.StandardMaterial).transparencyMode = 0; // <-- set to opaque
// mat.needAlphaTesting = () => true; // disable this
mat.specularColor = BABYLON.Color3.White().scaleInPlace(0.05);
merged.material = mat;
// merged.useVertexColors = true; // disable this
// merged.hasVertexAlpha = true; // and this
Result:
Ok. How it works?(my assumption)
- we set
vColor for each vertex based on texture. For some of them we use alpha=0, because we want to disable it.
- but, as we don’t use alpha testing, we get the black vertex(because for disabling I use
Color4(0,0,0,0).
Is it right?
If so, may I solve it with opaque material?
If not, how to fix issue from the beginning?)
Ok, I think I’m man of the night, because 2am and I found a good solution - reset y pos for vertex based on alpha:
mat.Vertex_MainEnd(`
float vId = float(gl_VertexID);
float partId = ${rangesStr};
float instanceId = color.r;
vColor = texelFetch(texVertColors, ivec2(partId, instanceId), 0);
gl_Position.y = vColor.w * gl_Position.y;
`);
@Evgeni_Popov Is it optimal and safe way to “discard” vertex from screen?
I’m not sure to understand everything, a repro would help!
However, having transparent objects is always a problem to deal with (Transparent Rendering | Babylon.js Documentation).
As I can see it, your character does not need transparency, so you should avoid having transparent materials: you see the mustache in the backward side because it is drawn last, or at least after the head. Because transparent objects don’t write to the depth buffer, the mustache won’t be occluded by the head.
1 Like
Thank you! What you think about gl_Position.y = vColor.w * gl_Position.y; hack?)
You should not have to use these hacks if all your meshes are opaque and your materials are not transparent.
we set vColor for each vertex based on texture. For some of them we use alpha=0, because we want to disable it.
Why do you need to do this? What do you mean by “we want to disable it”?
I have one mesh and for each vertex I set a color(all my “clothes” + body + eyes is a single mesh)
The vertex doesn’t really transparent, because I use alpha channel for manipulating like this gl_Position.y = vColor.w * gl_Position.y;
Ok, so having not transparent materials should work and gl_Position.y = vColor.w * gl_Position.y; looks ok to me.
1 Like