Best practice with cloning a character with different variations of clothing

[Kite]

Hello, I’m a beginner at using Babylon.js and I had a few questions hopefully someone can steer me in the right direction.

I have a character base model with various meshes that are armor, clothing, hair, etc attached to one armature/skeleton with various different animations - all in one blender file which I plan to export as one glb file.

What I am trying to achieve is possibly load the one glb file and clone or instantiate the character base model with selected armors and separate animations (but from the same glb import) to create several different characters to control/use. TL;DR - Different meshes combined but from the same glb import.

Questions
1 What would the best way/practice to do this be?

2 Since the armor/weapon/or whatever it may be is a different mesh from the character mesh and I am now cloning/instantiating them, will I have to use play() on each mesh (combined to create the one character) to play a single animation in sync with the character mesh (all meshes use the same armature and have the same animations in the glb file). I can’t help but think this will affect performance especially if I’m hoping to clone/instantiate tons of characters.

[Tricotou]

@Kite

Since the armor/weapon/or whatever it may be is a different mesh from the character mesh and I am now cloning/instantiating them, will I have to use play() on each mesh

Nope !

---

Let’s take an example from the DOC on Animating Characters :
The doc links to this Playground which loads a glb, and launch an animation group by name (“Samba”)

---

Now, when you clone a mesh, only the mesh is cloned. If you open the inspector and walk through the assets, you will see that the skeleton is still “one skeleton”, meaning any anymation launch is still one single line of code. And all meshes (cloned or not) parented to this skeleton will be animated.

---

Here is a custom example from the previous playground :
It appears that the pink short is the mesh named "HVGirl_primitive8";

• I clone it, and that’s all. Still affected by the animation.
• I give a new red material to the clone
• I setup a setInterval to show that any mesh can be toggled while anim is playing

var short1;
var short2;
var showShort1 = true;

newMeshes.forEach(function(mesh){
    if(mesh.name=="HVGirl_primitive8"){
        short1 = mesh;
        short2 = mesh.clone();
        var shortRed = new BABYLON.StandardMaterial("shortRed", scene);
        shortRed.diffuseColor = new BABYLON.Color3(1, 0, 0);
        short2.material = shortRed;
        short2.setEnabled(false);
    }
});

setInterval(function(){
    showShort1 = !showShort1;
    short1.setEnabled(showShort1);
    short2.setEnabled(!showShort1);
}, 1000);

And here we are :

Now, consider the pink and red shorts as some different weapons… You can clone, toggle them, etc… Regardless of the main character animation

Playground

---

++
Tricotou

[Kite]

Thank you for your response,

I may have miscommunicated what I was trying to achieve. I’m ultimately trying to cherry pick certain meshes from a container to instantiate (not clone) a separate model that has it’s own individual skeleton/armature

Rather than instantiating the entire container which may have hundreds of different variations but to pick a few, instantiate them with a instance of the skeleton to play it’s own animation.

Is this possible?

[Joe_Kerr]

@Kite

2 Since the armor/weapon/or whatever it may be is a different mesh from the character mesh and I am now cloning/instantiating them, will I have to use play() on each mesh (combined to create the one character) to play a single animation in sync with the character mesh (all meshes use the same armature and have the same animations in the glb file). I can’t help but think this will affect performance especially if I’m hoping to clone/instantiate tons of characters.

If you use palette textures or use small enough painted textures that can fit into an atlas, you can as well merge all meshes into one. (1 draw call!) Not sure about performance when dynamically changing equpiment (instead of once at startup).

Also beware: do make stress tests. Can you get high enough FPS with the max number of characters? If not, you might need VATs.

@Kite

1 What would the best way/practice to do this be?

I have used this as a baseline: Loading character with animations to scene and cloning it with animation groups - #9 by Panuchka

[Kite]

I’m so glad you brought that up! I did run a stress test with about 50 characters (2.5k tris per character total, 4-6 meshes each) and the CPU was at 99% on a high spec alienware. Is this because of the animations? or the draw calls? maybe both? Absolute FPS is still relatively high according to the inspector but does drop from 1800 to 300-500… 50 characters is something that I wouldn’t want to worry about on one screen together…

So I began working on merging the meshes together. I’ve managed to merge the meshes together but can’t for the life of me figure out how to apply a skeleton to it and now I’m learning that there are VATs but the subject seems so complex to me at the moment.

[Kite]

So I’ve been looking around into VAT more… I really don’t want to lose out on the performance as I’m trying to build a multiplayer game.

There are a few things I’m curious about:

1. VATs are pre-baked meaning each vertex location of the mesh is stored per animation frame? So does that mean I would have to compute each VAT per combination of merged meshes?! ex: character base & hair = 1 vat, character base & shirt = 1 vat. Please correct me if my understand of VAT is completely off. The documentation doesn’t really say much and it’s not coming easy to me.

2. How does it compute the VAT? Does it use BOTH AnimationGroups AND Skeleton?

Please bear with me, I’m totally new to all this.

[Kite]

retargetAnimationGroup(animationGroup: any, targetSkeleton: any){
        animationGroup.targetedAnimations.forEach((targetedAnimation: any) => {
                const newTargetBone = targetSkeleton.bones.filter((bone: any)=>{return bone.name === targetedAnimation.target.name})[0]
            targetedAnimation.target = newTargetBone? newTargetBone.getTransformNode() : null
        })
    }

    createEntity(entity: any) {
        
        let allChildMeshes =  this.assets.base.meshes[0].getChildMeshes(true)
        let merged = BABYLON.Mesh.MergeMeshes(allChildMeshes.filter((m: any) => m.geometry), false, true, undefined, false, false)
        if(merged){
            merged.name = 'character' + entity.nid.toString()
            entity.container =  new BABYLON.AssetContainer(this.scene)
            entity.container.meshes.push(merged)
            entity.container.skeletons.push(this.assets.base.skeletons[0].clone('skeleton' + entity.nid.toString()))
            entity.container.meshes[0].skeleton = entity.container.skeletons[0]
            entity.container.skeletons[0].overrideMesh = entity.container.meshes[0]
            
            this.assets.base.animationGroups.forEach((animationGroup: any) => {
                let clone = animationGroup.clone()
                entity.container.animationGroups.push(clone)
            })

            entity.container.animationGroups.forEach((animationGroup: any) =>  {
                this.retargetAnimationGroup(animationGroup, entity.container.skeletons[0])
                animationGroup.stop()
            })
            
           entity.container.meshes[0].parent = entity.mesh
           entity.container.animationGroups[Math.round(Math.random() * 5)].play(true)

        }

Though I’m cloning and retargeting bones, the animations are still not separate on merged mesh (merged to reduce draw calls) Could anyone please help?

[BitOfGold]

This thread is about a similar problem (baking animation groups into VAT), and attaching weapons to characters.

Baked Texture Animations with Animation Groups?

Hello, I’ve tried working with the Baked Texture Animations feature in 5.0, and there doesn’t seem to be a way to bake Skeleton animations with Animation Groups (coming for example in a GLB file from Blender). There seems to be some discussions around the topic, and all the examples on the docs and in the playgrounds use the babylon files to load vertex animations, but don’t specify what shape the animations need to have in those files (I’m a noob on the subject as well ). How woul…

I still not say I understand and use it (I also try to bake animations) but maybe starts you on a good direction!

[Kite]

Sorry for the spam.
I tried another way that worked: Instead of cloning everything separately, I used instantiateModelsToScene, mergedMeshes, and then applied skeleton to the mergedMesh.

It was much simpler than I thought

[Joe_Kerr]

@Kite

I tried another way that worked: Instead of cloning everything separately, I used instantiateModelsToScene, mergedMeshes, and then applied skeleton to the mergedMesh.

Sorry for the delayed reponse.

re: VATs.

I have not used VATs myself yet, partly because blending does nt work (so easy) anymore and it is a rather big change in my assets workflow. But I know that @oriongu did some pioneering work over here (just browse the thread backwards).

@Kite

Is this because of the animations? or the draw calls?

Can you load the character models without animations? I think this would be an important performance test and the safest thing to answer the question (more reliable than stats I mean).

---

One last thing: as far as I could read, the number of bones is what slows down animations. If by chance you use Mixamo animations, notice that half of the bone count is due to the fingers! If you do not need detailed finger movement, best case would be to just delete the finger bones in Blender (or whatever you use). I would hope that this gets a big performance boost. I have not tried this but it sounds like less work than using VATs.

[oriongu]

Hi Kite,

I can see what you trying to achieve, this was my goal at the start too but project has changed/evolved since.

What I’m doing now is I run 1 VAT per “class” (mage, knight, rogue) and all the equipment is then attached dynamically to the monsters/players (and to the vat of course). For each class variation (red rogue/black rogue/yellow rogue for example), I clone the base mesh and apply the texture, attach it to the right VAT and then instantiated it accordingly. This allows for a fair amount of variety.

In my current project, I only ever have 3 vat running and performance seems to be good. I have not tried running 20 vat at the same so do not know what impact that has.

Would this work for your case where you want different clothing, hair, armor, etc… I don’t see why not. The trick would be to manage you bases meshes, merge whatever meshes should be merged, texture accordingly, attach to the right VAT and finally instantiate.

The tricky part is thing like the armor parts, as attaching to a bone may not work very well visually, possibly these may need to be rigged and added in Blender and show/hide relevant meshes accordingly.

You can check these part of my project for more details:

• t5c/src/client/Controllers/VatController.ts at main · orion3dgames/t5c · GitHub
  • check the prepareMesh function for how I prepare the class base meshes
• t5c/src/client/Entities/Entity/EntityMesh.ts at main · orion3dgames/t5c · GitHub
  • and here you can see how I actually instantiate the mesh and attach the items
• t5c/src/client/Entities/Entity/EntityAnimator.ts at main · orion3dgames/t5c · GitHub
  • this part deal with the animations, it’s looks complex but it really isn’t once you get into it.

This is by no means a perfect solution but it does work. Any suggestions are more than welcome,

You could also maybe ask @Alexander_Sosnovskiy as I think he does it in a different way and probably have some great advice on this matter.

Keep the questions coming,

Cheers,
Orion

[Alexander_Sosnovskiy]

Hey!

If you are planning to support 50+ characters then yes, you definitely need to use VAT.

You can use a single Skeleton for any number of meshes.

For example, in my project, I have “Humanoid” skeleton and use it for Human, Orc, Golem etc. You can even use it for “Ghosts” - creatures without legs or hands and its ok.

I have similar animations for them, so I can reuse VAT too. So I have a bunch of animations and play it accordingly to the current mesh.

About VAT

• baking of animations is a separate process. You have to do it manually.
• to gain maximum performance, you should limit a number of bones per vertex (Mesh.numBoneInfluencers) and total bones per skeleton. Less bones, more performance(but be careful and test you animations for glitches). I use 2 bones per vertex and 18 bones per humanoid skeleton.

Number of bones per vertex = number of texture fetches in Vertex Shader x 4
So if you use 2bones per vertex you have 8 texture fetches in Vertex Shader.

Number of bones per skeleton = VAT’s width / 4 - 1
So if you have 18 bones, you have texture with width = (18 + 1) * 4 = 76

total number of frames in animations = VAT’s height.

VAT’s size is matter because it uses RGBA32F, so per each pixel you need 16bytes of memory.

About customisation

I have a similar number of verteces per character(3500) so I decided to merge all parts together and show/hide it via VertexShader. Each part contains partId as a vertex’s attribute. And there is a separate data texture with width=num of parts and height=num of characters. So each pixel contains info about show/hide or which texture will be applied to this part.

I test it with 200 characters and it works good.

PS VAT should be named BAT(bones animation texture, because the texture contains bones transforms for each animation’s frame, not the vertex)

(240 enemies, 6 different skeletons via VAT)

[Kite]

Thank you for such a detailed response. I’m still wrapping my head on how it runs so please bear with me

1. By baking of animations - does this mean to generate a texture?

2. Bones per vertex… does this mean how many bones can affect a single vertex?

3. My previous understanding was that a texture was generated and saved → loaded and was somehow read and moved verteces. But it’s actually the bones… and it’s loaded to influence the verteces? So if that’s the case couldn’t one VAT affect two different meshes that similar but has different verteces?

[Alexander_Sosnovskiy]

1. You are right! Something like this Baked Texture Animations with Animation Groups? - #11 by Evgeni_Popov

2. Yeap

3. As I mentioned in my prev post, you can use the texture(from VAT) for any meshes as long as they use the same skeleton. So it will work for Hat, Armor, Boots, Body, etc parts if they all rigged to the same skeleton.

When you “bake” texture, you run animations for your Skeleton. So you only need the Skeleton and any mesh(because without mesh it doesn’t work AFAIK)

[Panuchka]

I’m currently trying to do my game without any skeletal animations Just using Babylon animation editor to animate the bodies of players/whatever and their hands with just rotation/scaling/transform, has been working pretty well (and performantly) so far With many animated skeletal entities you might need something like vats though. Last time I checked it was pretty hard to do the animation blending with these, is that resolved somehow?