Applying bone weights causes mesh geometry to unpredictably "explode"

[rdw1]

Hello!

I’ve been learning about skeletal animation and while trying to implement it in Babylon, I’m facing a strange issue: I was able to create a skeleton and assign it to some meshes, but as soon as I do this the geometry seems to “explode” (for lack of a better word).

Here is a Playground to demonstrate what I mean: After 5 seconds it will begin the animation, but you can disable this to see just the mesh geometry on its own (it looks right).

As a side note, I wasn’t able to import all the animations so it’s expected to look a little bit off, but not THAT much. For me locally it looks very similar; here’s a short video showing both my rendition and how it is supposed to look when animated. The same “repositioning” problem is still present, however, and I’m unsure what’s causing it.

As soon as skeletons are disabled, the mesh looks “almost right” so it must be the bone influences messing it up. Since the skeleton and bone animation itself looks correct, the geometry must be mapped incorrectly or use the bone weights wrongly perhaps. The values themselves should be correct, as I have verified them manually (head attaches to head bone etc, checking boneWeights with the inspector), and they were also exported as-is on my end.

My guess is that the bone’s initial position isn’t filtered: If the “head” geometry is attached to the “head” bone (second from the top), I would expect the geometry to move only very slightly if that is the only bone influence. Instead, the individual parts move all over the place.

I’ve spent several days trying to find the root cause of this issue, but I can’t seem to get anywhere. It feels like I don’t understand enough about how the skeleton/bone/mesh interaction is supposed to work in BJS, since it clearly doesn’t quite do what I expected it to.

Here’s just a few things I’ve tried (that I can think of):

• skeleton.setCurrentPoseAsRest() (no discernible effect)
• mesh.applySkeleton() (messes it up even more)
• skeleton.needsSkinMatrix (no effect)
• Creating a parentMesh and anchoring the others to it (does nothing)
• MergeMeshes(didn’t help)

Perhaps you’ve seen something similar before or could point out some ways how to better debug this kind of problem? As always, thank you for your time and expertise!

[sebavan]

@Evgeni_Popov might be able to help with this but on a general note rigging by code is extremely hard and error prone.

Why not rigging in lets say Blender and only creating the animations in code ?

[Evgeni_Popov]

Maybe this thread and related links can help:

Trouble creating programmatic skeleton and bones

Hello, I am trying to create a simple skeleton that consists of individual meshes that are attached to each bone, no weights. I’ve created a basic skeleton with pelvis, thigh and shin bones, but I’m not getting the result that I expect. The skeletal viewer shows a strange visualization of bones, where they are all just pointing straight up. And, only the LINES visualization seems to display anything, the SPHERES and SPHERE_AND_SPURS style displays nothing. I was expecting the lines between…

But as @sebavan explained, creating rigged meshes by code is not an easy task…

[webgrid]

Hi,

I would recommended you to use blender of similar software, if you cannot I have made a tool that retarget bone weights for a skeleton.

Here is the code: glbtest/computeWeights.ts at main · webgrid/glbtest · GitHub

Warning: You have to modify the code to make it work for your project. Sorry I don’t have time to setup a playground for this.

I use this code in a web-worker so there are no Babylon references and you don’t have to apply the voxel data as I use that for more accurate bone weight mapping.

[rdw1]

Thank you! That thread did indeed solve the problem:

Instead of setting the rest pose after creating the bones, or even via the (confusingly-named?) “rest pose matrix” parameter, I had to set it via the “local pose matrix” parameter in the Bone constructor.

For what it’s worth, I have no idea what the difference is between the three matrices that one might pass to it, and the documentation doesn’t provide any details either. I actually tried setting the “rest pose matrix” before opening this thread, but it only makes the model vanish, whereas “local pose matrix” did what I’d expect the rest pose to do.

Very strange, but at least it’s working now!

[sebavan]

@PirateJC I know you love the docs.

[PirateJC]

@rdw1 - How would you feel about doing a PR to update the docs to be more helpful to future users on this subject?

Thoughts?

[rdw1]

I’m happy to do that, but I don’t know how these parameters are supposed to be used so I’d have to guess/check the code and hope I get it right.

[Pryme8]

Take a look at the skeleton debugger, I’m thinking that the spurs and spheres technically have the process you are wanting to do applies to them.

It’s kind of a pain to procedurally generate weights unless they are solid meshes and 100% influence. You could do a per vertex distance and threshold algo that would auto weight the mesh depending on their proximity to the transit vector between bones. If you are looking to do that then you are in for a fun time . The easiest way would prolly be to just set the vertex to 100% bone influence by the closest proximity from a cylinder between bones and then a secondary pass to blur the separate weight zones. There are also a few algos on github you could convert and save some time.

I could possibly put together a simplified demo for you, which could demonstrate the idea but as far as full deployment it’s gonna be on you (if I even find the time to do that).

Wow just realized how tardy I am to this party… Whoops.

[rdw1]

Thanks, but it seems you’re referring to a different problem (generate bone weights from scratch). My problem is (was) the following:

• Apply existing bone weights and animations while creating the entire skeleton/bone/geometry structure from scratch
• Tangential: Figure out why the SPHERES_AND_SPURS mode of visualization wasn’t working (only LINES did)

As mentioned above I have solved it by doing the following (thanks to the link posted by Evgeni_Popov):

• Pass the transformation matrix that describes the rest pose matrix as the THIRD (“local matrix”) parameter to the bone constructor

Previously, I was applying the rotation etc. manually after creating the bone. This was then overridden by the animations since they don’t take into account the rest pose unless set properly (?).

Afterwards I tried setting the rest pose (FOURTH) parameter, to no effect. I also tried using various combinations of setCurrentPoseAsRest(), returnToRest() etc. on both bones and skeletons, none of which did what I was expecting (use the rest pose matrix/fourth parameter as a basis for the animations).

All this just for anyone reading this in the future and getting stuck there as well.