BUG: scaled meshes don't get binded to bones properly

Hi Jedi Council,

There is a bug with scaled meshes being binded to skeletons.

In this scene, the tree mesh here has 0.01 scale. I bind this mesh weights to a skeleton bone, which is linked to the box.
So when the box rotates, the bone rotates, and you would expect the mesh to rotate around the box/bone right?
WRONG!!

The tree mesh ends up rotating around its own pivot, NOT the box/bone.
Here’s the PG: https://playground.babylonjs.com/#SYQW69#1079

Hacky Workaround
BUT, if I run mesh.bakeCurrentTransformIntoVertices, the vertices are baked, the scale = 1, and the mesh works normally by rotating around the actual bone/box.
Though this is obviously a bad hack/workaround.
image

Why does this happen / Is there a fix?

@Blake @Deltakosh

This is not a bug, the skeleton and linked transform hierarchy is pretty complex and always related to the skinned mesh.

@bghgary can provide more info :slight_smile: but I guess we should have more doc about this as we see an increasing number of questions.

1 Like

what is the elegant way around this?
Do I have to modify the skinned mesh? I think manually modifying and baking the vertices is a last resort

Here’s one way using a parent bone instead of linking. I couldn’t get the same behavior using the linked node thou because the bone uses its local matrix…

1 Like

Thanks @Blake ! This is helpful, though I’m still looking for a more generic solution.
This PG assumes bones are created just to match the mesh position, though in a real usecase it’s the other way round - the mesh should follow the bones.
Imagine binding multiple meshes to bones etc

@bghgary do you have any suggestions? :slight_smile:

Ooh yep I see how it wouldn’t work for more complicated situations with multiple meshes per bone… I’m stumped for now then but hopefully bghgary or someone else can help with a more general, full solution. :slight_smile:

1 Like

I think we have a misunderstanding of how bones work?

I’m not quite sure why scaled meshes have anything to do with this. Whether it’s scaled or not doesn’t seem to have any impact on the result? E.g., if I use the inspector to change the scale of Sphere001 from 0.001 to 1, it just gets bigger. Do you have an example where it behaves the way you want when it’s not scaled?

Just as FYI, the linked transform node has nothing to do with this. This behavior is the same with or without it. The code just copies the local transform of the linked transform node to the bone when they are linked.
PG: https://playground.babylonjs.com/#SYQW69#1082

If you only have one bone, it will just rotate the skinned mesh. This behavior is correct. You need two bones to create a rotation around an axis.

The reason why this “works” is because the position of the mesh is being baked into the mesh and thus the origin of the mesh is now at the base of the tree. It’s still just rotating the single skinned mesh because there is just one bone. The scale is not related.

While I agree this is not a bug, this is not the same category as the other issues users are reporting. The other issues we are seeing are because of the changes we made to glTF skeletons. This issue is just how skeletons work in general. The glTF being loaded has no skinning.

Maybe you can add some more context. I don’t understand what you are trying to do.

1 Like

Yea I know most of what you mentioned, the linked box/transformNode was just for visualization purposes, to show the box moving. The question wasn’t about tnodes

Maybe I’m misunderstanding it, but have a look at this PG: https://playground.babylonjs.com/#SYQW69#1084

The mesh pivot is on the base of the tree, as you pointed out, but it is still correctly rotating around the box/bone. This is the correct behavior I’d want for this mesh, all the time, regardless of its scale.
image

But probably I’m not fully understanding how it all works.
Isn’t the mesh supposed to rotate around the bone, as if the bone is the pivot? And if not, how do I get it to do that? Do I need to set up some sort of bone pivot native to the mesh?

You have one bone. It will only rotate around that bone. I said skinned mesh before because the bone’s transform is the identity matrix so it is equivalent to the skinned mesh. I probably should have said bone instead. In your current example, you moved the node linked to the bone, which moves the bone. The mesh is rotating around that bone with an offset from baking the transform into the vertices of the mesh.

Can you describe the whole scenario? I still don’t understand what you are trying to do.

Heya, here’s a little material plugin that applies the world transform first, then the skeleton, to hopefully match the behavior of baking the transform into the vertices, to help provide more flexibility… Warning thou: it seems to work well here but still needs more testing so let me know how it works out for you. :slight_smile:

1 Like

Thanks a lot @Blake ! I had no idea you could add ‘plugins’ like this in BJS :smiley:

Do you think this will look the same if we export it to blender etc, or would this only work for BJS scenes?

1 Like

They’re a super useful new way to customize PBR and standard materials that I’m loving too :star_struck: but AFAIK the plugins are limited to Babylon scenes and wouldn’t work in an external tool like Blender thou…

2 Likes

I’m still pretty certain it’s a bug
If I export the whole original PG as GLTF, and re-import it into sandbox.babylonjs.com, the skinned mesh is completely blown out of proportion

I’m trying to rig this tree properly, and I can’t do that if the skinned vertices rotate around the mesh pivot instead of the bone itself. It doesn’t just affect one bone, it’s literally all bones.
In this video, the skinned mesh vertices rotate around the mesh pivot instead of the bone, which is counter-intuitive
Not to mention, the GLTF export completely blows the mesh out of proportion (mentioned above)

The below is the behavior one would expect for bones, but I can only achieve this if I use mesh.bakeCurrentTransformsToVertices

Is there a better fix for this, that doesn’t use bakeCurrentTransformsToVertices, that can also export properly as a GLTF?
Otherwise, as it stands, I’m pretty certain this is a bug that we should fix (at the very least the GLTF export part)

1 Like

I wouldn’t be surprised if there are issues in the exporter. The export code is relatively new and not that well tested. I would be very surprised if there are issues with the bone calculations at runtime. This code path has been tested quite thoroughly.

Thanks for the context. Are you trying to rig this at runtime? Can you use a DCC tool (Max/Maya/Blender) to rig it and then compare the results with what you are doing at runtime?

EDIT:
One thing that is missing from your PG is the bone matrices. If the bone matrices are all identity, it probably won’t work for typical rigs (like this tree). This glTF tutorial about skinning gives some idea of how rigging works. It might be worth a read. Note that Babylon doesn’t take inverse bind matrices directly and thus the math doesn’t match up exactly with the tutorial, but hopefully it gives you an idea.

Do you think if this can be resolved with changing the mesh.poseMatrix?
Or should I manually change the bonesTransformMatrices?

You should not be modifying either of these. They are both internal. I would expect you need to pass in non-identity matrices to the bone constructor.

changing the bone matrices will affect all the other meshes.
Sure, I could scale the bone matrices to fit just that mesh, but if there are other meshes binded to the bone it’ll not work well. It’s not a generic solution

AFAIK Blender uses additional matrices to preserve mesh->skeleton bind state. Bjs currently doesn’t have that. And I think we need it

Blender is able to export to glTF. Babylon.js supports glTF. Can you try exporting from Blender to glTF and see what results you get?