Inconsistent GLTF bone transformations

Hi,
some GLTF characters have their bone positions and/or rotations messed up. I can’t say what’s exactly wrong, but BoneAxesViewer also gets it wrong. That’s regression in v5.
Playground: Babylon.js Playground

pinging @bghgary

I’m doing a first check to see if I can fix within the next hour

Ok I found the root cause :slight_smile:
The way we handled skeleton within glTF was a bit too convoluted in 4.2

So we decided to simplify it a bit. From the breaking changes of 5.0:
https://doc.babylonjs.com/whats-new#breaking-changes

Loading glTF assets with skins now places skinned meshes as siblings of the corresponding skeleton root nodes instead of using skeleton.overrideMesh

(@bghgary I think we should document that and how it works in the skeleton documentation)

Here is the fix for your PG:
GLTF character bone positions | Babylon.js Playground (babylonjs.com)

The idea is to figure out which transformNode is the root of the actual mesh

1 Like

Maybe we need a link. I documented how this works for gltf here: glTF 2.0 Skinning | Babylon.js Documentation

2 Likes

Even better:
GLTF character bone positions | Babylon.js Playground (babylonjs.com)

I’m using node.getTransformNode() on the root bone to get the mesh to use with BoneAxesViewer

… and that seems to be container.skeletons[0].bones[0].getTransformNode() - at least as far as BoneAxisViewer is concerned, at least in the playground :slight_smile:
Locally I have a twilight zone: two BoneAxisViewers, for upper and lower leg, and they work fine as long as I update lower one first :grin: Otherwise I get like


Something seems off with transformation hierarchy…

I have update on this: GLTF character bone positions | Babylon.js Playground (babylonjs.com)
This looks fine; but create root mesh and bone absolute position goes way away.

So really, what do I need to do to get actual bone position?

cc @bghgary

Sorry, I’m not understanding this. What do you mean?

@bghgary see the playground above, in particular

            // replace this with with that to mess it up
            var root = container.meshes[0];
            //var root = container.createRootMesh();

and

            var absolutePos = bone.getAbsolutePosition(transform);
            console.log(absolutePos);
            var sphere = BABYLON.MeshBuilder.CreateSphere();
            sphere.position = absolutePos;

The absolute position of the bone changes from
_x: -1.195454716682434, _y: 1.6862452030181885, _z: -0.047228556126356125
to
_x: -3.9090933799743652, _y: 33.72490310668945, _z: 0.9445710778236389
when the root mesh is created. Not very absolute is it? :wink:
So is it expected behavior? If it is, what do I need to do to get actual absolute position of the bone?

You would need to force computing the world matrices if you do changes before using them: https://playground.babylonjs.com/#H925CH#8

2 Likes

Oh dang, but on transform node!
I’ve been trying on bone :slight_smile:
Alright thanks @sebavan

What a mess :grin:
Took me quite a while to pinpoint where things have gone wrong, had to look for babylon 4 on cdns to compare.
So Bone.getAbsolutePosition() used to return:
{X: 0.1375466307330413 Y:1.6750438330035953 Z:0.01731602581149172}
with babylon 4, and with 5 returns
{X: 0.13754705781588328 Y: -0.017311424858179172 Z: 1.6750443792873426}
Care to guess how it affects IK? :wink:
No big deal, says me, I’ll just use Bone.getAbsolutePosition(someTransform), right?
Yeah right.
This position mess only reflects what happened with rotation matrix. Rotation of the same bone was
{X: 0.7772532463104096 Y:0.40210674488017334 Z:0.40210645730669714 W:-0.26925494988232973}
with 4, and with 5 is
{X: -0.35920901177680326 Y: -3.02053400145857e-7 Z: -0.5686636932234524 W: 0.7399930535454382}
And I can’t create playground example with babylon 4, right?
So instead of wasting even more time, let me simply ask.
For IK I need two vectors: current arm vector, and target vector.
Then I simply
var finalRotation = new BABYLON.Matrix();
BABYLON.Matrix.RotationAlignToRef(armVector.normalizeToNew(), targetVector.normalizeToNew(), finalRotation);
and use this rotation quaternion for either directly setting arm quaternion or animation.
So how do I get target vector and arm vector, in the same coordinate system that bone quaternion uses?
I see bunch of methods containing ‘local’ and ‘absolute’ in their names, but absolute obviously is not absolute, and what’s ‘local’ supposed to mean?
So please, how can I convert a vector in global coordinate system (intentionally avoiding word ‘absolute’) to coordinate system of a bone?
Some context - this doesn’t work with babylon 5:

What bone are you referring to here?

You can create a playground with 4 by selecting the version near the top right.

If you can get it working with Babylon 4.2.1 in the playground, I can look into why it doesn’t work in 5.x.

1 Like

@sebavan I appreciate it, but ‘magic’ is not really a way to go :slight_smile: Clear way to convert from this to that coordinate system is, and it might already be there.

@bghgary that’s the catch, it’s some bone (arm,leg…) with some characters. When I look at the avatar structure, there’s exactly one difference: TransformNode child of RootNode (model correction matrix) is rotated by 90 degrees for some models and not rotated for others, see attached images (both characters are supposed to touch the sphere). But all characters are rotated by -90 in RootNode (gltf orientation matrix). So for the characters that have these rotations zeroing each other out, my math works fine, and for others, not.
And more than that I really can’t say :slight_smile:
Because, well everyone works with predefined character structure, right? Well I’ve come up with the system that recognizes structure of some 96% rigged characters. I just download them from sketchfab, and have no idea where they come from, how they are made, and what bone orientations are.
You know they say, give me a firm spot on which to stand, and I shall move the world :wink:

OK thanks, I’ll come up with a playground.


I’m not asking for generically what bone. I need a specific example, so I can debug the problem.

By magic I meant a flag loading as before but it was for a totally different issue

@bghgary here’s the playground 1, for babylon 4:

Some bookkeeping but all fine.
And here’s the same with babylon 5:

Well I just can’t figure out how to get simple position of a bone.
Funny though, things change with call to container.createRootMesh(). It almost becomes usable, except left and right side switched places :slight_smile: