Inconsistent GLTF bone transformations

Maybe we need a link. I documented how this works for gltf here: glTF 2.0 Skinning | Babylon.js Documentation

2 Likes

Even better:
GLTF character bone positions | Babylon.js Playground (babylonjs.com)

I’m using node.getTransformNode() on the root bone to get the mesh to use with BoneAxesViewer

… and that seems to be container.skeletons[0].bones[0].getTransformNode() - at least as far as BoneAxisViewer is concerned, at least in the playground :slight_smile:
Locally I have a twilight zone: two BoneAxisViewers, for upper and lower leg, and they work fine as long as I update lower one first :grin: Otherwise I get like


Something seems off with transformation hierarchy…

I have update on this: GLTF character bone positions | Babylon.js Playground (babylonjs.com)
This looks fine; but create root mesh and bone absolute position goes way away.

So really, what do I need to do to get actual bone position?

cc @bghgary

Sorry, I’m not understanding this. What do you mean?

@bghgary see the playground above, in particular

            // replace this with with that to mess it up
            var root = container.meshes[0];
            //var root = container.createRootMesh();

and

            var absolutePos = bone.getAbsolutePosition(transform);
            console.log(absolutePos);
            var sphere = BABYLON.MeshBuilder.CreateSphere();
            sphere.position = absolutePos;

The absolute position of the bone changes from
_x: -1.195454716682434, _y: 1.6862452030181885, _z: -0.047228556126356125
to
_x: -3.9090933799743652, _y: 33.72490310668945, _z: 0.9445710778236389
when the root mesh is created. Not very absolute is it? :wink:
So is it expected behavior? If it is, what do I need to do to get actual absolute position of the bone?

You would need to force computing the world matrices if you do changes before using them: https://playground.babylonjs.com/#H925CH#8

2 Likes

Oh dang, but on transform node!
I’ve been trying on bone :slight_smile:
Alright thanks @sebavan

What a mess :grin:
Took me quite a while to pinpoint where things have gone wrong, had to look for babylon 4 on cdns to compare.
So Bone.getAbsolutePosition() used to return:
{X: 0.1375466307330413 Y:1.6750438330035953 Z:0.01731602581149172}
with babylon 4, and with 5 returns
{X: 0.13754705781588328 Y: -0.017311424858179172 Z: 1.6750443792873426}
Care to guess how it affects IK? :wink:
No big deal, says me, I’ll just use Bone.getAbsolutePosition(someTransform), right?
Yeah right.
This position mess only reflects what happened with rotation matrix. Rotation of the same bone was
{X: 0.7772532463104096 Y:0.40210674488017334 Z:0.40210645730669714 W:-0.26925494988232973}
with 4, and with 5 is
{X: -0.35920901177680326 Y: -3.02053400145857e-7 Z: -0.5686636932234524 W: 0.7399930535454382}
And I can’t create playground example with babylon 4, right?
So instead of wasting even more time, let me simply ask.
For IK I need two vectors: current arm vector, and target vector.
Then I simply
var finalRotation = new BABYLON.Matrix();
BABYLON.Matrix.RotationAlignToRef(armVector.normalizeToNew(), targetVector.normalizeToNew(), finalRotation);
and use this rotation quaternion for either directly setting arm quaternion or animation.
So how do I get target vector and arm vector, in the same coordinate system that bone quaternion uses?
I see bunch of methods containing ‘local’ and ‘absolute’ in their names, but absolute obviously is not absolute, and what’s ‘local’ supposed to mean?
So please, how can I convert a vector in global coordinate system (intentionally avoiding word ‘absolute’) to coordinate system of a bone?
Some context - this doesn’t work with babylon 5:

What bone are you referring to here?

You can create a playground with 4 by selecting the version near the top right.

If you can get it working with Babylon 4.2.1 in the playground, I can look into why it doesn’t work in 5.x.

1 Like

@sebavan I appreciate it, but ‘magic’ is not really a way to go :slight_smile: Clear way to convert from this to that coordinate system is, and it might already be there.

@bghgary that’s the catch, it’s some bone (arm,leg…) with some characters. When I look at the avatar structure, there’s exactly one difference: TransformNode child of RootNode (model correction matrix) is rotated by 90 degrees for some models and not rotated for others, see attached images (both characters are supposed to touch the sphere). But all characters are rotated by -90 in RootNode (gltf orientation matrix). So for the characters that have these rotations zeroing each other out, my math works fine, and for others, not.
And more than that I really can’t say :slight_smile:
Because, well everyone works with predefined character structure, right? Well I’ve come up with the system that recognizes structure of some 96% rigged characters. I just download them from sketchfab, and have no idea where they come from, how they are made, and what bone orientations are.
You know they say, give me a firm spot on which to stand, and I shall move the world :wink:

OK thanks, I’ll come up with a playground.


I’m not asking for generically what bone. I need a specific example, so I can debug the problem.

By magic I meant a flag loading as before but it was for a totally different issue

@bghgary here’s the playground 1, for babylon 4:

Some bookkeeping but all fine.
And here’s the same with babylon 5:

Well I just can’t figure out how to get simple position of a bone.
Funny though, things change with call to container.createRootMesh(). It almost becomes usable, except left and right side switched places :slight_smile:

Well if that solves issues, it would make sense. This is a show stopper, not just for me, but for anyone out there using gltf characters. For example I see that framevr uses v5.0.0-beta.11 :slight_smile: I’m not affiliated with any organization, but I know of handful of startups using the code.
So show-stopper for adoption of babylon 5 really.

There was a change from 4.x to 5.x where we changed how the skeletons are constructed in the glTF, but the usage of any of these bone functions is the root cause. These bone functions expect the mesh associated with the skeleton to be passed in. This is necessary because the skeleton maybe applied to multiple skinned meshes with different transforms. What used to happen in v4 is that the whole chain of node transforms was put into the skeleton and that’s why it was okay to not put the associated skinned mesh.

Here is how you could do it in 5.x: https://playground.babylonjs.com/#H925CH#37

This is a similar problem to this issue: Parenting a transform node to a bone

EDIT: another way you can do it (only for glTF) without needing to know which skinned mesh is to use the linked transform node: https://playground.babylonjs.com/#H925CH#38

Really curious to know why you removed the second avatar and hid the axesviewer. =)

@bghgary thanks for the explanation, but the way I see it, root cause is that you don’t see any issues here :slight_smile:
Look how one line of code breaks your ‘solution’ : https://playground.babylonjs.com/#H925CH#39
Now why would I be using container.meshes[1], why not container.meshes[3] or container.meshes[0]? Is there any indicator anywhere to tell me which one is the right mesh? Why not bones[0].getTransformNode() that I was using in the example?

Going off topic, but this entire discussion strongly reminds me of a joke:
A helicopter was flying around above Seattle when an electrical malfunction disabled all of the aircraft’s electronic navigation and communications equipment. Due to the clouds and haze, the pilot could not determine the helicopter’s position and course to fly to the airport. The pilot saw a tall building, flew toward it, circled, drew a handwritten sign, and held it in the helicopter’s window. The pilot’s sign said “WHERE AM I?” in large letters. People in the tall building quickly responded to the aircraft, drew a large sign and held it in a building window. Their sign read: “YOU ARE IN A HELICOPTER.” The pilot smiled, waved, looked at her map, determined the course to steer to SEATAC airport, and landed safely. After they were on the ground, the co-pilot asked the pilot how the “YOU ARE IN A HELICOPTER” sign helped determine their position. The pilot responded “I knew that had to be the Microsoft building because, like their technical support, online help and product documentation, the response they gave me was technically correct, but completely useless.”