Return to Rest not Behaving as Expected?

https://playground.babylonjs.com/#ZJIBBU#1
I dont think this should do this, but maybe I dont understand really whats going on.

https://playground.babylonjs.com/#ZJIBBU#2
Actor is fine when pausing animations before I do that.

https://playground.babylonjs.com/#ZJIBBU#3
and then right back to spaghetti monster as soon as animations come back on

Interesting question for @cedric or @Evgeni_Popov

It looks like there is an issue between the mesh geometry scale and the animation scale.
I guess the rest root bone scales down the mesh but the animation bones matrix are made for a scaled up geometry.
The rest matrices make the geometry smaller. If the animation doesnā€™t update all the bones, then there is a mismatch of scale between some.
Do you know why there is so much difference between the rest and the animation?
Also, is each bone animated ?
I guess, only setting back to rest the animated bones will work.

1 Like

No clue, itā€™s just one of the test assets that we are trying to have parse correctly. It happens on quite a few glbs that we have imported a d tested though.

Figured even if they are that miss matched on scale returning to rest should still work Iā€™d assume.

Iā€™ve been digging pretty hard into the skeletal animation stuff and the farther in I get the more problems we are having :sob:

By chance, are there Blender files that these .glb files were exported from?

There would be no way for me to know in most cases, this one was a fbx that sketchfab had hosted for free. Both the sketchfab conversion to glb and our proprietary glb converter have the same results.

I have other examples I could post but this one is a good representation of whats going on.

Is there a plausible workaround?

Iā€™ve had some rest-pose issues as well. In my current project I chose to do the folowing:

In the skeletons rest-pose:

for(let i=0;i<this.mesh.skeleton.bones.length;i++){
    const bone = this.mesh.skeleton.bones[i];
    const position = bone.getPosition(BABYLON.Space.LOCAL,this.mesh);
    const quaternion = bone.getRotationQuaternion(BABYLON.Space.LOCAL,this.mesh);
    this.localPositions.push(position);
    this.localOrientations.push(quaternion);
}

Then whenever Iā€™m done animating, in my case a ragdoll, I do the following:

for(let i=0;i<this.mesh.skeleton.bones.length;i++){
    const bone = this.mesh.skeleton.bones[i];
    const position = this.localPositions[i];
    bone.setPosition(position,BABYLON.Space.LOCAL,this.mesh);
    const quaternion = this.localOrientations[i];
    bone.setRotationQuaternion(quaternion,BABYLON.Space.LOCAL,this.mesh);
}

Ill test that.

I thought it worked initially, but that did not seem to do the trick.

https://playground.babylonjs.com/#ZJIBBU#8

Oh shit. I forgot that I actually had my rest-posed baked-in with the rest of the animations as one frame in order to do that. I more or less stopped digging after that, as it satisfied my own requirements.

Here are my findings after having a look at it:

The problem comes from the bone._absoluteTransform matrix that is updated when you call returnToRest() and that is not recomputed when you call setPosition / setRotationQuaternion.

Hereā€™s your PG where the absolute transform matrix is saved along the position/quaternion before calling returnToRest() and set after the call:

https://playground.babylonjs.com/#ZJIBBU#11

As you can see, it does work (note that I had to access the _absoluteTransform directly to change it as there is no accessor for it).

You could avoid the hack of saving/setting the absolute matrix by hand by calling bone.computeAbsoluteTransforms(). The problem is that the result of the computation is not the same as the matrix you saved before the call to returnToRest()ā€¦

Thatā€™s something I donā€™t understand and people more knowledgeable than me regarding skinning may have a look at:

  • just after loading, bone #0 and bone #1 both have a _absoluteTransform == identity
  • however, the scale of bone #1 is (100, 100, 100) and the quaternion is (-0.7071068286895816, 0, 0, 0.7071067336835166)
  • so, it seems either the absolute transform is wrong or the computation is not the one I think it is

It would seem the absolute transform is not wrong as the mesh is displayed correctly with this matrix and wrongly when calling computeAbsoluteTransformsā€¦ So, the computation is not the one I think it is, but computeAbsoluteTransforms() is performing this very same computation (namely, multiply all the parent local matrices together down to the current bone) and that seems logical to me. So, I donā€™t understand the relationship between the local position/rotation/scale properties and the value of the absolute transform matrix of a bone.

Note that there is another possibility, which is that the position/quaternion/scale are wrong and it is the absolute transform which is okā€¦

[EDIT]
Another possibility is that both position/rotation/scale AND the absolute matrix are ok. But in that case I may need some explanations on how to compute the absolute transform and why computeAbsoluteTransforms is doing it as it does currently (which would be wrong in that case).
[/EDIT]

2 Likes

Not possibleā€¦ lol Evgeni Popov you are our only hope! Naw that scares me when you say that though for real.

I donā€™t know a lot regarding skinning and animations and those are complex topics, so some experts are needed to go further :slight_smile:

So here is an example where I am struggling to get the expected output.
Everything works in a setup like this:
https://playground.babylonjs.com/#LS8CPQ#5

but when I do a glb or a skeleton with the problems like above I am able to get the ā€œbonesā€ in the right place:
https://playground.babylonjs.com/#LS8CPQ#7

Then when I apply the skeleton the weights take effect and everything goes crazy!
https://playground.babylonjs.com/#LS8CPQ#8

How can I get a new mesh in the correct place and weight it with setups like this?

Here is the model in blender one of the glb just loaded and then the second of her in what blender effectively sees as ā€œreturnToRestā€

While in comparison to what BJS does:

<-- approxemently taken from the same angle as the ones above. (just quite a bit zoomed in)

Which is really odd, because the bones stay in the same spot as if they never moved and behaves more like if I removed the skeleton from the meshes while not doing anything to the bones themselves.

It would be my argument that a skeleton in BJS that is returned to rest should have the same effect as in blender toggling the armatures pose image

Ideally if I can get in BJS the rig to return to this pose:

I can get my stuff to work.

It seems the mesh is in the same ā€œposeā€ than in Blender, only that it is rotated 90Ā° around the X and 180Ā° around the Y coordinates?

kinda, but in BJS the bones stay in the action pose and the mesh scales down by 100 units then is rotated in the manor you say.

But in blender the ā€˜returnToRestā€™ or toggling the armature puts her in the right spot the bones go to the right place everything acts the way it should and its the same file so its not like blender is just assuming that data, it came in somewhere.

https://playground.babylonjs.com/#ZJIBBU#12 The bones should return to the same T-Pose as the last image I posted.

While the behavior is currently the same as If I removed the mesh and did nothing to the bones:
https://playground.babylonjs.com/#ZJIBBU#13
Which I would assume returnToRest should have different results then detaching the skeleton.

@Evgeni_Popov is it because the transform nodes are not getting returned to the restPose so then the bones dont?

whats making me trip out is that the debug layer shows the bones still in the action pose while the mesh is back in a kinda T-Pose (though not the one I would expect)

https://playground.babylonjs.com/#LS8CPQ#10

Here is another GLB that the stuff ā€œworksā€ onā€¦ but it seems the system still comes in inverted.

Really I donā€™t know, I donā€™t know enough about skeleton/bones to be able to answer this, I would need to spend some time studying (and understanding) the code.

What I can tell is that bone.returnsToRest() does:

    public returnToRest(): void {
        this.updateMatrix(this._restPose.clone());
    }

So it would seem the rest pose matrix of each bone are indeed used to (re)set the current matrixā€¦