Return to Rest not Behaving as Expected?

There would be no way for me to know in most cases, this one was a fbx that sketchfab had hosted for free. Both the sketchfab conversion to glb and our proprietary glb converter have the same results.

I have other examples I could post but this one is a good representation of whats going on.

Is there a plausible workaround?

I’ve had some rest-pose issues as well. In my current project I chose to do the folowing:

In the skeletons rest-pose:

for(let i=0;i<this.mesh.skeleton.bones.length;i++){
    const bone = this.mesh.skeleton.bones[i];
    const position = bone.getPosition(BABYLON.Space.LOCAL,this.mesh);
    const quaternion = bone.getRotationQuaternion(BABYLON.Space.LOCAL,this.mesh);
    this.localPositions.push(position);
    this.localOrientations.push(quaternion);
}

Then whenever I’m done animating, in my case a ragdoll, I do the following:

for(let i=0;i<this.mesh.skeleton.bones.length;i++){
    const bone = this.mesh.skeleton.bones[i];
    const position = this.localPositions[i];
    bone.setPosition(position,BABYLON.Space.LOCAL,this.mesh);
    const quaternion = this.localOrientations[i];
    bone.setRotationQuaternion(quaternion,BABYLON.Space.LOCAL,this.mesh);
}

Ill test that.

I thought it worked initially, but that did not seem to do the trick.

https://playground.babylonjs.com/#ZJIBBU#8

Oh shit. I forgot that I actually had my rest-posed baked-in with the rest of the animations as one frame in order to do that. I more or less stopped digging after that, as it satisfied my own requirements.

Here are my findings after having a look at it:

The problem comes from the bone._absoluteTransform matrix that is updated when you call returnToRest() and that is not recomputed when you call setPosition / setRotationQuaternion.

Here’s your PG where the absolute transform matrix is saved along the position/quaternion before calling returnToRest() and set after the call:

https://playground.babylonjs.com/#ZJIBBU#11

As you can see, it does work (note that I had to access the _absoluteTransform directly to change it as there is no accessor for it).

You could avoid the hack of saving/setting the absolute matrix by hand by calling bone.computeAbsoluteTransforms(). The problem is that the result of the computation is not the same as the matrix you saved before the call to returnToRest()

That’s something I don’t understand and people more knowledgeable than me regarding skinning may have a look at:

  • just after loading, bone #0 and bone #1 both have a _absoluteTransform == identity
  • however, the scale of bone #1 is (100, 100, 100) and the quaternion is (-0.7071068286895816, 0, 0, 0.7071067336835166)
  • so, it seems either the absolute transform is wrong or the computation is not the one I think it is

It would seem the absolute transform is not wrong as the mesh is displayed correctly with this matrix and wrongly when calling computeAbsoluteTransforms… So, the computation is not the one I think it is, but computeAbsoluteTransforms() is performing this very same computation (namely, multiply all the parent local matrices together down to the current bone) and that seems logical to me. So, I don’t understand the relationship between the local position/rotation/scale properties and the value of the absolute transform matrix of a bone.

Note that there is another possibility, which is that the position/quaternion/scale are wrong and it is the absolute transform which is ok…

[EDIT]
Another possibility is that both position/rotation/scale AND the absolute matrix are ok. But in that case I may need some explanations on how to compute the absolute transform and why computeAbsoluteTransforms is doing it as it does currently (which would be wrong in that case).
[/EDIT]

2 Likes

Not possible… lol Evgeni Popov you are our only hope! Naw that scares me when you say that though for real.

I don’t know a lot regarding skinning and animations and those are complex topics, so some experts are needed to go further :slight_smile:

So here is an example where I am struggling to get the expected output.
Everything works in a setup like this:
https://playground.babylonjs.com/#LS8CPQ#5

but when I do a glb or a skeleton with the problems like above I am able to get the “bones” in the right place:
https://playground.babylonjs.com/#LS8CPQ#7

Then when I apply the skeleton the weights take effect and everything goes crazy!
https://playground.babylonjs.com/#LS8CPQ#8

How can I get a new mesh in the correct place and weight it with setups like this?

Here is the model in blender one of the glb just loaded and then the second of her in what blender effectively sees as “returnToRest”

While in comparison to what BJS does:

<-- approxemently taken from the same angle as the ones above. (just quite a bit zoomed in)

Which is really odd, because the bones stay in the same spot as if they never moved and behaves more like if I removed the skeleton from the meshes while not doing anything to the bones themselves.

It would be my argument that a skeleton in BJS that is returned to rest should have the same effect as in blender toggling the armatures pose image

Ideally if I can get in BJS the rig to return to this pose:

I can get my stuff to work.

It seems the mesh is in the same “pose” than in Blender, only that it is rotated 90° around the X and 180° around the Y coordinates?

kinda, but in BJS the bones stay in the action pose and the mesh scales down by 100 units then is rotated in the manor you say.

But in blender the ‘returnToRest’ or toggling the armature puts her in the right spot the bones go to the right place everything acts the way it should and its the same file so its not like blender is just assuming that data, it came in somewhere.

https://playground.babylonjs.com/#ZJIBBU#12 The bones should return to the same T-Pose as the last image I posted.

While the behavior is currently the same as If I removed the mesh and did nothing to the bones:
https://playground.babylonjs.com/#ZJIBBU#13
Which I would assume returnToRest should have different results then detaching the skeleton.

@Evgeni_Popov is it because the transform nodes are not getting returned to the restPose so then the bones dont?

whats making me trip out is that the debug layer shows the bones still in the action pose while the mesh is back in a kinda T-Pose (though not the one I would expect)

https://playground.babylonjs.com/#LS8CPQ#10

Here is another GLB that the stuff “works” on… but it seems the system still comes in inverted.

Really I don’t know, I don’t know enough about skeleton/bones to be able to answer this, I would need to spend some time studying (and understanding) the code.

What I can tell is that bone.returnsToRest() does:

    public returnToRest(): void {
        this.updateMatrix(this._restPose.clone());
    }

So it would seem the rest pose matrix of each bone are indeed used to (re)set the current matrix…

Shouldn’t they move though? What would be keeping them in the action pose while the mesh updates?

Sorry if I am hammering on this, but it really seems like something that should be solved as it kinda is making me thing there is a underlying problem with our skeleton stuff.

Yes I do think there’s a problem somewhere (or something more fundamental is escaping us), the thing is to find it…

1 Like

Ok cool, just making sure we were on the same page. I feel like once we figure it out we are both gonna palm our foreheads.

I think I have narrowed it down.

In Blender the “restPose” for this model is when all the bones are set to (-0.707, 0, 0, 0.707) rotation, but I do not see these numbers in any matrix in BJS?

How is blender getting these numbers when they import the GLB its the same file so it has to be somewhere.


^I think the solution might come from here?

Looking at the hierarchy, I can see that blender uses a scaling node to convert between the scene units:

This works for glTF, since we can set an arbitrary node as our skeleton root, but in babylon we extend the skeleton to use all nodes in its hierarchy, up to the scene root:

If we look at the implementation of returnToRest(): I believe that we’re currently resetting the scale of that root helper node in addition to the rest of the skeleton:

currently if i disable resetting the bones that our loader adds, the rest position mesh starts looking a bit more proper:

    /**
     * Forces the skeleton to go to rest pose
     */
    public returnToRest(): void {
        for (var index = 0; index < this.bones.length; index++) {
            if(this.bones[index]._index != -1) {
                this.bones[index].returnToRest();
            }
        }
    }

yields:

But the skeleton helper is still generated in a posed position… even though we “returned” our skeleton to rest position, since the bones are linked to a transformNode that drives their transform, that may explain why it doesn’t seem to work in reverse. The transform nodes still appear to have their posed orientation and we may have reset and re-rendered the skeleton using the reset position, I’m not sure if this won’t be re-written by linked transform node…

4 Likes