Root Motion Animation System

Yo @sebavan (FYI @Deltakosh)…

Hey guys. I need help with implementing Root Motion in BabylonJS

I cant find anything written in stone as to how it actually works. I seen some folks talk about simply taking the XZ position and apply is to the parent game object instead of the root bone. But that does not seem complete to me.

Also i dont know how to turn the data in the animation clip for transform position to Additive Deltas so that when the animation loop, the transform does NOT go back to the starting position when the animation started playing, but rather add to its current position and rotation.

I really need to talk to someone about these root motion concepts so i can better make the exporter support root motion on playing animations. The most important thing i can think of… How to apply the Position and Rotation from the animation clip as delta movements to the parent node transform node, so that motion from the animation clip drives the parent game object containing the bones ???

This will be a very kool animation system for babylon… Will totally eliminate things like foot slide when trying to match the actual movement of the character with the movement from the animation clip.

Please help me :slight_smile:

2 Likes

:slight_smile: 16 days ago, and no replies, MK24? What the heck, huh?

Been here? AnimationTree — Godot Engine (stable) documentation in English

If so, you’re kind-of sick and demented.

What you are talking about here… is avatars/skeletons. I’m not sure that “core” library coders would want to code/install “Poser Stuff” into/onto the nice clean webGL engine.

BabylonJS COULD have a powerful extension or tool for avatars/characters, but that is a “specific use-case” for BJS, and the core-folk likely don’t wish for this “Unity-specific game logic” to be part of “core”.

Unity “wider-than-core features” don’t align-with BabylonJS core-purpose… very well, do they? Avatar behaviors are seen as “game logic” to pure BJS core-coders, I think (mentioned just above).

I believe this is why… Sebavan and DK are acting like your arm-pits stink, Mackey. hehe. (But I’m easily and often wrong, as we all know-well).

Exporting always has limitations. Likely, for max compat/power, you will need to develop a Unity-conforming BJS armature/skeleton-class… for/by YOURSELF, because it is a “special thing” requiring a special highway.

And, it would be a class that needs maintenance EVERY TIME Unity evolves its armature/character (root motion-activated) class. You could go insane… Mackey.

The “animation blending” problems ALONE… in honoring Unity-export-dictated… motion-on-the-root-bone… might take a complete re-write of BJS bone transformation stuff.

(Like I have a clue.)

In short… I think character animation is too “specific” and/or “specialized”… to garner much attention from core-folk.

Actually, I just needed some typing practice, so I thought I would aimlessly yammer on some stale thread. heh. Party on.

Completely missed this post and i feel bad about it :frowning: but even if I did not miss it I am clueless regarding the request :slight_smile: maybe @bghgary would know ???

I got most of the concepts down expect using the animation position changes as delta movements to move the root transform Mesh itself…

I don’t understand the vector math I need to do to take the delta movements from the animation track and use that to move

Yo @Wingnut … no need for all core stuff you talking about… I re created a unity style animation system using a simple script component that you can add to your project or not … so I already use Babylon code to update the animation transform , weather it’s a skeleton or not

I have the perfect spot to handle root motion … I just can’t get my head around the technique of taking the delta movements from the animation so I can apply to the root transform

1 Like

Also I want to get across … all this stuff I am creating is for web game development mechanics…

If you wanna create real games using Babylon … your gonna have to deal with the actual mechanics of things like fully interactive and animated characters… how to actually move them around in a game scenario… things like character controllers, things like animation state, things like vehicles :red_car:

And most importantly… I’ll teach how to actually setup and code these features out… using modern unity style game mechanics :toolbox:

2 Likes

Maybe @PatrickRyan has dealt with this before. I don’t think we have a built-in equivalent to Unity’s root motion. I only have a high-level understanding of how it works. It may be worth adding something to Babylon to support this?

2 Likes

@MackeyK24, I am in the same boat as @bghgary in that I have a high level understanding of the system and have tried it out when I was working in Unity regularly, but I can say that I was less than impressed with the system. I think the biggest challenge with it is that while it does work with animation that has a forward motion like a walk or run, if you do anything like a sidestep or strafe the system fails.

If I were to think of how I would want to implement a root motion system, I would want the following types of concessions:

  • We have to determine which types of animations are eligible for root motion and develop a system that works with both root motion animation and non-root motion animation. An example would be a run animation that is eligible for root motion and then a jump animation that does not utilize root motion because we want to control the height and distance of the jump without needing to tie that to the animation.
  • The root motion would be taken from identified foot bones based on when the foot is in contact (with a small range of error to account for animation issues) with the ground. We would record the delta of the position of the foot bone position in the animation from the animation from the previous frame and apply the inverse of that delta to the character root (which may or may not be the skeleton root in the case that the character has any attachments not made a child of a skeleton bone).
  • There will need to be some sort of inertia added to the motion with a falloff to that motion to account for the distance traveled while both feet are off the ground in something like a run cycle.

My thought with a system like this is that if we are only concerned with deltas of the feet when they are in contact with the ground, that we could solve for a strafe or sidestep as well as a walk/run.

The edge cases in my head would be how do handle motion like a character leaning forward to transfer their center of gravity to a position in front of their feet to launch from that position. The feet may not move very much if the animation leaves IK feet in position and animates the root forward leaving no real inertia for that initial burst of speed. The animation would need to leave the root in place and shift the center of gravity by sliding the feet in FK to transfer the motion to the character root. As with any root motion solve, this would need to be designed to work with specific animation practices to get acceptable results.

Now, rotation is a different beast all together because you can travel in a straight line while rotating your body 360 degrees every X steps like a dancer. This rotation could really whack out the root motion if we also look at the rotation of the foot per frame. This also ignores the fact that a lot of motion we perform as bipeds does require a slide in rotation in preparation for a change in direction of our motion. Even looking at the rotation of the skeleton root for a forward vector gets problematic when you think of a strafe. I think I would approach it from the standpoint of the motion offset I mentioned above and see where we need to go from there in terms of rotation data as it seems like it may be helpful for something like a blended lean to change a straight run into a turning run, but I think that extra layer may need to be a unique add to the root motion based on your animation sets like the jump I mentioned earlier.

I know this doesn’t help you solve the math or implementation of a system for root motion, but I am hoping that the way smarter members of the team (I’m looking at you @bghgary, @sebavan, @Cedric, @Drigax, and DK) can take this concept and actually translate that more concrete suggestions for @MackeyK24 because I agree that adding something like this would be a large step toward improving the process of making games in Babylon.

2 Likes

It looks like we need a locomotion system. Based on animations (half turn, turn, walk, run,…), we compute a parametric 2D Delaunay triangulation by extracting linear velocity (Y axis) and angular velocity (X axis). Then, from the user input angular and linear velocity (think of the player moving his pad direction stick), 3 animations are blended using the barycentric coordinates of the Delaunay triangulation. To get smooth transition, the barycentric can also be interpolated.
That’s for selecting the animations.
image
Side note: it’s possible to compute mirrored animations and put them in the locomotion graph (like mirroring 45deg turn right to get 45deg turn left)
Then comes the animations blending.
We had good results in my previous company by having local animation time per leg in order to preserve stance. This was not an easy task as the preprocess has to detect when the foot is on ground. But this gives the ability to orient the foot on non planar ground and do foot prediction for ground adaptation.
When an animation in added to the parametric, root delta position and delta rotation are stored per animation and also blended.
Every animation must loop perfectly but it’s also possible to add, during the preprocess, a loop detection.
That’s 2 levels of animation (locomotion + ground adaptation) and you need a 3rd level: the motion state.
It’s a state machine where each state can be lomotion, locomotion + additive animation, transition, …
Imagine a running character you control with your pad (locomotion). you press a button to jump into water (transition animation), once in water, the character swims and you control his yaw/pitch (second locomotion graph).

4 Likes

I ended up using the Unity Style blending formulas…

From Rune Skovbo Johansen Master’s Thesis

So i implemented these formulas in my BABYLON.AnimationState script component for Unity Style blend trees

  • 1D Simple Linear Interpolation
  • 2D Simple Directional
  • 2D Free Form Directional
  • 2D Free Form Cartesian

So far looks and works great (I am really stuck trying to calculate speed ratios for and weight multipliers for nested blend trees)

But going great so far using the formulas above… I get the same output as unity animator :slight_smile:

6 Likes

what i wanna say is, this is useful. i’m now implementing the character climbing which needs this Root Motion to match the position. and actually this is also useful for the character skills, for example for a skill “jump up and hit the targeted enemies far away”

1 Like

For the love of god… Is there no one out there to help here :slight_smile:

I cant believe im stuck here… I have actually re-created almost the entire unity mecanim animation system except two (you would think real easy) calculations that need to be done on the interpolated keyframes from the Babylon.Animation track.

1… How to mirror a pose from the animation key frames.
- I want to be able to just use one left and right animation. I to serialize
one walk right animation and use is for walking right, also use if for walking left… Just
mirrored. How To Mirror Animation Key Frames

2… How to take the root bone position values from the animation key frames and use that as
delta movement to actually move the mesh transform node and not the skeleton root bone?

Here is the scenario:

1… I have a buffer to hold the last position vector (this._lastMotionPosition)
2… I am reading the root bone position vector key frame value from animation track (this._lastMotionPosition)

and finally a delta position vector buffer that is the difference of last position minus current position (this._deltaPosition)

on every frame i update these values like so:

if (this._lastMotionPosition == null) this._lastMotionPosition = this._rootMotionPosition.clone();    
this._rootMotionPosition.subtractToRef(this._lastMotionPosition, this._deltaPosition);
this._lastMotionPosition.addInPlace(this._deltaPosition);

and i can move the mesh transform node like so:

this.transform.position.addInPlaceFromFloats(deltaPosition.x, 0, deltaPosition.z);

The problem is when the animation loops… the transform position goes back to beginning instead of being additive

If its playing a walk forward animation the mesh transform node should just keep moving forward in the world… But it does not… it loop back to start because the deltaPosition goes back to start value…

I obviously am not doing the whole additive thing right…

Yo @Deltakosh … Or maybe @sebavan@PatrickRyan

Can one of guys please help me out…
How do I mirror key frame pose values ???

And

How do i turn these root bone key frame positions to additive delta movements i can use to actually move/translate the mesh/transform node ???

Thanks As Always
:slight_smile:

Check out my root motion problem

1 Like

@MackeyK24, thank you for such a detailed video. I wonder if this._rootMotionPosition coming from Unity is resetting itself every time the animation finishes a loop?

If this._rootMotionPosition keeps its position at the end of an animation cycle and continues in the same direction, then I think your code would work?

captured (7)

Are you using animations from Mixamo? If so, the walk animation may be resetting its position every loop (if the In Place checkbox in Mixamo is unchecked). My guess is that Unity does some processing behind the scenes to make the character appear to be always walking forwards as shown at time 00:32 in your video. Maybe, we somehow need to add the distance traveled in one animation loop to this._rootMotionPosition.

Thank you for your hard work on this!

Yep its a Mixamo animation and yes it does reset… basically if i trace the z output…

it starts at zero and goes for 1.7 then hits the end and loops and the z output is zero again…

I dont know how Unity is converting that z movement from the animation key frame into to actual direction or velocity vector that is additive… Thats the main problem. I was hoping someone can see what im doing and show me how to make that movement additive so i can add the root motion to the parent transform node :slight_smile:

I wonder if it would be a scalable solution to read the position at the last frame (z = 1.7 as you mentioned) and then adding that this._rootMotionPosition before running

if (this._lastMotionPosition == null) this._lastMotionPosition = this._rootMotionPosition.clone();    
this._rootMotionPosition.subtractToRef(this._lastMotionPosition, this._deltaPosition);
this._lastMotionPosition.addInPlace(this._deltaPosition);

Just curious if there was a reason you wanted this translation/movement along with the animation? For a character controller, I thought it would be easier to have animations that didn’t move the whole character, since you could just add that movement in later if you wanted either by directly translating the mesh’s position or by using a physics engine. Though, perhaps for a cutscene, including horizontal movements along with the animation is preferred.

I don’t know how Unity works, but it would be funny if they created the illusion that the character is forever walking forward by keeping the character in place and just moving the ground beneath it :stuck_out_tongue:

Maybe it just takes the center of the bounding box at frame 0 / last frame, subtract the two and update the position property of the mesh with this vector at the end of the animation? Or instead of taking the center of the bbox, just takes position of the the root node at frame 0 / last frame.

Well, it seems like it is what @gbz is saying, but I would update the mesh.position property instead.

1 Like

So can you kind of pseudo that out for me explain what you mean 0 / last frame … please

I mean the first frame and the last frame of the animation (before looping). So you would calculate deltaPosition = (position_last_frame - position_first_frame) and add this to mesh.position juste before looping.

Holy Crap… I think i finally fixed it… @Evgeni_Popov was right about the looping…But i came up with a better way of reseting the last motion buffers. So on _frametime = 0… I simply reset the last motion buffers and it worked :slight_smile:

This is how i update root motion delta position and rotation

if (this.applyRootMotion === true && dirtyMotionMatrix != null) {
    this._rootMotionMatrix.decompose(this._rootMotionScaling, this._rootMotionRotation, this._rootMotionPosition);
    // ..
    // Reset Last Frame Buffers
    // ..
    if (this._frametime === 0) { 
        this._lastMotionPosition.copyFrom(this._rootMotionPosition);
        this._lastMotionRotation.copyFrom(this._rootMotionRotation);
    }
    // ..
    // Update Current Delta Position
    // ..
    this._rootMotionPosition.subtractToRef(this._lastMotionPosition, this._deltaPosition);
    this._lastMotionPosition.copyFrom(this._rootMotionPosition);
    // ..
    // TODO: Optimize - Update Current Delta Rotation
    // ..
    const q1:BABYLON.Quaternion = new BABYLON.Quaternion(0,0,0,0);
    const q2:BABYLON.Quaternion = new BABYLON.Quaternion(0,0,0,0);
    const v1:BABYLON.Vector3 = new BABYLON.Vector3(0,0,0);
    BABYLON.Quaternion.InverseToRef(this._lastMotionRotation, q1);
    q1.multiplyToRef(this._rootMotionRotation, q2);
    q2.toEulerAnglesToRef(v1);
    BABYLON.Quaternion.FromEulerAnglesToRef(0, v1.y, 0, this._deltaRotation);
    this._lastMotionRotation.multiplyToRef(this._deltaRotation, this._lastMotionRotation);
}

The animation state component stores that root motion and can be be access via the:

  • getDeltaSpeed()
  • getDeletaPosition()
  • getDeltaRotation()

Using a Forward Only type movement controller:

const locomotionSpeed:number = (this.animationState != null) ? this.animationState.getDeltaSpeed() : 0;
this.transform.forward.scaleToRef(locomotionSpeed, this.playerMovementFinalVector);

or for strafing type movement:

const deltaPosition:BABYLON.Vector3 = (this.animationState != null) ? this.animationState.getDeltaPosition() : null;
if (deltaPosition != null) this.playerMovementFinalVector.copyFrom(deltaPosition);

Then the playerMovementFinalVector can easily be used with the character controller or even used to directly translate the mesh position

if (this.characterController != null) {
    this.isCharacterGrounded = this.characterController.isGrounded();
    this.isCharacterJumpFrame = false;
    if (this.isCharacterGrounded === true) {
        this.isCharacterJumpFrame = (BABYLON.SceneManager.GetKeyboardInput(this.keyboardJump) || BABYLON.SceneManager.GetGamepadButtonInput(this.buttonJump));
        this.characterController.move(this.playerMovementFinalVector);
        if (this.isCharacterJumpFrame === true) {
            this.characterController.jump(this.jumpingVelocity);
        }
    }
}

Works Beautifully… No More Animation Foot Slide… And now animation that speed up and slow down within the actual animation play perfectly synced.

Thanks for the help in narrowing down the root motion issue :slight_smile:

P.S.

I still need help with the mirror animation key frame issue:slight_smile:

3 Likes

Root Motion Is Beautiful… You just gotta figure out how and most importantly when to use the root motion from the animation to drive the movement of the parent mesh. And its mostly for your grounded locomotion animations

But again, you can choose to implement the root motion any way you want. You can use your normal movement code but use animator.getDeltaSpeed() to scale your movement direction and your movement will always match the motion of the animation.

You can also use animator.getDeltaPosition() as velocity input for your character controller move function or your rigidbody set linear velocity function

Or can can even use the root motion deltas to directly translate and rotate the transform as well… Like So:

module PROJECT {
    /**
    * Babylon Script Component
    * @class TestRootMotion
    */
    export class TestRootMotion extends BABYLON.ScriptComponent {
        private animationControllerNode:string = "YBot";
        protected m_animator:BABYLON.AnimationState = null;
        public constructor(transform: BABYLON.TransformNode, scene: BABYLON.Scene, properties: any = {}) {
            super(transform, scene, properties);
            this.animationControllerNode = this.getProperty("animationControllerNode", this.animationControllerNode);
        }

        protected start(): void {
            if (this.animationControllerNode != null && this.animationControllerNode !== "") {
                const botmesh = this.getChildNode(this.animationControllerNode, BABYLON.SearchType.ExactMatch, false) as BABYLON.AbstractMesh;
                if (botmesh != null) {
                    this.m_animator = BABYLON.SceneManager.FindScriptComponent(botmesh, "BABYLON.AnimationState");
                    if (this.m_animator == null) {
                        BABYLON.Tools.Warn("Test Root Motion: Failed to locate animation state for bot mesh: " + botmesh.name);
                    }
                } else {
                    BABYLON.Tools.Warn("Test Root Motion: Failed to locate bot mesh: " + this.animationControllerNode);
                }
            }
        }

        protected update(): void {
            if (this.m_animator != null) {
                const deltaPosition:BABYLON.Vector3 = this.m_animator.getDeltaPosition();
                const deltaRotation:BABYLON.Quaternion = this.m_animator.getDeltaRotation();
                // ..
                // Test Transform Node Translation
                // ..
                this.transform.position.addInPlace(deltaPosition);
                this.transform.rotationQuaternion.multiplyInPlace(deltaRotation);
            }
        }
    }
}
3 Likes