Introducing Animation Retargeting

:tada: Introducing Animation Retargeting — Give Any Animation to Any Character!

Hey Babylon.js community! :waving_hand:

We are thrilled to announce a brand-new feature landing in Babylon.js v9.0: Animation Retargeting! This is something many of you have been asking for, and it’s finally here — fully integrated and ready to use.


:thinking: What is animation retargeting?

In a nutshell: take an animation made for one character and apply it to a completely different character. Different skeleton, different proportions, different bone naming — no problem! No more authoring unique animations for every character in your scene. One walk cycle, one combat combo, one dance → works on all of them. :sparkles:


:hammer_and_wrench: The interactive Retargeting Tool

The best way to get started is the Animation Retargeting Tool — an interactive playground that requires zero code.

The tool gives you a split viewport — your target character on the left, the source animation skeleton on the right — plus a full control panel:

  • Built-in characters and animations to experiment with right away
  • Bone Gizmos to interactively adjust the rest pose of both the avatar and the source animation in-browser, to align skeletons before retargeting
  • All IRetargetOptions exposed as checkboxes and dropdowns
  • A Console that shows any bone-matching warnings after retargeting
  • An Export to Playground button that generates a standalone, ready-to-run code snippet capturing your exact setup — including all gizmo adjustments and bone name remappings!

:gear: The API

Prefer to work in code? The new AnimatorAvatar class exposes retargetAnimationGroup():

const avatar = new BABYLON.AnimatorAvatar("hero", heroRootNode);

const retargeted = avatar.retargetAnimationGroup(sourceGroup, {
    animationGroupName: "hero_walk",
    fixRootPosition: true,
    fixGroundReference: true,
    rootNodeName: "Hips",
    groundReferenceNodeName: "LeftFoot",
    mapNodeNames: new Map([
        ["mixamorig:Hips", "Hips"],
        ["mixamorig:LeftFoot", "LeftFoot"],
        ["mixamorig:RightFoot", "RightFoot"],
    ]),
});

retargeted.play(true);

Under the hood the system does several smart things automatically:

  1. Name-based bone matching — with an optional mapNodeNames table for rigs that use different conventions (like Mixamo’s mixamorig: prefix)
  2. Keyframe transform compensation — mathematically remaps each keyframe from the source bone’s reference frame into the target bone’s frame, so world-space motion is preserved
  3. Root position scaling — scales root displacement proportionally so a shorter character doesn’t slide and a taller one doesn’t take tiny steps
  4. Ground reference correction — keeps feet planted; includes a dynamic mode that handles any bone temporarily dipping lower than the reference bone (e.g. during a kick or crouch)
  5. Morph target support — facial and shape-key animations retarget too!

:video_game: See it in action

Live demo — a hip-hop animation retargeted onto a character: playground #K1QFZD#2


:open_book: Full documentation


A big thank you to everyone who contributed ideas and feedback! We can’t wait to see the characters and animation libraries you’ll build with this. Drop your questions, feedback, and cool demos right here — we’re all ears! :rocket:

(post written by AI)

20 Likes

Congrats, this is a feature that I’ve been hoping for ages! Great to see these kind of improvements, which really benefit the gaming scene, being baked into the engine! :star_struck:

I can’t keep up with all the awesome features, I have yet to properly use the frame graph haha.

Anyhow that’s going to be very useful! Now we can just load a bunch of animations once and use them for any number of NPC model we want :ok_hand:

Thinking aloud here while sketching a new work flow.

A - Lead char

  1. Pick one humanoid’ish “lead” character model
  2. Upload to Mixamo
  3. Download animations as required
  4. [optional] Adapt, correct, or change animations

B - Additional chars

  1. Acquire arbitrary humanoid’ish character model
  2. Normalize using the Mixamo upload tool
  3. Download normalizsed T-Pose (skeleton)

C - Build

  1. Foreach additional char’s skeleton retarget by lead char
  2. Write retargeted animations (#todo export as glb from headless engine)

D - In-game

  • No change :money_mouth_face: in workflow if animations are already per skeleton
    e.g. dir struct: ../HumanoidDefault/animations*.*; ../HumanoidBigGuy/animations*.*

OMG!! :exploding_head:

AA

  1. Acquire arbitrary humanoid’ish animation from a source other than Mixamo!!
  2. Upload lead mesh to non-Mixamo provider (e.g. AccuRig, #todo check license)
  3. Get “external” skeleton with animation
  4. retarget back to lead (-> C2/A3)
  5. Now external animation is lead animation which can be retargeted to additional chars
1 Like

It’s a super cool feature.
I think it’s going to be useful for my editor to redo the system I made in my character editor. Currently I make a simple copy from a source that I apply to each character who has the same number of bones and the same mesh.
But with this system, the size of the character or the number of bones no longer matters.
It really great.

Thank you for this new feature.

1 Like

I tested the ART (Animation Retargeting Tool) with a cube parented to single bone exported out as traditional .babylon file. Console complains that the avatarSkeleton is undefined. It appears that AnimatorAvatar can’t find the skeleton? or is it not meant to work with traditional .babylon exports?

From the api: This node and its descendants will be scanned for meshes, skeletons and morph target managers. So I assumed it should have worked.

Repro here (just in case): Babylon.js Playground

( post written by NI (natural intelligence) )

I think I read that it only supports TransformNode bones rather than Bone bones.


Found it: Note: The current implementation only supports source animation groups that animate TransformNode objects, not bones directly. This is the standard case for glTF assets, where animations always target transform nodes.

erm, I hadn’t even gone as far as exporting a glb Animation yet :sweat_smile:, which I think was what the note meant. Chk out the repro PG? Maybe I misunderstood how the tool is supposed to work, another set of eyes would help!

You’re right, the root node was not processed. Here’s the fix:

Oops, sorry for the confusion. Not enough coffee yet :face_with_crossed_out_eyes: (or too much :thinking:)

If you fix this way, glb imports now fail for AnimatorAvatar. Console complains getTotalVertices is not a function.

Repro: Babylon.js Playground

I think the predicate chk needs to be kept for glb?

Yes, I’ve already fixed it in my local repository, but I have other fixes in progress! I’ll create a PR shortly.

[EDIT] Fixed in Animation Retargeting: Fix regression in #17994 by Popov72 · Pull Request #17996 · BabylonJS/Babylon.js · GitHub

3 Likes

Thank you for this. I’ve wanted this for ages.

2 Likes

I had done a whole app with and interface and a python backend to accomplish this. The fact we can do it now straight through BJS is epic. The whole community surly appreciates this. Currently Im working on an editor that accepts plugins for defining available entities, components and editor panels etc the whole shabang. Currently its only importing Havok physics as a extension package, but now I’m going to include this and the panels/editor stuff to easily create the remapping. Thank you very much for making this possible.

1 Like