Hello everyone,
This work is inspired by the topic BABYLON.Animation is not synchronized with the rendering loop? By default the BABYLON animations are not synchronized with a gaming loop, but sometimes we desperately need those. I will explain later when and why.
I created a separate class that will help to make BABYLON animations work in synchronous way. It’s a demo, that you will be able to adapt and extend for your project needs.
My approach relies on some JavaScript very specific features. Before proceeding, please be sure that a) you are familiar with such features as difference between var and let, b) you know when JS copies values and when it copies links during assignments, c) and you know how JS closures work. This topic is for PRO! 
Let the game begin!
Foreword. When we need synchronous animations?
Animations can be:
a) Not synced. Such animations progresses apart from gaming loop. For those animations it doesn’t matter how many rendering events passed and how many frames the animation lost. Such animation tends to finish within time range of real world (for example, 5 seconds of real world time). For example, if you want to animate the watch that shows how long current gaming round goes, then I think you want to use this approach. In JavaScript not synced animations can be implemented using setInterval/setTimeout. And I guess, this is how BABYLON animations work.
b) Synced. Such animations progresses on every determined amount of game state update events (or ticks). Do not be confused between rendering events and game state update events. Frames per seconds (aka FPS) is the amount of rendering events per seconds. Ticks per seconds (aka TPS) is the amount of game state update events per seconds. FPS could be different on every computer (it may depend on monitor refresh rate, for example), but TPS should be the same, it is determined by developer in the code. Usually synced animations progress on every tick if there is no TPS lock or other rate function. Animation duration should be determined by the number of ticks instead of number of seconds. For example, if you want an animation for approximately 5 seconds and your TPS is 60, then you will probably want to create animation consisting of 60 * 5 = 300 ticks. Such animations don’t care about the real time, it will last exactly the determined number of ticks, despite how long these ticks will be. Game state update may take longer time on slow computers. That means that you should never think about such animations in terms of ticks/tps = realTime. Compose such animations only relying on desired number of ticks that will fit the game flow. However, such animations have benefits. For example, if the length difference between two animations is 200 ticks, then this difference will always be like that. No exceptions. Synced! This fact is very critical for animations that affect gameplay (and ability to win): weapon switching, skill reloading. Synced animations also allow to pause the game correctly. You might be sure that the game state will be exactly the same for the synced processes before and after the pause. In order to pause the game, you just need to pause the game loop and all synced processes will be paused automatically.
You can make animations synced with rendering events (based on fps) or game state update events (based on tps). For first one you should use onBeforeRenderObservable. But such animations may run at different velocity, depending on monitor refresh rate or fps. For second one you should use onBeforeStepObservable, tps is hardcoded parameter. Always use the second one for gameplay critical processes.
When to use?
When animations are not important to game state it still could be not synced animations, because it’s easier to use them in BabylonJS. I assume that you will not worry much if blinking advertisement on the building or weapon bouncing will lose some frames after the pause, it doesn’t affect the gameplay at all and probably will not be noticed by the most of players.
However, such things as weapon switching, firing and reloading determine when you will be able to do the next shot, your life depends on it. Synced approach will provide the more consistent game state each round and your game strategy will be more protected from random factors, such as CPU performance.
Issues
Can we be sure that not synced animations always corresponds to real time? Of course no. JavaScript is single-threaded. If you have intensive function running right now, then your setTimout(myFunc, 5000) may fire much later.
How the screen rendering rate should be synchronized with animation tick rate? In the middle 10’s the 60 FPS was the de facto standard for screen refreshing rate. Running the game with higher FPS had no sense, so all synced animations could be bounded by 60 ticks per second. Now when the high Hz monitors became widely available, we run into a trouble. We don’t know the target refresh rate and we don’t know how the particular user PC will be able to maintain such rate. The 200 frames (screen updates) could take about 3.33 and 1.22 seconds on 60 and 164 frames per second. But the amount of frames in our synced animation (game state updates) and its duration will be the same all the time. Of course, we can just bound our animations speed by old good 60 TPS. But when you are rendering the game at 164 FPS and your computer plays it smoothly, the 60 TPS animations look very unnatural. For example, you can even see that in Doom Eternal during some glory kills (some boss fights). So, yeah, this is a pretty big concern. But from my experience, 200 TPS is a good ticker rate for BabylonJS projects, because a) it works pretty good with physics calculations and collision detection (at least with CannonJS), b) the time between ticks is a precise value (1/200 = 0.005), c) it’s not very big value for modern computers (in general the higher TPS gives better precision, but CPU performance and your code optimization is the limit).
The synced animations may create a gap with a real time. If your animation is long and it suppose to produce sounds (for example, long weapon reloading animation), you can face the sound out-of-sync problem. So, advise here is not to use the single audio file to reproduce the whole sound sequence. Just split your animation into small parts and play each sound separately, when the target frame is reached. This will protect you even if the target PC did the bad performance at some point. This is fair advise for both types of animations, by the way.
0. What we have?
So, the BABYLON already has a good way to describe the animations: target property, keys and some other parameters. All we need is to make it run synchronously (I dream we could have just a flag in Babylon for that). So, all we need to change is the way how those animations are processed.
First of all, don’t forget to enable deterministic lockstep in BabylonJS:
var engine = new BABYLON.Engine(canvas, true, {deterministicLockstep: true, timeStep: TICK_TIME});
TICK_TIME is the delay between ticks (1/TPS). It’s up to you to decide which value to use. For example, timeStep for 200 TPS will be 0.005. Be aware, it also affects physics calculation steps. But usually you want to have the physics steps and synced processes at the same rate.
1. Preparation. Interpolation map.
Most key framed animations are based on interpolation. We calculate how we should change the value on each frame between two key frames. Of course, the value that we should add or subtract could be different on each segment of animation. For this purpose I create a so called animationMap from animation key frames. This map might be named also as incremental map or interpolation map.
It does the following:
// converts this
const TICKS_PER_SECOND = 200;
const keys = [];
keys.push({
frame: 0,
value: -3
});
keys.push({
frame: 2 * TICKS_PER_SECOND,
value: 3
});
keys.push({
frame: 3 * TICKS_PER_SECOND,
value: 3
});
keys.push({
frame: 5 * TICKS_PER_SECOND,
value: -3
});
// into this
{
400: 0.05
600: 0
1000: -0.05
}
Yeah, it just shows how much we should add to our target property before reaching the next key frame. So, before the frame 400 we add 0.05, before frame 600 we add 0, and before frame 1000 we add -0.05.
Don’t forget to create animationMap, before playing the animation. I recommend to do that right after keys. For example:
sphereAnimation.setKeys(keys);
// !!! Additional important step.
sphereAnimation.animationMap = AnimationHelpers.getAnimationMap(sphereAnimation);
sphere.animations.push(sphereAnimation);
Here is the function for that:
static getAnimationMap(animation) {
const animationMap = {};
const keys = animation.getKeys();
if (animation.dataType === BABYLON.Animation.ANIMATIONTYPE_FLOAT) {
for (let k = 0; k < keys.length - 1; k++) {
const nextKeyFrame = keys[k+1].frame;
const incrementValue = (keys[k+1].value - keys[k].value) / (Math.floor(nextKeyFrame) - Math.floor(keys[k].frame));
animationMap[Math.floor(nextKeyFrame)] = incrementValue;
}
}
else if (animation.dataType === BABYLON.Animation.ANIMATIONTYPE_VECTOR3) {
for (let k = 0; k < keys.length - 1; k++) {
const nextKeyFrame = keys[k+1].frame;
const incrementValue = keys[k+1].value.subtract(keys[k].value).scale(1/(Math.floor(nextKeyFrame) - Math.floor(keys[k].frame)));
animationMap[Math.floor(nextKeyFrame)] = incrementValue;
}
}
else if (animation.dataType === BABYLON.Animation.ANIMATIONTYPE_MATRIX) {
for (let k = 0; k < keys.length - 1; k++) {
const nextKeyFrame = keys[k+1].frame;
const incrementValue = BABYLON.Matrix.Lerp(keys[k].value, keys[k+1].value, 1/(Math.floor(nextKeyFrame) - Math.floor(keys[k].frame)));
animationMap[Math.floor(nextKeyFrame)] = incrementValue;
}
}
return animationMap;
}
For now it supports types: FLOAT, VECTOR3 and MATRIX. You can add other types later. Be careful we can not use Lerp function from BABYLON libraries here. Because Lerp in BabylonJS is initialValue + incrementValue, but we need only the incrementValue for the map. Lerp is OK for MATRIX though, because MATRIX is used by imported animations mainly which describes every frame with no gaps. So later we will just overwrite the target property instead of incrementing it. However this will not work if our MATRIX animation has gaps between key frames.
You may notice some code duplication in the function above, but it’s intentional. For CPU it’s harder to cache for->if than if->for. I am not sure how it’s right for JavaScript, but in general it’s better to avoid if inside of for in performance critical places.
Since we are here, let’s check how we will modify the target property soon.
static modifyTargetProperty(obj, animation, value) {
const targetPropertyPath = animation.targetPropertyPath;
const length = targetPropertyPath.length - 1;
for (var i = 0; i < length; i++) {
obj = obj[targetPropertyPath[i]];
}
if (animation.dataType === BABYLON.Animation.ANIMATIONTYPE_FLOAT) {
obj[targetPropertyPath[i]] += value;
}
else if (animation.dataType === BABYLON.Animation.ANIMATIONTYPE_VECTOR3) {
obj[targetPropertyPath[i]] = obj[targetPropertyPath[i]].add(value);
}
else if (animation.dataType === BABYLON.Animation.ANIMATIONTYPE_MATRIX) {
obj.updateMatrix(value, false, true);
}
}
As I said, we increment FLOAT and VECTOR3 and overwrite the MATRIX. Be careful, we use var in the loop intentionally, because we want to use this value after the loop. We don’t traverse the property path to the end, otherwise we will receive the value itself instead of the ‘place’ where it should be saved. It’s PRO coding. 
And here is the helper function that extracts current IncrementValue from the animationMap. Nothing hard, but I sort keys for extra safety. timer here represents the current frame.
function getAnimationCurrentStep(animationMap, timer) {
const keys = Object.keys(animationMap).map((key) => parseInt(key)).sort((a, b) => a - b);
const maxK = keys.length - 1;
let k = 0;
while (timer > keys[k] && k < maxK) {
k++;
}
return animationMap[keys[k].toString()];
}
2. Main processing.
In BabylonJS we had an Animatable object. It is a special control object that keeps a link to running animation and allows us to control this animation. We need something similar for our synced animations, however I decided that we don’t need to recreate the interface of original Animatable object completely. Let’s just create only what we need for now. A helper function for that is:
static generateAnimatableObject() {
return {
timer: 0,
paused: false,
stoped: false,
pause: function () {
this.paused = true;
},
stop: function () {
this.stoped = true;
},
reset: function () {
this.timer = 0;
}
};
}
And here we go. The main function:
static beginAnimation(scene, animation, animatedObject, startFrame, endFrame, onAnimationEnd, animatable) {
animatable = animatable || AnimationHelpers.generateAnimatableObject();
const keys = animation.getKeys();
const lastFrame = keys[keys.length - 1].frame;
endFrame = endFrame > lastFrame ? lastFrame : endFrame;
animatable.timer = startFrame;
const frameDuration = 1/animation.framePerSecond;
let lastFrameTime = 0;
animation.runtimeAnimations.push(animatable);
const animationHandler = function () {
if (animatable.stoped) {
scene.onBeforeStepObservable.removeCallback(animationHandler);
animation.runtimeAnimations.splice(animation.runtimeAnimations.indexOf(animatable), 1);
}
if (!animatable.paused && AnimationHelpers.frameTimePassed(lastFrameTime, frameDuration)) {
animatable.timer++;
const incrementValue = getAnimationCurrentStep(animation.animationMap, animatable.timer);
lastFrameTime = (new Date()).getTime();
if (incrementValue !== 0) {
AnimationHelpers.modifyTargetProperty(animatedObject, animation, incrementValue)
}
if (animatable.timer >= endFrame) {
scene.onBeforeStepObservable.removeCallback(animationHandler);
animation.runtimeAnimations.splice(animation.runtimeAnimations.indexOf(animatable), 1);
if (onAnimationEnd) {
onAnimationEnd();
}
}
}
}
scene.onBeforeStepObservable.add(animationHandler);
return animatable;
}
First it generates the control object if it wasn’t provided. Then setups important variables.
Important step is animation.runtimeAnimations.push(animatable);. We may sometimes use runtimeAnimations to check that animation is currently running, don’t forget about that. Please also don’t forget that our animatable object is different, comparing to BABYLON Animatable.
Then the function creates animationHandler. It’s a callback function that runs on every tick, it increments timer and target property value accordingly. It also checks stoped and paused flags and fires onAnimationEnd callback when it’s time. animationHandler can also detach itself from ticker, when no longer needed.
frameTimePassed is a simple FPS/TPS lock:
static frameTimePassed (lastFrameTime, frameDuration) {
return (((new Date()).getTime() - lastFrameTime) / 1000) > frameDuration;
}
Saving concerns.
mesh.serialize saves animations as well. It would be bad for our case. Because after loading we will have two different animation objects. One is from the code and one is from the save file. The animation inside of animated object and animation created in the code are not the same anymore.
Probably the easiest way is to do not save the synced animation:
for (let a = mesh.animations.length - 1; a >= 0; a--) {
if (mesh.animations[a].animationMap) {
serializationObject.animations.splice(a, 1);
}
}
Just save animation name and current timer value somewhere. On loading just do again sphere.animations.push(sphereAnimation); and run beginAnimation with saved timer value as startFrame.
3. Group animation.
Most of the time we need to animate more than one property of animated object. For this purposes BabylonJS has grouped animations, which are processed by beginDirectAnimation function. And we need synced version of it as well.
First we need more complex control object. The helper function below creates the grouped control object of given capacity.
static generateDirectAnimatableObject(n) {
const animatables = [];
for (let i = 0; i < n; i++) {
animatables.push(AnimationHelpers.generateAnimatableObject());
}
return {
animatables,
pause: function () {
for (let i = 0; i < n; i++) {
this.animatables[i].pause();
}
},
stop: function () {
for (let i = 0; i < n; i++) {
this.animatables[i].stop();
}
},
reset: function () {
for (let i = 0; i < n; i++) {
this.animatables[i].reset();
}
}
}
}
And below is the function itself.
static beginDirectAnimation(scene, animations, animatedObject, startFrame, endFrame, onAnimationEnd, animatable) {
const length = animations.length;
animatable = animatable || AnimationHelpers.generateDirectAnimatableObject(length);
let callbackCounter = 0;
for (let a = 0; a < length; a++) {
animatedObject = animations[a].animatedObject || animatedObject;
AnimationHelpers.beginAnimation(scene, animations[a], animatedObject, startFrame, endFrame, function () {
callbackCounter++;
if (callbackCounter >= length && onAnimationEnd) {
onAnimationEnd();
}
}, animatable.animatables[a]);
}
return animatable;
}
It’s pretty straightforward. We launch beginAnimation on every included animation passing to them the dedicated control object from grouped control object. The only interesting thing is how we fire the callback. We count how much of animations have finished. When all of them have finished we fire the callback of grouped animation.
4. Improvements and alternatives.
As you can see, my example doesn’t support animation speed and loop parameters. But it is easy to add them if you need.
Instead of creating incremental map we can just try to generate every missing frame between key frames, then we can just overwrite the target property on every tick. Probably it will increase the processing performance a little bit, however it will increase the preparation time and consuming of memory as well. It will also allow us to use Lerp function everywhere, like we did it for MATRIX values. Probably I should think about it more seriously. However, we will lose the ability to skip the identical frames without comparing them. This is what we can do now if incrementValue is 0.
5. Objects Violations.
Some people might be not happy that we substitute some BabylonJS objects with objects that don’t obey the initial interface. TypeScript projects will be espicially angry about that. Well, to keep it simple I just implemented what we really need right now. If someone is not happy about that, it’s possible to implement the full interface or to extend the initial BabylonJS objects.
6. Demo.
Of course I did a playground. Unfortunately, I don’t know how to control the rendering loop in playground, that’s why my demo doesn’t show anything that original BABYLON animations can not do. However I left comments on how it can be used with rendering loop. Particularly like this:
engine.runRenderLoop(function () {
if (!paused) {
currentScene.render(); // Pause the main scene.
}
mainMenu.mainMenuScene.render(); // But keep the main menu always alive.
});
Manipulating the paused flag here you can stop and resume all synced animations instantly. No additional actions are needed.
I have put the whole AnimationHelpers class to demo, despite that I don’t use all features in this demo. But we have covered all of them in the text above.
The Link: https://www.babylonjs-playground.com/#EWLG0L#2
UPDATED ON 2022-08-22:
Since this topic one can use as learning material, I updated the information above according newly learned information. Previously it contained some conceptual flaws, I fixed them. Probably not all. 
Update summary:
- Explained difference between rendering events and game state update events. I replaced all
onBeforeRenderObservablebyonBeforeStepObservable. - Added
Math.floor()around animation frame numbers ingetAnimationMapfunction. It will prevent the error if the animation key frame was declared as decimal number. Before that interpolation worked incorrectly with such key frames, because it generated steps with slightly higher or lower values. AlsoMath.floor()is better here thanMath.round(). If our last key frame is a decimal, we ensure that animation will give the last target value to the target object before the animation will be cut off on the last frame.
- Updated playground link.