Camera bugs related to delta time

Hi, Babylon team!

A few weeks ago I reported an issue with the camera related to inertia:

Thank you all who helped implement a temporary solution!

Since then, I’ve continued to work on the camera experience for my company and I’ve found a few more issues that I’d like to report here. It was mentioned in the other forum post that the Babylon team has a camera overhaul planned. I want to make sure that these issues are included in that, and are hopefully fixed.

1. The camera applies inertia without respect to delta time.

Take a look at the checkInputs method of the arc rotate camera (used for reference; issue exists for all camera types).

This code is meant to apply inertia over time. The issue is that “over time” is how fast the checkInputs method is called. At 240 FPS, the inertia will dissipate four times faster than 60 FPS.

The solution is to include delta time:

const deltaTimeScale = this.getEngine().getDeltaTime() / Frequency;
const inertia = this.inertia ** deltaTimeScale;
this.inertialAlphaOffset *= inertia;
this.inertialBetaOffset *= inertia;
this.inertialRadiusOffset *= inertia;

(the double star ** is intentional)

2. The camera inputs don’t respect delta time

As far as I can tell, this affects all the camera input classes. I’ll use ArcRotateCameraPointersInput to illustrate though. The onTouch method in particular.

The onTouch method adds to the camera inertia every time it’s called. This is then applied in the camera checkInputs method. The problem here is that onTouch is called by the browser (per event), while checkInputs is called by Babylon (per frame). These operations can easily happen out of sync with one another. Say, the browser is measuring pointer events at 60 FPS and the render engine is running at 30 FPS. Ultimately, this causes the camera to move at different speeds depending on frame rate when inertia is on.

The solution is to include delta time:

public override onTouch(point: Nullable<PointerTouch>, offsetX: number, offsetY: number): void {
    const deltaTimeScale = this.camera.getEngine().getDeltaTime() / Frequency;
    if (this.panningSensibility !== 0 && ((this._ctrlKey && this.camera._useCtrlForPanning) || this._isPanClick)) {
        this.camera.inertialPanningX += -offsetX / this.panningSensibility * deltaTimeScale;
        this.camera.inertialPanningY += offsetY / this.panningSensibility * deltaTimeScale;
    } else {
        this.camera.inertialAlphaOffset -= offsetX / this.angularSensibilityX * deltaTimeScale;
        this.camera.inertialBetaOffset -= offsetY / this.angularSensibilityY * deltaTimeScale;
    }
}

3. The camera inertia cutoff doesn’t scale with delta time

This is the issue that I originally reported in this post. I’m putting it here so that it is included with the others since they all, in part, relate to each other.


You may have noticed the Frequency constant used in the above solutions. This is defined as follows:

const Frequency = 1000 / 60;

While not strictly necessary to fix the above issues (delta time is the only necessary part), without it all the numbers would be scaled. This would cause everyone’s existing projects to behave very different. By including this frequency, inertia will behave exactly like it does at 60 FPS regardless of the actual FPS.


I can put together some playgrounds with the above solutions tomorrow. I’ve already implemented these solutions for my company’s project though. You can see it in action here:

cc @georgie

thank you! @amoebachant and I will discuss when we are back in office next week! Good timing as we were discussing this topic yesterday :slight_smile:

1 Like

Hi.

I’ll add my 5 cents.

The AutoRotateBehavior is also driven by checking inputs via camera.onAfterCheckInputsObservable

It kinda does respect delta time though. Although not via Engine.getDeltaTime() but with its own this._lastFrameTime = now

However, when scene rendering is suspended by Engine.stopRenderLoop (e.g. in some visibility checking), camera continues to check inputs and the delta time accumulates. When rendering is resumed after awhile, camera jumps to unpredictable position.

Disabling camera does not help: disabled cameras still check inputs and run their behaviors.
Detaching controls also does not help, because the last frame time is recorded.

Workaround on app-level is to add additional handler like this:

camera.onEnabledStateChangedObservable.add(() => {
    if (camera.isEnabled()) {
        camera.useAutoRotationBehavior = myoptions.autoSpin;
        camera.attachControl(); 
    } else { 
        camera.useAutoRotationBehavior = false;
        camera.detachControl();
    }
});

And the visibility checker should enable/disable cameras (and hope that camera won’t change in suspended scene)

Hey @reedsyllas and @qwiglydee, thanks for reaching out! As @georgie mentioned, we’ve been talking about this area recently, and your observations and suggestions are very helpful. We’d like to address these issues with a fresh approach that also simplifies the camera motion APIs at the same time, and these thoughts will be very helpful as we dig into that. You can track the work using this issue and please feel free to leave any other feedback or ideas you have there.

Thanks again!

2 Likes

@amoebachant

Any news about the progress?

Camera inertia needs upgrading.

Hi @a_bolog the new Geospatial Camera uses a different approach for inertia and velocity. All of the calculations for movement / speed live in the CameraMovement class – which takes the velocity from the previous frame and uses it for the next frame (applying inertial decay). This approach should be framerate agnostic. And the units for the speed and movement vars are more intentionally defined (ex: zoomSpeed: Desired coordinate unit movement per input pixel when zooming) vs arbitrary numbers.

We’d like to test out this system and if it meets our needs for fixing bugs related to delta time, we can port the other cameras to using the CameraMovement system.

You can test out the camera using this playground, and read more about the camera in the docs
Geospatial Camera | Babylon.js Documentation. The docstrings in the code also provide a bunch of details about the units used for movement / speed variables.

Please test out the GeospatialCamera and let us know if this approach would solve the concerns!

Hey @georgie , that looks great, the inertia is consistent across multiple framerates.

But how can we use this inertia with FreeCamera and FPS moving with arrow keys?

@a_bolog great! I will work on introducing the camera movement system to the other cameras and let you know once available

3 Likes

@a_bolog this PR has the change, however I’m going to wait to merge it until after the Babylon 9 release (as it touches critical camera code)
Introduce framerate-independent movement to cameras by georginahalpern · Pull Request #18030 · BabylonJS/Babylon.js

You can test using the snapshot link from the PR and if you notice any issues I can address at checkin time!
Babylon.js Playground

@georgie Thanks, that’s great. I’ve noticed a little bug, the inertia doesn’t work when enterPointerLock().
Check this out: https://playground.babylonjs.com/?snapshot=refs/pull/18030/merge#YN2Y2C#1

And another thing, when moving with arrows in FreeCamera, the speed inertia is significantly slower when 15FPS vs 120FPS.

Also moving diagonally is a bit strange. When walking forward and then moving left right, is somehow overwriting the inertia.

1 Like

thanks for testing! will look into the above and perform further testing on the branch as well