Camera inertia causes inconsistent motion

Bugs
1

When inertia is enabled, rotation and panning behaves unpredictably. Identical inputs (i.e. dragging with the mouse) can result in different camera positions. In comparison, with inertia off the same input reliably gives the same motion.

To prove this (and remove any human error), I created a macro that mouses down, moves 50px to the right, waits 200ms, then moves 50px to the left. With inertia on, the camera doesn’t return to it’s original position. Even worse, it doesn’t respond to mouse movement consistently at all. Sometimes it overshoots, other times it doesn’t. I can only speculate that FPS is a factor.

Video for reference with inertia enabled:

Playground with inertia enabled:

Video for reference with inertia disabled:

Playground with inertia disabled:

Using the same macro in three.js (not shown) results in the expected, reliable motion when inertia is on. Here’s the three.js example I was using to test this: three.js examples.

Additional context:

  1. BabylonJS and three.js both show the render running at 240 FPS (matching my monitor) but it periodically flickers to 239 FPS.
  2. Both videos (inertia on vs off) show the mouse being moved exclusively by the macro; there’s no human input to taint results.
1 Like
2

Hi @reedsyllas, thanks for reaching out and for the detailed explanation! Your test methodology sounds helpful - is the macro something you can share so I can reproduce exactly what you are seeing? Thanks!

3

Absolutely. It’s an AutoHotkey v2 script: Mouse move macro - Pastebin.com. Due note that it’s bound to F5 and set to jump the mouse to x 1920 y 540 on your screen before dragging so please adjust accordingly.

Edit: @amoebachant I realize I should have clarified that AutoHotkey is a Windows-only program, but I’m sure you could extrapolate to a Mac/Linux macro software. Here are the steps taken:

  1. On “F5” keypress:
  2. Move mouse to x 1920, y 540 (for consistency in my testing, probably not important though)
  3. Wait 200ms
  4. Press mouse left down
  5. Move mouse to right 50px
  6. Wait 200ms
  7. Move mouse to left 50px
  8. Release mouse left
4

Initially, I thought that it could have been the browser (chrome in my case), javascript reporting pointer events wrong, the drag being too subtle to measure inertia properly, or something else that’s out of Babylon’s control.

Though, as mentioned, I tested three.js and didn’t notice the same behavior with their camera which eliminates those cases, I think.

6

Hi @reedsyllas, thanks for all of the info, I was able to repro what you were seeing. It looks like your script is starting the second drag before the inertia of the original drag has totally dissipated, so it doesn’t always get back to the same starting position, depending on which frame the second drag happens to start in.

When I altered your script to wait a little longer until starting the second drag, it returned to the starting position each time. Try changing the second sleep to 500 and see if it returns to the original position as expected.

Thanks!

7

Hi @amoebachant, thanks for looking into this. I really appreciate it.

Regarding your instructions, I tried increasing both delays to a couple different values (500, 1000, 2000). Sadly, I experienced the same issue.

I don’t want to just reply ‘that didn’t work’, so I tried tweaking each variable of the macro and tested it to see what would happen. I’m now confident I know where the randomness comes from, but perhaps more importantly, why I have such a hard time using Babylon’s camera controller.

There is some kind of input threshold before inertia activates. If the mouse is moving fast, the camera has inertia as expected. However, when below a threshold, the camera has no inertia at all. I tested and confirmed that it (that is, moving the mouse slow enough to be under the threshold) yields identical motion when camera.inertia = 0 vs on.

This threshold is more apparent/reproducible in this playground:

^^^^
Here, the inertia is set to 1. Thus, when you get the camera moving, it should never stop. But, if you drag slowly, you can see that the camera sticks to your cursor. No inertia. Only after moving the mouse fast enough does it begin spinning forever like it’s supposed to.

As to where the randomness comes from: my testing revealed that the culprit is crossing the threshold. My guess is that the frame when the mouse velocity goes from one side of the threshold to the other can vary a little bit. Now, this randomness is annoying, but I’m not sure I should care about it. I think its just noticeable because of the bigger issue here.

That issue is the dramatic jump in speed when inertia kicks in.

This leap in speed makes the camera very difficult to control (anecdotally, anyway) because I move the mouse in a feedback loop with I see on my screen. When I accelerate my mouse just a little and the camera moves a little faster in response, perfect. But, when I accelerate it just a little more (crossing the threshold) and it goes flying, it is rather annoying. I often end up straddling the threshold and fighting the camera. I tell it to move slower, but then the speed plumets because it dropped below the threshold, making the camera feel stuck. I try to speed back up, but then it overshoots. It’s a battle.

After playing more with three.js’s camera, I see that they are smoothing all mouse movement, regardless of speed or distance. Even one pixel of cursor movement produces an eased camera animation. It tells the camera it needs to move some amount, then the camera applies that motion over time. I suspect that they are using the ‘approach’ technique and that their inertia calculation doesn’t jump when ‘activating’ like Babylon’s current implementation does.

I apologize for any repetition or wordiness. I’m having a hard time explaining the issue. Please feel free to ask any follow up questions. I’d be happy to answer.

8

I looked into the babylon code today and I believe I figured it out. Take a look here:

The inertia delta (seen as the variables inertialAlphaOffset, inertialBetaOffset, and inertialRadiusOffset) is set to zero if it drops below the epsilon (0.001).

This logic is necessary, but as written, it suffers from an issue. Let me explain.

The delta is the difference from the last frame to the current one. At 240 FPS, this delta is very small; much smaller than at 60 FPS. This causes the inertia to be clamped to zero more often, especially with an epsilon that large. Therefore, only big mouse movements at 240 FPS translate into a large enough delta to escape this clamping logic.

It’s possible that using a smaller epsilon is all that’s needed, but if my frame rate hypothesis is correct, there’s probably a better solution.

I quoted the ArcRotateCamera logic because I felt the issue in the code was the easiest to read there, but this problem appears to exist in TargetCamera too.

9

Following up with my last message. Below is a playground where I shrunk the epsilon (0.00001) and the problem disappears for me. I can’t speak to the proper solution for babylon’s source, but I’m happy there’s a workaround. I’m still open to investigating/testing/discussing further, if it helps.

10

@reedsyllas can you check whether you repro the original issue with this playground?

Epsilon is reverted back to original, and instead of using speed it is using this._computeLocalSpeed. Let me know what u see!

Camera inertia fixed | Babylon.js Playground

11

Hi @reedsyllas - I’m actually out sick with a cold, but I wanted to thank you for the detailed investigation! @georgie, thanks for jumping into the conversation - I’ll check back in with this thread when I’m feeling better.

1 Like
12

Yes, @georgie. I cannot feel any difference between that playground and the epsilon = 0.00001 one. It works perfectly.

@amoebachant Sorry to hear that, I hope you feel better soon! Thanks for helping with this.

What are next steps? Anything you need from me?

13

Great! Here is PR updating cameras to use local calculated camera speed vs constant .speed.

Calculate inertia using localspeed rather than const speed in target/arcrotate cameras by georginahalpern · Pull Request #17197 · BabylonJS/Babylon.js

1 Like
14

@reedsyllas sorry for the delay, can u also test this?
Camera inertia fixed | Babylon.js Playground

i think its a bit better to just scale epsilon vs rely on the computeLocalSpeed fn

if the above suits your need i will update PR

15

also @reedsyllas thank you for the super detailed investigation / explanation! as you mentioned there are improvements we could make to our camera interaction system to bake framerate into speed calculation to produce more of an eased animation. this is in our roadmap and we will definitely let you know once we begin on that work!

btw i see you are new on the forum though seemingly quite well-versed in the graphics space. how did you get started with babylon?

16

Hi, @georgie. Sorry for the delay in testing that link.

It does not work the same as before. I looked into why and I saw that scene.getEngine().getFps() returned 60 for me which is not correct. Therefore, the calculation 0.001 * 60 / scene.getEngine().getFps() resulted in 0.001 (60 FPS) instead of 0.00025 (240 FPS).

As for your second message, thank you! I know how difficult it can be to debug issues with limited information. I’m looking forward to those camera improvements you guys have planned. Babylon is a fantastic project and has a truly promising future.

I am new to the Babylon forums, but I’ve been using Babylon for a few years. I am one of the developers of this project (the other founding member):

3 Likes
17

The problem with that link is that FPS changes over time. Yet, the function in there is called only once. Moving the call to the _checkInputs method resolves this as seen here:

While that works, is there a way to get the deltaTime from the last check? I believe that would be the most ‘accurate’ solution.

Of course, that might not be possible. I know you just mentioned a refractor scheduled for the speed. Would we have to wait for that or is deltaTime available?

18

I put together a playground that uses scene.getEngine().getDeltaTime() instead:

The calculation used is:

(BABYLON.Epsilon / 10) * (scene.getEngine().getDeltaTime() / (1000 / 60))

Here’s the explanation:

  1. scene.getEngine().getDeltaTime(): the time since the last frame in milliseconds.
  2. (1000 / 60): one sixtieth of a second (in milliseconds) used to normalize the deltaTime to 60 FPS.
  3. scene.getEngine().getDeltaTime() / (1000 / 60): averages 1 at 60 FPS and 0.25 at 240 FPS.
  4. BABYLON.Epsilon / 10: the constant epsilon. The / 10 is there because even at 60 FPS, 0.001 is still a little too large of an epsilon for this use case. It could be dialed in to a smaller denominator than 10, I’m sure. Edit: BABYLON.Epsilon / 4 feels like enough.
  5. Finally, (BABYLON.Epsilon / 10) * (scene.getEngine().getDeltaTime() / (1000 / 60)): a constant epsilon scaled by the time since the last frame. At 60 FPS, it should equal ~0.0001. At 240 FPS, it should equal ~0.000025.

The reason why it’s approximate is because the time between each frame varies a little. You could imagine a scenario where a couple of frames take longer (resulting in 200 FPS, for example). In that case, the epsilon would be slightly larger, matching the slightly larger camera delta. I believe this is desired behavior.

1 Like
19

If the team is okay with the camera delta speed epsilon not matching what it currently is at 60 FPS, a much simpler calculation can be used:

scene.getEngine().getDeltaTime() / 100_000

Which boils down to just including delta time in the calculation. The delta time is the important part. It’s what’s missing from the calculation currently. The 100,000 above just scales it to an appropriate magnitude for checking against the camera’s delta velocity.

Edit: thinking about it more, I believe the 100,000 constant could be derived from the value it’s trying to clamp. For example, I imagine the arc rotate camera alpha and beta would use radians. That way, instead of some magic number like 100,000, it would be the number in radians below which inertia should stop being calculated. I’ll see if I can figure this out.

1 Like
20

I think I’ve finally figured the math out.

The camera position and rotation deltas are calculated as the motion from the previous frame to the current one. Thus, it changes with framerate. More frames per second = less time between frames = smaller delta.

To normalize this we can divide the delta by delta time. This converts the delta into delta over time. To us help understand this, let’s write out some examples:

  1. A drag happened at 60 FPS. The delta is 8 radians and the delta time is 16.666 ms. 8 radians / 16.666 ms = 0.48 radians per ms.
  2. A drag happened at 120 FPS (same mouse speed). The delta is 4 radians and the delta time is 8.333 ms. 4 radians / 8.333 ms = 0.48 radians per ms.
  3. Finally, a drag happened at 240 FPS (again, same mouse speed). The delta is 2 radians and the delta time is 4.166. 2 radians / 4.166 ms = 0.48 radians per ms.

You can see that all of the above examples result in 0.48 radians per ms. It’s in milliseconds because engine.getDeltaTime() gives milliseconds.

Now that we have this normalized value, it’s trivial to compare it with a constant. Example:

delta / deltaTime < epsilon

Here’s the playground link:

1 Like