Universal camera with mouse scroll wheel and touch pinch

Hihi,
So my project is coming along nicely.
So nicely i decided to look into better camera control.
I’m using the Universal Camera. I want to be able to move it around the scene and view from any angle. Ideally i’d be able to do this with either just mouse or mobile touch. (Think Google Earth UI.)

So what’s missing? Basically i need another axis of motion.
When using mouse or touch the Universal Camera can pan left-right and up-down but not in-out.
My mouse has a scroll wheel… (Well actually, a touch pad emulating a scroll wheel…)

Looking in Babylon.js/src/Cameras/Inputs at master · BabylonJS/Babylon.js · GitHub it doesn’t look lie i’m missing anything (although i do notice version 4.2 will have keyboard control of 3 axis instead of just the current 2).

So,
is there any appetite for me to do some refactoring?
Last time i ran into similar issues as this i created Babylon.js/BaseCameraPointersInput.ts at master · BabylonJS/Babylon.js · GitHub
With that as the base, getting multi touch working should be fairly straight forward.

This would probably replace both freeCameraMouseInput.ts and freeCameraTouchInput.ts with a unified replacement but they are both much less complicated than followCameraPointersInput.ts which i originally created BaseCameraPointersInput.ts for.

anyone got opinions / input / etc before i start?
dunk.

Actually,
looking at this on my phone this morning, it appears i d have 3 axis on mobile; Double finger drag allows pan up/down.
It is very slow… Need to tweak sensitivity settings.

So wile using BaseCameraPointersInput.ts would give more options for touch input,
i think implementing scroll-wheel is higher on my list of priorities.

So, same question for scroll-wheel on the Universal Camera; is there any appetite for me to do some refactoring?
Is this something that would likely get merged if i implemented it?

dunk.

any refactoring is welcome as long as:

  • it adds something new and useful
  • it protects the backward compatibility

Well i’m Covid locked-down again so stuck in the hose for the weekend.
On the plus side, that means i have an implementation of Mouse Wheel for FreeCamera and derivatives working quite nicely.

I have a few UI questions though.
Mouse wheels can theoretically have up to 3 axis.
I’m not sure what the hardware for the 3rd would look like (pinch multitouch maybe?)… but most laptops have an X and Y axis.

It would make sense for a user to be able to plumb these 3 axis to a range of camera properties.
My current implementation allows the following to be controlled form any wheel axis:

  1. Position of camera relative to the camera. Any 1 axis. (eg: Fly forward.)
  2. Position of camera relative to the scene. Any 1 axis. (eg: Change camera height.)
  3. Rotation of camera. Any 1 axis. (Can’t think why this would be useful but it’s easily done.)
    3x3x3 = 27 different combinations. Boolean toggles clearly aren’t the way to go for the UI here.

Can anyone suggest any thing in the current API that is typical of the style we want to achieve?

Left to my own devices i’d probably have
an Enum of properties and an Enum of axis and assign each mouse wheel axis one of each?

You can see the private variables i need to control here:


Look for the definition of _wheelXAction and the following lines.

dunk.

So i came up with a API for this that is usable.
The following snippet would disable the mouse wheel’s Y axis from controlling the “move forward/backward” default behavior:
camera.inputs.attached["mousewheel"].wheelYMoveRelative = null;
The following would attach the same axis to camera height in the scene instead:
camera.inputs.attached["mousewheel"].wheelYMoveScene = BABYLON.AXIS.Z;

Thoughts?

I like it. This will also require a doc update

let me try to ping @PolygonalSun to get his review (don’t hold your breath as he is fighting against a water leak right now ;))

no hurry.
this is just a fun distraction for me.

I’m just looking at src/Cameras/Inputs/freeCameraMouseInput.ts and thinking about a similar interface for it.
We could make the defaults match the current implementation so existing users would not see any change. UnitTests would confirm this is the case.
I’d like to see the mouse capable of controlling the following camera properties:

  • Position in scene.
  • Position relative to camera.
  • Position relative to ground plane. (Like Google maps in 3D mode. Mouse drag moves camera over ground without adjusting angle.)
  • Camera angle.

I’d also like these properties to be affected dependent on whether a particular mouse button was pressed.
Eg: Left button drag moves the position relative to ground plane, Right button drag changes camera angle.

Bonus points for allowing different keyboard modifier keys to give the mouse different behaviors.

What would the interface look like for that?
I’m 2nd guessing the way i’ve done it for mousewheel as there are just too many combinations to be practical with the mouse: 2 axis of moment, 8 button combinations (with 3 mouse buttons), ~4 camera properties to modify.
We need something that can take arbitrary input properties and map them to an arbitrary action.

Maybe something like this to have click and drag the Y axis on the mouse move the camera up and down the scene?
camera.inputs.attached["mouse"].setMoveAction([MOUSE_PROPERTY.AXIS_Y, MOUSE_PROPERTY.BUTTON_0], CAMERA_PROPERTY.MOVE_SCENE_Z);

In the underlying implementation, the collection of properties could be a bitmap that includes many mouse buttons, keyboard modifiers, etc. A bitmap would allow a simple switch statement to pick the required action cheaply.

It’s not really in the same style as other Babylon components though.
Let’s try again:
camera.inputs.attached["mouse"].action[MOUSE_AXIS.Y][MODIFIER.BUTTON_0] = CAMERA_PROPERTY.MOVE_SCENE_Z;

This scheme would be a little annoying if we want to support more than one modifier; Either the user needs to understand bitwise operators (which isn’t really a thing i’d expect frontend folks to be thinking about often) or we’d need an Enum with a value for each combination of modifiers.

I don’t like it.
Let’s try again:
So in the last example if we were willing to have a big Enum with many, many values we might as well have many, many variables. And so we are back at something similar to my mousewheel implementation:
camera.inputs.attached["mousewheel"].mouseYButton0 = CAMERA_PROPERTY.MOVE_SCENE_Z;.

We’d need a full map of variables to assign properties to. eg: mouseYButton1, mouseYButton2, mouseYButton01, etc.

I still don’t particularly like it.
Maybe i’m approaching this wrong.
Instead attaching camera properties to mouse action, attach the mouse action to the camera property.
camera.inputs.attached["mouse"].moveSceneZ.mouseAxis = MOUSE_AXIS.Y;
camera.inputs.attached["mouse"].moveSceneZ.modifiers = [MODIFIER.BUTTON_0];

I originally (when i was thinking about mousewheel) discounted this approach as it complicates verifying there isn’t more than one thing being controlled by a single input. You end up with a lot of getters and setters in the implementation.
I think it might be worth it though.

If i didn’t have to match existing style, I think my first example is what i’d choose. setMoveAction(mouseProperties, action).
Any Babylon devs care to comment on what the rules are on UI?

Anyone get this far?
Thoughts?

dunk.

Hey @dunk,

Sorry about the delay on a response here. I’ve been dealing with RL stuff but I should be able to take a look at things and provide proper feedback today.

No worries. It’s given me time to think some more.

The previous post is a bit of a stream of consciousness… Sorry about that,
Don’t know if it makes sense or not.

To summarise, I’m asking how to approach API inputs when we want to map a matrix of inputs to a matrix of outputs.
I think it makes sense to pick something that can be applied to any future camera input.

So just got around to checking back on the MouseWheel with FreeCamera / UniversalCamera / etc.
Working as intended:
https://playground.babylonjs.com/#DWPQ9R#1

2 axis of control on my laptop touchpad.
You can also adjust which camera properties are controlled.
I’m just looking into documenting it now.

Note that you need to enable it; It’s not on by default.

2 Likes

Since, your PR has been merged and it’s working as intended, I’m gonna go ahead and close this thread. Thanks for getting that squared away @dunk!