GeospatialCamera and Genericized Camera Inputs

I stumbled across the Experimental GeospatialCamera and incorporated it into my work-in-progress Tile Map Server client using Babylon. It’s showing promise.

I wanted to add Touch and Pinch support so I used the ArcRotateCameraPointersInput as a starting point.

ArcRotateCameraPointersInput is already almost generic enough to be usable with other cameras, if a few tweaks were added.

Most of the code is supporting internal state information and not application specific.

A big question relating to abstracting/separating camera from inputs: why are some properties available on the camera (e.g. pinchToPanMaxDistance, ) and some are available on the input (e.g. angularSensibilityX, pinchPrecision, static MinimumRadiusForPinch)? It looks like for the ArcRotateCamera, properties on the ArcRotateCameraPointersInput are given accessors on the camera. Is that really necessary? It seems to unnecessarily tie the ArcRotateCamera to specifically-coded PointersInput.

It seems plausible to adapt a generic TouchInput class to work with a variety of cameras so that adding specific input controls is less tedious.

For the touch control at the end of the playground, I am simply outputting the parameters to a popup box to play around with the control to see how it works.

The primary functions that a user would need to define are:

  • _computeMultiTouchPanning(previousMultiTouchPanPosition, multiTouchPanPosition)
  • _computePinchZoom(previousPinchSquaredDistance, pinchSquaredDistance)
  • getClassName
  • getSimpleName

The list of properties that control this input method include

  • panningSensibility
  • pinchInwards
  • multiTouchPanAndZoom
  • multiTouchPanning
  • pinchZoom
  • useNaturalPinchZoom
  • pinchDeltaPercentage
  • pinchPrecision
  • angularSensibilityX
  • angularSensibilityY
  • static MinimumRadiusForPinch
  • (super?) _ctrlKey
  • camera._useCtrlForPanning
  • _isPanClick

The PointersInput does a pretty good job of isolating the functionality within the class and keeping track of its own state with respect to multi-touch pinch vs zoom. I’ve renamed it to GeospatialTouchInput, but it could be adapted so touch-based pinch/zoom could then easily be incorporated into other cameras. I’m not sure if it should be adapted as an inherited class or with a constructor that takes the customized methods, or an instance to which the user assigns custom methods.

I have some recommendations on improving GeospatialCamera, but I’ll save those for another post.

1 Like

cc @georgie

Hey @HiGreg thanks for posting! You’re speaking to a problem that is very top of my mind, as i am currently working on some changes to the geospatial camera system to de-obfuscate how the camera is controlled by user (so that the properties to control it are not spread across input classes and camera) and so that the classes have very clear responsibilities (input classes gather input deltas , camera handles recalculating matrices to account for per-frame movement, and new movement class is responsible for translating pixel deltas into per- frame movement, taking limits/ speed/ intertia into account. It should be live by EOW. After that gets pushed I’ll ping here so that you can incorporate the new API into your project (as its experimental it will not be back compatible) and we can see how it relates to your suggestions!

for the feedback about geocam, it’s possible I’m currently addressing but feel free to share :slight_smile:

2 Likes

What you’ve written sounds great!

Issues I have with the current GeospatialCamera are

  • the forced use of “pick,”
  • the spinning during pan at/near north and south poles,
  • Zoom doesn’t always stop at surface+altitude

I’d prefer to not have to use pick and instead use a position/radius.

I’m not sure if my ideas are fully baked, but here they are:

  • clearly define reference points, such as
    • forward (e.g. lookAt, target) as TransformNode or Vector3
    • up,
    • geoCenter as TransformNode or Vector3
  • persuant to “clear responsibilities,” offer a few different ways to move the camera, such as
    • straight-line movement (vector3 supplied, could be forward)
    • movement along radius around geoCenter (specified as 2d tangent vector, but what reference point?)
    • optional update forward Vector3 unless forward is TransformNode.
    • zoom by forward movement, or by fov
    • (possible) zoom to radius/intersection then remaining zoom by fov
    • pivot axis/angle around specified Vector3 point with optional “change look” afterwards.

Then if generic input classes are defined that accept callbacks (or used as extended classes), the input actions can be tied to the desired selection of camera actions. (“generic” in the sense of being able to be used with various cameras snd tying them to desried camera actions.)

I kind of prefer zoom by specifying “amount to increase the size of a (hypothetical) object at specified distance.” But the math is more complicated than moving the camera position by a Vector3.

For a distance d, a width of one unit (world space) is:

2*d*Math.tan(camera.fov/2)

(see my dolly zoom post to see changing fov and zoom such that width at a distance is constant.)

I’m very interest in seeing your next release of GeospatialCamera!

A few more observations after digging into cameras and inputs. These are just what I hope to help implement.

I don’t prefer picking but I do think it is useful. I discovered that it is part of _applyZoom, which itself is within the input/render loop. That makes sense if the zoom is part of wheel movement. For touch input and pinch to zoom, Pick should be done only on the initial onMultitouch, not continuously during pinching. (I’m coding a touch-aware input class.)

It might be useful if multiple inputs on a camera can coexist. This might only be an issue with wheel versus pinch, but I also speculate keyboard input might conflict with keyboard modifiers on mouse movements. POINTER events aggregate both mouse and touch, so it makes sense to watch out for conflicts between inputs responding to pointer, mouse, and/or touch. Each input class should (IMO) be able to stand alone and play nicely with others.

Many parameters are sensibility based. Is it prudent to gather those into an object? I’d guess that would be an input object, but then how can the interaction between different settings on different inputs look? Can they be correlated when needed?

Can the target change dynamically? If I have two spheres in the scene, can I easlily switch the camera to reference one sphere or the other? At minimum, the radius/altitude could be different.

How are bumpmaps or height maps accomodated?

It would be interesting to be able to mimic an ArcRotateCamera or an ArcRotateCamera with look != target, or an ArcRotateCamera that doesn’t lock at poles. Could this also be a substitute for UniversalCamera or FreeCamera? Maybe I don’t know enough of the inner details, but it will be interesting to explore.

Last thing for now: the mechanism of adding an input through inputManager.add() creates nested function call for each input.checkInputs() in the chain. Seems like this would be better implemented as an array of checkInputs functions that are called in a forEach loop on the array. What about something like:

attachedInputs.forEach(i=>i.?checkInputs())

@georgie, looking forward to your updated GeospatialCamera!

1 Like

I’m trying to somewhat copy the controls from ArcGIS webscene viewer touch controls.

Incomplete list:

  • double-touch swipe vertically: camera pitch (with limits)
  • double-touch spin: camera spin on radial
  • single-touch swipe: camera geodesic pan

Note how you can get close enough to the surface and pitch the camera to see elevation (go near Alps, Andes, or Rocky Mountains).

There is some combination of position/translation zoom and fov zoom happening that I haven’t figured out.

Digging through existing code, I’ve created some additional functions to manipulate camera. Not sure where these will ultimately live or if they are duplicates of existing functions.

cameraRotateByQuaternion(axis,q) {
    this.camera.upVector.applyRotationQuaternionInPlace(q)
    this.camera._lookAtVector.applyRotationQuaternionInPlace(q)
    this.camera._isViewMatrixDirty = true;
}

Rotate around radial to globe.

this.camera.position.subtractToRef(sphere.position,axis);
cameraRotateOnAxis(axis,amount) {
    BABYLON.Quaternion.RotationAxisToRef(axis,amount,this.tempQuaternion)
    this.cameraRotateByQuaternion(axis,this.tempQuaternion)
}

Rotate around normal (equivalent to “pitch”)

BABYLON.Vector3.CrossToRef(this.camera.upVector,this.camera._lookAtVector,normal)
this.cameraRotateOnAxis(normal,amount)

For camera.fov modifications, note that only ProjectionMatrix is affected. So anything attached to camera.onViewMatrixChangedObservable is not called. Only camera.onProjectionMatrixChangedObservable sends notifications.

And in the current (about to be replaced) code,

line 181 in _applyGeocentricTranslation
scale new position by length of current position
seems to presume center of globe is at 0,0,0

Hi there, yes the upcoming PR will address quite a bit of the feedback(timing is a bit longer than originally estimated, I will ping this thread once complete)

I have also created this github issue to track the scope of work for all geospatial features. I have taken some of your suggestions and added to the list (some as required and some as nice to haves)

Thanks so much for your thoughts! Once this next round of features is checked in we can discuss more :slight_smile:

Geospatial Features (Large World Rendering, Geospatial Camera, 3D Tile Loader) · Issue #17451 · BabylonJS/Babylon.js

2 Likes

Some of this probably repeats some previous posts. Bit these are targeted at the specific Geospatial Features list (thank you for the list!)

Geospatial Features

WGS84 and ECEF - “Math.geo converters between wgs84 and ECEF coordinate systems”

This is a fairly complex topic. Perhaps it is easier than I’ve outlined below. I would suggest for “conversion” at least allowing the definition of an ellipsoid (usually with semi-major axis and a flattening ratio, or equivalently the semimajor axis and semiminor axis). And optional specification as a sphere (with radius parameter). A sphere is an ellipsoid with flattening of 0.

Technical note: WGS84 does define an ECEF coordinate system. I think you may be referring to translation between WGS84 and an ECEF Cartesian Coordinate System (both are ECEF).

Edit

I looked up the actual values and the question below about “which wgs84?” is moot. Here is the definitive reference from the source:

geoid:

now using a separate Earth Gravitational Model (EGM), with improved resolution and accuracy. Likewise, the World Magnetic Model (WMM) is updated separately. The current version of WGS 84 uses EGM2008 and WMM2020.

For mapping, charting and navigational users, these improvements are generally negligible. They are most relevant for the geodetic user and other high accuracy applications.

From NIMA TR8350.2; THIRD EDITION; AMENDMENT 1; 3 JANUARY 2000

Table 3.1 WGS 84 Four Defining Parameters

Semi-major Axis (a) = 6378137.0 meters
Reciprocal of Flattening (1/f) = 298.257223563
Angular Velocity of the Earth (ω) = 7292115.0 x 10-11 rad/s (7292115.0e-11)
Earth’s Gravitational Constant (Mass of Earth’s Atmosphere Included) (GM) = 3986004.418 x 10^8 m^3/s^2 (3986004.418e8)

Summary: The WGS 84 ellipsoid is defined with
semi-major axis (“a”) = 6378137.0 meters, and
reciprocal of flattening 1/f = rf = 298.257223563

end Edit

The first question is “which wgs84?” wgs84 in this context usually refers to a reference ellipsoid, and the wgs84 ellipsoid changes periodically, though the differences in large-scale mapping are minor. Another complication is that some mapping systems assume a spherical earth (e.g. Web Mercator used in TMS, “slippy maps”).

It is probably not necessary to delve fully into definitions of geoid, mean sea level, height above ellipsoid, altitude, barometric altitude, orthometric height, geopotential height, etc, etc, but some elevation models, including wgs84 altitude (“height above wgs84 ellipsoid”), depend on some consideration of mapping vs height data derived from other sources or using other reference points (e.g. airplanes, satellites, clouds).

To make sure the mapping is compatible with other applications, especially when altitude data is comingled, it should be very clear what the mapping application is using as “altitude.”

Touch input (multi-finger pan / zoom)

In-work GeospatialTouchInput: (should it be GeospatialMultiTouchInput?)

getClassName(){//: string {
    return "GeospatialTouchInput";
}

/**
 * Get the friendly name associated with the input class.
 * @returns the input friendly name
 */
//public getSimpleName(): string {
getSimpleName() {
    return "touch";
}

API changes to support setting yaw/pitch/radius/center directly

What do you think API should look like?

// state is kept in camera.upVector and camera._lookAtVector
// when either is changed, set camera._isViewMatrixDirty to true, then
// recalculate viewMatrix on next getViewMatrix call.
// To calculate normal, use:
// BABYLON.Vector3.CrossToRef(this.camera.upVector,this.camera._lookAtVector,normal)
// can camera.getNormal() be treated as viewMatrix, recalculate when _isViewMatrixDirty?

// pitch: normal; yaw: camera.upVector; roll: camera._lookAtVector

// the following Quaternion methods are handy with cameraRotateByQuaternion(), below:
FromEulerAnglesToRef
FromEulerVectorToRef
RotationAlphaBetaGammaToRef
RotationAxisToRef
RotationQuaternionFromAxisToRef
RotationYawPitchRollToRef

// maybe camera rotation methods can be modeled on Quaternion's methods?
// if needed at all? maybe just let user use cameraRotateByQuaternion() below?
cameraRotationEulerAngles // Rotation instead of "From"
cameraRotationEulerVector // Rotation instead of "From"
cameraRotationAlphaBetaGamma
cameraRotationAxis // (same as cameraRotateOnAxis() below)
cameraRotationQuaternionToAxes // (I prefer "ToAxes" here, but Quaternion uses FromAxis)
cameraRotationYawPitchRoll
cameraYaw(radians) {    }
cameraPitch(radians) {    }    
cameraRoll(radians) {    }
cameraRotateOnAxis(axis,radians) {
    BABYLON.Quaternion.RotationAxisToRef(axis,radians,this.tempQuaternion)
    this.cameraRotateByQuaternion(this.tempQuaternion)
}
cameraRotateByQuaternion(q) {
    this.camera.upVector.applyRotationQuaternionInPlace(q)
    this.camera._lookAtVector.applyRotationQuaternionInPlace(q)
    this.camera._isViewMatrixDirty = true;
}

Glad to see an update!

I’m still reviewing, but I had to work around GeospatialCameraKeyboardInput not having _engine or _scene set:

    camera = new BABYLON.GeospatialCamera('geoCam',scene,{planetRadius:sphereRadiusWorld,minAltitude:1,})
    camera.inputs.attached.keyboard._engine = camera.getEngine();
    camera.inputs.attached.keyboard._scene = camera.getScene();
    camera.attachControl(canvas, true);

Otherwise, I get a error in Console:

TypeError: Cannot read properties of undefined (reading 'onCanvasBlurObservable') at e.attachControl (geospatialCameraKeyboardInput.ts:99:51)

Oops! PR fix here GeospatialCamera keyboardinput set scene/engine by georginahalpern · Pull Request #17478 · BabylonJS/Babylon.js

Regarding the input control feedback, now that the GeospatialCameraMovement class is in place (to hold the movement-related logic) and distinct from the pointer input classes (which simply handle input and calling into movement functions vs trying to calculate movement logic themselves) we are set up well to introduce different more generic input classes.

After my recent updates the pointerinput class handles touch as well as pointer (including multiotuch). calling it ‘pointer input’ aligns w the basepointerinput which handles touch as well, so I think we are fine to keep it named pointerinput

@HiGreg please feel free to create a PR demonstrating your proposal for genericized camera input class, separating the nested addInput() functions, or additional movement functionality (such as dolly zoom). I am super open to additions here.

Regarding sensibility parameters – those are now cleaned up and the movement class holds the relevant speed variables.

Are you seeing the current input classes not playing niceley with eachother? They are designed to coexist.

Target cannot change dynamically yet – but that is in list of potential future changes

Regarding WGS84 conversions – yes I am referring to the translation between WGS84 and ECEF Cartesian :slight_smile: that is more accurate description. And for the base tile loader I will implement in a way which allows for conversion functions to be supplied so that users can override our default.

Thank you for all of the thoughts! Glad to see the camera being used :slight_smile:

Not fully functional class for multiple Tile Map Service texture. A good playground for GeospatialCamera when it can accomodate multiple targets. Right now, target is on the earth. Zoom out to activate TMS (bug to be fixed) and rotate to see moon.

1 Like