How to set camera orientation to that of an empty transform node that is nested in a parent hierarchy?

[johntdaly7]

Hello,

So I have a bunch of rooms in a space I am showing, and I would like to be able to snap the camera to a good orientation to view the rooms for editing furniture, etc. In my blender file, I placed in empty objects and oriented them in certain positions and rotations in order to have the camera viewing the space that I want to view in that area properly. Each empty camera anchor is a child of an “Area” empty object, and each “Area” empty object is a child of a “Building” empty object in the scene hierarchy. When converting from .blend to .glb for view in Babylon, all of the empty objects obviously get converted to TransformNodes. In code, I would like to set the camera (which is not a child of any node) to the position, rotation (orientation), etc. of each respective transformNode anchor, when appropriate.

The issue I am getting is that the camera is not being placed in the proper position or rotation when the position and rotation of the transformNode anchor are assigned as the camera’s position and rotation.

I tried a couple of methods so far. First, I tried parenting the camera under the same Area anchor, and then setting the camera’s position and rotation to that of the transformNode (due to the difference of local vs global coordinates, especially how blender calculates them versus how they are saved in glb/how babylon calculates them/reads them. This worked for 2/3 of the transformNode anchors, but none of the rotations were applied correctly. Parenting them also oddly mirrored the scene in the camera viewport, so that all the assets appeared mirrored horizontally (not sure why). Because of those issues, I then tried gathering up and performing vector addition on all of the positions of the parents in the hierarchy that I would have to go through to get to the same local position reference as the transformNode anchor. This got the position close, but was slightly off for some unknown reason, and the rotations were still off. Lastly, and what has worked the best but still not sufficiently, I calculated the worldMatrix as such:

const worldMatrix = cameraAnchor.computeWorldMatrix(true);
var newCameraPosition = Vector3.TransformCoordinates(cameraAnchor.position, worldMatrix);
//I applied the same to the newCameraRotation, and also tried just using the regular: newCameraRotation = cameraAnchor.rotation;

This put the camera in the correct position again for only 2/3 of the transformNode anchors (the third anchor is following the same parenting structure as the other two, and is in the same sibling level as the other two in the hierarchy as well, so I am not sure why that is standing out…could it be because the Area associated with this third anchor has more InstancedMeshes and those are somehow throwing off the local coordinate system reference?). It also did not do anything to set the rotations properly.

I tried to reproduce this issue in a playground, but the way the meshes and transformNodes are set up is too complicated to be feasibly shown without having the ability to import my own glb file. Plus, my code is pretty convoluted in setup that that might also be a part of the issue. So really I am looking for any useful thoughts and ideas of what to look into/what functions or collections of functions to try out to fix this issue. Thanks!

[sebavan]

Could you use node.absolutePosition and absoluteRotationQuaternion which are the world space pos and rot of the node to apply to your camera ?

Also a quick playground repro would be great

[Evgeni_Popov]

Try to set scene.useRightHandedSystem = true just after scene creation to be sure that the handedness of the scene does not get into your way…

Note also that glb/gltf are setting the rotationQuaternion property, not rotation, so you should set rotationQuaternion and not rotation.

A repro would definitely help here!

@johntdaly7

without having the ability to import my own glb file

There are several ways to host assets to be used in the PG: Using External Assets In the Playground | Babylon.js Documentation

[johntdaly7]

I tried that and got nowhere. The absolute position and absolute rotation quaternions are all 0,0,0 upon load from glb. the only things that have values are the positions. So I don’t know if that’s a bug or is due to the parenting structure. But the global (absolute) positions do not seem to be converted based on the local positions by following the hierarchy.

[johntdaly7]

I’ll give that a shot

[johntdaly7]

Do I set handedness after I load in the assets or before? Because I applied it beforehand and it skewed EVERYTHING.

Also, is there a way to get the global rotationQuaternion using the world matrix? My absoluteRotationQuaternions are all turning out to be (0,0,0,1), which I know is not the way the empties are facing.

[johntdaly7]

Also, using the cameraAnchor.rotationQuaternion for the camera makes the camera angled diagonally (the roll is skewed).

[sebavan]

This sounds stranger and stranger, a repo in the playground would definitely help

[Evgeni_Popov]

The handedness should be set right after the scene is created and before you load the assets. That will flip the display but it is expected.

After loading, the rotationQuaternion property of the camera/meshes should be non null/non unit quaternion if they are rotated in Blender.

The absolute rotation quaternion returned by .absoluteRotationQuaternion is the right one, it is recomputed in case it would not be up to date. If you get a unit quaternion, it means the corresponding transform node is not rotated.

[johntdaly7]

It seems to me that there is an issue with how BabylonJS reads in GLBs, in my opinion.

Here are a couple of updates:
I tried changing the handedness, and this is the result I got:

Normal handedness:

Using Right-handedness (setting scene.useRightHandedSystem = true; before calling load function):

When it comes to using absolute (global) values…it seems all of them are automatically set to (0,0,0) upon load. I know this isn’t correct because my assets are obviously rotated in blender, I don’t just have them all looking the same way. I think there’s an issue with how Babylon interprets the position/rotation values from the blend file when it comes to global values vs local values (especially if an asset is in a multi-ancestor hierarchy). Then again, I could be wrong, but I created this basic playground repo to tinker with it, and it’s still out of wack, so I am at a loss for what else to consider: https://playground.babylonjs.com/#2J8957#1

The blend file I am using can be found at:
https://av2-assets.s3.amazonaws.com/1399+NYAV+R5+V4+HIERARCHY.blend

Here are some snapshots of the differences of how the camera anchor’s transform appears in blender vs babylon inspector vs console in babylon playground:

Conference room anchor:

floorplan anchor:

table area anchor:

!

Thanks for the patience and help with figuring out this (what should be trivial) issue!

[johntdaly7]

@sebavan
@Evgeni_Popov
Just tagging you both to make you aware since this is an older thread now

[johntdaly7]

this is the link to the glb, by the way. (it also can be found in the playground):
https://av2-assets.s3.amazonaws.com/1399_NYAV_R5_V5.glb

[sebavan]

At a first glance the anchor 1 (floorplan) sounds at the right place:

I simply attached a red cube to it and this is what it gives.

Anchor 0 (conference room) looks at the correct place:

and Anchor 2 (table) as well:

The various tricks for handedness correction make it difficult to match the transforms in a flipped camera.

Here is an easy solution:

Babylon.js Playground

[johntdaly7]

thank you so much @sebavan ! I will attempt to integrate this solution into my project. One followup question though: how would you animate the change made with anchor.getDirection (camera’s rotation/direction, currently, to its new setting) ? I know how to animate the position and rotation values, but usage of that throws a bit of a wrench into what I already have in that regard. Thanks!

[sebavan]

Animating to the new setting will be weird cause you ll cross walls I would probably fade to black switch camera and then fade back from black.

[johntdaly7]

Thanks @sebavan. That’s a good suggestion, but I need to animate without fading to black. I’m not worried about going through walls. Does setTarget change the recorded value of the camera’s rotation? Because if so then I could setTarget and then grab the new rotation of the camera (would I have to recalculate world matrix?) and then animate the camera from original camera rotation to new camera rotation? Not sure how to animate to the result of the function setTarget, since it doesn’t return anything, and uses a direction vector instead of rotation.

I also need to be able to set the camera to look at the floorplan but, when looking at floorplan, have camera constraints in regards to the camera rotation so you can only look a certain number of degrees up and down and rotate around the y axis from where the camera’s rotation is initially set when the camera is placed at the floorplan anchor. Is there a way to do this while using setTarget, like in your example? Thanks!