Proper way to sync two camera's pose

Questions
1

I have two scenes and two cameras in my app (say scene1 with camera1, scene2 with camera2), see this PG

scene2 is supposed to be rendered before scene1, and I want camera2 to be driven by the user interactions of camera1.

const render = () => {
    scene2.render();
    scene1.render();
}

const syncCamera = () => {
    camera2.position.copyFrom(camera1.position);
    camera2.rotation.copyFrom(camera1.rotation);
}

I have tried to call the syncCamera at different times but there are always gaps. I want to know where exactly I should do this.

If the poses of the two cameras are synchronized properly, the two spheres should always be overlapped.
image

However, currently the two spheres have a gap when rotating.
image

2

I guess when you write :
camera2.rotationQuaternion.set(position.x, position.y, position.z, position.w);
You mean
camera2.rotationQuaternion.set(rotation.x, rotation.y, rotation.z, rotation.w);
Right ?

Did you try to simply attach both camera 1 and camera 2 to the canvas ?

1 Like
3

Want to try reproducing it on the playground? This way we can all “play” with the same code and find the best solution.

Small note - vector3 and quaternion have both the copyFrom function, which allows you to pass a different vector3/quat and copy their values

1 Like
4

Try this:

It is from GUI `linkWithMesh` not behaving properly using right handed system - #9 by CrashMaster for reference

3 Likes
5

Sorry for the typo, the actual running environment is correct, I have modified the question description.
Render the two scenes into canvas directly without RT & blending has same behavior with current workflow. It seems to be only related to pose syncing procedures.

6

I have add a playground Babylon.js Playground

However the behavior of PG does not match my local environment, I guess there’re some different since my local context are huge.


    const syncCamera = () => {
        camera2.position.copyFrom(camera1.position);
        camera2.rotation.copyFrom(camera1.rotation);
    }

    const render = () => {
        syncCamera();    /// inproper
        scene1.render();
        // syncCamera(); /// proper
        scene2.render();
        // syncCamera(); /// inproper
    }

    /// proper
    // camera1.onViewMatrixChangedObservable.add(() => {
    //     syncCamera();
    // });

In the PG if I called syncCamera between the two render() or within the onViewMatrixChangedObservable() there’s no delay. Otherwise there will be delay.

Thank you @CrashMaster , in your pg and mine, similar problem can be solved by syncing camera within the viewmatrix change callback. But this still has no effect in my local environment.

I’ll try to reproduce my problem in the PG.

7

If the goal is to have both cameras aligned, why don’t you attach both ?
You could also set camera1 as parent of camera2 ?

8

EDIT: I’ve just realized you added this 10 minutes ago already … I’ve got the same solution just for the FreeCamera

const syncCamera = () => {
    camera2.position.copyFrom(camera1.position);
    camera2.rotation.copyFrom(camera1.rotation);
}
1 Like
9

Thanks! Add camera2 as children of camera1 is good.

Because I want to hide the internal camera (camera2) of my toolkit to users. Users are supposed to create their own camera (camera1) and interactions, and use my interface (something like a syncCamera()) to drive the internal one. Actually I have other cases that use Three.js or pure WebGL to create camera1, so I want the bahavior of all cases to be the same.

10

I have found the difference between the old PG with my real app. This new PG can explain my problem exactly.

The difference is I want to render scene2 before scene1, but use camera1 to drive camera2.

If the syncing is correct the two balls should completely overlap each other on rotating. But now they will have some gap like this.

image

11

I can’t see any gaps when syncing the camera before rendering the scene:

13

That’s strange, i opened your playground directly and rotate with mouse, there is a gap. :joy:
image

14

You mean that you can both spheres? It’s because of z-figthing.

15

It’s does not look like z-fighting I think :rofl: because if the camera is not moving/rotating, the two sphere can completely overlap each other.

And if you move the camera far to 50 meters (PG), which makes the rotating very fast, you can even see two balls far away from each others.

var camera1 = new BABYLON.FreeCamera("camera1", new BABYLON.Vector3(0, 5, -50), scene1);

image

16

It looks like it is because you copy the old position of cam1 in cam2 before rendering it so you always have one frame delay between both ? https://playground.babylonjs.com/#EYOOKM#6

changing the render order of your scene should fix it

2 Likes
17

Yes changing render order works :rofl: but render scene2 before scene1 is needed due to some transparency ordering reason.

My workaround was to use a third camera for user interaction (no rendering), and set pose of camera1 and camera2 from it. But this looks a little bit ugly…