Multiple viewports and mouse pickInfo issue

Questions
1

i have a scene with multiple active cameras, each having a set viewport
(e.g. two cameras as split-screen) and that works fine.

const cameraOne = new ArcRotateCamera('one', 0, 0, 90, new Vector3(0.5, 0.5, 0.5), scene);
cameraOne.viewport = new Viewport(0.0, 0.0, 0.5, 1.0);
const cameraTwo = new ArcRotateCamera('two', 0, 0, 90, new Vector3(0.5, 0.5, 0.5), scene);
cameraTwo.viewport = new Viewport(0.5, 0.0, 0.5, 1.0);
scene.activeCameras = [poseScene.camerasOne, poseScene.camerasTwo];

but when listening for mouse events, pickInfo only gets set for camera with last viewport and is null otherwise.

scene.onPointerObservable.add((pointerInfo) => {
  if (pointerInfo.type === PointerEventTypes.POINTERDOWN) console.log(pointerInfo.pickInfo?.hit)
}

when clicking on mesh in first viewport, it returns false and when clicking on the same mesh in second viewport, it returns true

if i have single camera/viewport - seems like pickInfo is only looking at last set camera?

changing with camera is attached doesn’t make a different

environment: babylonjs 5.0.0.beta4 on chrome 97

2

I think this playground reproduces a similar effect. the button on the left doesnt work, while the button on the right does.

3

Seems it depends on active camera’s order at the moment.
with

scene.activeCameras = [camera2, camera]

the left button works and right doesn’t.

4

figured it out - in case of multiple cameras/viewports, pointerInfo.pickInfo is useless.

instead need to

  1. determine in which viewport click occured (check bounding box vs mouse coordinates)
  2. run scene.pick with specific camera parameter that matches the selected viewport
const pickInfo = scene.pick(scene.pointerX, scene.pointerY, undefined, false, cameras[activeCamera]);

the problem seems to be that scene.pick looks at scene.activeCamera if camera is not set and that setting is irrelevant/incorrect when ``scene.activeCameras` is set to an array of cameras

that seems to be an oversight to me - i’d expect that scene.pick looks at ALL active cameras if camera is not specified?

2 Likes
5

hmm, I wonder how it would work when two cameras are on top of each other, maybe in some kind of window system

6

i’d expect it to pick the top-most mesh that doesnt have a mask set. same thing if there is a single camera and there are multiple overlapping meshes.

7

that seems like the most intuitive behavior to me also. it’s still relatively early in the US, hopefully someone from the bjs team will be able to give a proper reply soon.

8

Apologies for letting this slip through! Scene has a cameraToUseForPointers parameter ( Scene | Babylon.js Documentation (babylonjs.com)) that can be used to specify which camera you want to use, but I feel having the capacity to run pick for multiple cameras at once would be useful too… @sebavan @PolygonalSun what do you think?

9

@Blake is awesome and proposed to do it for after 5.0 to not introduced regression as it is a large change here: Second Camera does not call onLeftPickTrigger Action - #26 by Blake

3 Likes
10

@Blake you’re a ROCKSTAR!

1 Like