Multi Canvas Picture in Picture

Questions
1

So i know Multi canvas is possible as described in the documentations and I’ve been able to get it to work. But what about Picture in Picture.
More specifically when clicking on a mesh in my scene I create a camera and the code positions the camera in a perfect spot to allow the mesh to fill the camera. The camera rotates around the object. All of this is working perfectly as is the necessary cleanup when removing. (I did this by swapping active cameras as a test.)
Now to fully implement my system I have a smaller canvas in a UI element.
If I follow the documentation directions I can use engine.registerView() which will work except for the fact different Canvas sizes are not supported but as a test it worked. Now if I was to attach a newly created camera to the registerView both canvases will stop rendering.

I’ve also attempted to create a second engine and pass the current scene onto the second engine and render again.
For example

const pipCamera //Babylon Arc camera and position the camera
const engine2 = new BABYLON.Engine(pipCanvasElement, false);

engine2.runRenderLoop(() => {
  scene.render()
})

The second engine does initialize and a small fps drop does occur so I know something is going on but the pipCanvas is still black (The default background color is black).
I can see that the BabylonJs attributes have been added to the canvas element. Maybe this is not the best way to go about it.

Back to the registerView. As it seems it can only support a single active camera and if a view is given a new camera to render from it wont work. Which is probably what im experiencing. Does that mean I’m required to setup some logic to cycle the cameras to the different canvases to get them to render properly. There must be a simpler way that I may of overlooks.

Thanks.

2

It seems you only need a second camera to render with a different viewport in the same canvas here ?

3

Here is the example with 4 cameras at one canvas - https://playground.babylonjs.com/#7CBW04#707

4

Unfortunate this does not fit my purpose as the secondary canvas is part of a UI window which can be dragged around above the main canvas element, resized, minimized and destroyed.

But I think i may have a solution, I was going over the documentation last night and found that you can serialize meshes. So i might try that and pass that to a new scene and engine running on the secondary canvas.

Edit: For some reason the reply submitted mid typing.

My current setup would be impossible to do in the playground as it requires multiple canvases. I’ve seen the link in the docs to show off the multiple canvas views but as stated there is a limitation that all canvases must be the same size. Where my main canvas will fill your screen and the PiP is limited to 200x400.

5

Seems it is possible to change canvas sizes in this setup - Babylon.js - Multi views demo

1 Like
6

It looks like it yes. But this following note here.

Im unable to use this method has I have 2 activeCameras. A main camera and a GUI camera.

7

So I sort of have a solution and that is to create a separate engine to run a simple scene. The great thing about my setup is that I can grab a whats called a ObjContainer in my scene which can contain other containers and those containers may just have a transformNode with extended class or may also include a mesh/model or other data. I can basically copy this container object and use a small snippet of code to generate the scene that I need.

I just need to solve an issue now of where there is a stall in the main thread for 400-500ms while the engine is created. I might have to run it on a worker thread.

8

The ability to use multiple cameras in one view sounds like an interesting feature, so here’s a PR:

When the PR is merged, this PG will work:

You should see:

image
image814×1022 21.2 KB

The first camera (used for the main rendering) is different for each view and the second camera is the same for both (used for the small rendering in the top left corner).

4 Likes
9

This looks promising. It will mean I can assign my main camera and GUI camera to a single view and then apply a 3rd camera to the second view.

Does this take into account the limitation into a scenes active cameras denoted in the following section?

10

I will update this section of the doc once the PR is merged, as it will no longer apply: a view can be defined with a single or an array of cameras.

2 Likes
11

Thank you for the merge. Multi-view works very nicely.
Theres a bug I need to try and reproduce in the playground.

When initializing the register view for the the pip camera it seems if the cameras maxZ differs greatly from the main cameras maxZ meshes disappear from both views if they are not in the Pips camera frustum and to make it more complicated it only seems to be Line meshes. Other mesh types and model meshes dont seem to be affected.

12

@Evgeni_Popov I wonder if the latest multiview ubo changes are simply not accounted for in the shader material ?

13

@Aus_Karlos Do you reproduce with an older version of Babylon?

1 Like
14

So after more testing its not the MaxZ of the camera but occlusion testing.
Different culling strategies give different results.
Accurate, Strict and Standard seem to affect most of the dash lines
Accurate, Strict and Optimistic Inclusion then bshpere has the least effect.

I dont know why Dash lines are the only meshes to be affected when using registeredView and whats worse it only affects some of them. Some dashlines that are in 1 cameras view are still rendered if only 1 of the 2 cameras can see the mesh. But on some of the dashlines if 1 camera is rotated so that the mesh is not visible in the cameras view its occluded even though the 2nd camera can still see the mesh. This can be either camera.

All other meshes occlusion tests work fine.

I did a regression test back to 6.25.0 and its still there. So the array of camera changes to the registeredView is not the cause.

1 Like
15

Occlusion queries don’t work for multiple cameras, see my answer here:

1 Like