Multiple Scenes in React Application

I have a question regarding multiple scenes in React. What is the best way to create two scenes. One MainScene which is over the full window and one little scene which will be used in the bottom left corner for a navigation cube. Both scene will be displayed at the same time.

I have found this site How to use Babylon.js with ReactJS - Babylon.js Documentation with which I build my react babylon application. Now the Scene gets created in the following Code. How is the best way to add another scene in this code or somewhere else? And if so how?

import { Engine, Scene } from '@babylonjs/core'
import React, { useEffect, useRef, useState } from 'react'

export default (props) => {
    const reactCanvas = useRef(null);
    const { antialias, engineOptions, adaptToDeviceRatio, sceneOptions, onRender, onSceneReady, ...rest } = props;

    const [loaded, setLoaded] = useState(false);
    const [scene, setScene] = useState(null);

    useEffect(() => {
       if (window) {
           const resize = () => {
               if (scene) {
                   scene.getEngine().resize();
               }
           }
           window.addEventListener('resize', resize);

           return () => {
               window.removeEventListener('resize', resize);
           }
       }
   }, [scene]);

   useEffect(() => {
       if (!loaded) {
           setLoaded(true);
           const engine = new Engine(reactCanvas.current, antialias, engineOptions, adaptToDeviceRatio);
           const scene = new Scene(engine, sceneOptions);
           setScene(scene);
           if (scene.isReady()) {
               props.onSceneReady(scene)
           } else {
               scene.onReadyObservable.addOnce(scene => props.onSceneReady(scene));
           }

           engine.runRenderLoop(() => {
               if (typeof onRender === 'function') {
                   onRender(scene);
               }
               scene.render();
           })
       }

       return () => {
           if (scene !== null) {
               scene.dispose();
           }
       }
   }, [reactCanvas])

   return (
        <canvas ref={reactCanvas} {...rest} />
   );
}

Adding @syntheticmagus and @bghgary for their thoughts.

Hi Patrick_Luthi,

Really cool question! If there’s React-specific integration code for this, I’m not familiar with it; but in general, I think you should just be able to create and render the second scene the same way you rendered the first, as discussed in this docs page. For the “picture-in-picture” scenario you’re describing, all you should have to do is render the smaller picture second, leave clearing enabled, and set the viewport for the second scene’s camera to the area of the screen you want covered by that rendering. Another option would be to render the smaller scene first to a render target texture, then use that texture in the first scene, but I suspect that’s probably more elaborate than is necessary. Hope this is helpful, and best of luck!

3 Likes

If you want to use multi-camera and viewport then your code ends up something like this:

function onSceneMount(e: SceneEventArgs) {
  const { canvas, scene } = e

	var camera1 = new ArcRotateCamera("camera1",  3 * Math.PI / 8, 3 * Math.PI / 8, 10, new Vector3(0, 1, 0), scene);
  camera1.attachControl(canvas, true);
  
  var camera2 = new ArcRotateCamera("camera2",  7 * Math.PI / 8, 7 * Math.PI / 8, 10, new Vector3(0, 1, 0), scene);
	camera2.attachControl(canvas, true);

  camera1.viewport = new Viewport(0, 0.5, 1, 0.5);
  camera2.viewport = new Viewport(0, 0, 1, 0.5);
    
  scene.activeCameras.push(camera1);
  scene.activeCameras.push(camera2);
  
  scene.getEngine().runRenderLoop(() => {
      if (scene) {
          scene.render();
      }
  });
}

pointer events get interesting with viewports/scenes. You can create a second scene right after the first and then handle multi-rendering:

runRenderLoop(() => {
  // ..
  scene1.render();
  scene2.render();
})

I think for top scene it’s scene.autoClear = false; and it needs an alpha back color. I actually haven’t rendered two babylonjs scenes together on same canvas, only Babylon with PixiJS. This question does not look React specific, so you should be able to create a playground. Let us know how that works and maybe we can help easiest with a PG. cheers.

2 Likes

This solution is actually the cleanest and works the best for us. One thing that we are stuck on and we need to fix for it to be working, is that we need to ignore some meshes from being picked in the viewport. Is there any option to set meshes, for just one specific camera, not pickable?

I am not aware of a way to do set meshes only pickable for a specific camera! Let me take you down a rabbit hole and maybe that will provide a solution. I think you need to see which viewport was clicked on to know which camera and then use that. I’m just going to think out loud here and hopefully this will work for you. Otherwise you’ll need to explain more about your viewports/cameras.


const skipGroundPredicate = (mesh: AbstractMesh) => {
  return mesh.name !== 'ground' ;
}

function onPointerDown(evt: PointerInfo, scene: Scene) {
  if (evt.event.button !==0) {
    return
  }

  const x = scene.pointerX;
  const y = scene.pointerY;
  const renderWidth = scene.getEngine().getRenderWidth(true);
  const renderHeight = scene.getEngine().getRenderHeight(true);
  scene.activeCameras.forEach(c => {
    const y1 = renderHeight - y;
    const absoluteViewport = c.viewport.toGlobal(renderWidth, renderHeight);
    
    if (x > absoluteViewport.x && y1 > absoluteViewport.y) {
      if (x < absoluteViewport.x + absoluteViewport.width && y1 < absoluteViewport.y + absoluteViewport.height) {
        var pickinfo = scene.pick(x, y, skipGroundPredicate, true, c);
        if (pickinfo && pickinfo.hit) {
           const pickedMesh = pickInfo.pickedMesh;
        }
      }
    }
  })
  
}

const App = () => (
  ...
  <Scene onSceneMount={onSceneMount} onScenePointerDown={onPointerDown} />
)

I would have a look here.
https://doc.babylonjs.com/api/classes/babylon.scene#pick

If you need to pick “behind” a mesh then look at the multipick. You could also in the pointerdown before the scene.pick set your meshes isPickable property according to the camera. Let me know if that works out. Viewports can be tricky to work with… note that I am passing the camera into the scene.pick that is the main take away. Additionally I think you could inline the predicate function and get the camera through closure.

Just thought of another thing is that you could create your own pickMask that works like layer masks and may work for you.

mesh.pickMask = 0x10000000;

You could then assign a mask for a camera and multipick that way. If I had another coffee then I could probably write that one! :slight_smile: You’ll find all the code you need in babylon internals, but would end up like this (the single & is a bitwise and operator):

if (mesh.pickMask & camera.pickMask !== 0) {
   // mesh is for this camera
}

Is it possible to have two cameras usable for pick events at the same time?
We need to use a button inside the cameraView which is not used for Pointers.

@Patrick_Luthi You can ray cast from the camera on a button click.

let forwardRay = activeCamera.getForwardRay()

The ray has properties like direction and your camera has a position/fov, so you can do a lot with that for picking. Is that what you are after when you say not used for Pointers?

1 Like