Babylon in React

Hi!

It may be a dumb question, but do

var createScene = function () {

    return scene;

};

in Babylon PlayGround and SceneComponent.jsx with

c

onst onSceneReady = (scene) => {
    // something
};

const onRender = (scene) => {
    // something
};

export default () => (
  <div>
    <SceneComponent antialias onSceneReady={onSceneReady} onRender={onRender} id="my-canvas" />
  </div>
);

work in the exact same mechanism?

If so, just to be clear, once scene is created, it doesn’t re-create a scene when it requires an update (when an object moves or material color changes for examples), but it just applies differences to the scene created earlier. Is it correct?

The more I play with Babylon, it gets way more fun, but I am now getting worried about the performance when it’s used with React. I would like to keep the life cycle as performant as possible.

Happy coding, guys!! :smiley: Thanks always!

1 Like

Hey @Deltakosh and perhaps @msDestiny14 (not too sure who else to ask :slight_smile: ) got any clarification?

My uninformed take is that in our playground, the scene is created once, and only once on page load. It still renders each requestAnimationFrame regardless of whether or not the scene changed. Also, in order to apply any differences in scene, you would either subscribe to a rendering observable in your scene initialization code, or grab some reference to the things in scene you want to change from some other component?

I’m not sure about the react component lifecycle, but I’d assume that onSceneReady would be called on SceneComponent creation? I’m not too sure how we handle rendering…I’d assume that you could hook the render loop into react’s render() callback to only render when needed by the parent component, or setup your own renderloop to render some continuous scene…

1 Like

Adding @brianzinn

1 Like

hi @DOEHOONLEE - good question. Thinking about performance is crucial, especially since developers are often on fast machines. Good that you are asking this. One important thing about using React in babylonjs is the concept of rendering and performance.

In React your component will re-render every time there is a prop change (there are things like React.Memo and shouldComponentUpdate ( Optimizing Performance – React (reactjs.org)). Those are used to determine if your component needs to be reconciled or not and are DOM rendering considerations, so just for things like div and span.

In babylonjs your scene will usually render every frame. If you are 60fps then 60 times per second. The ‘onRender’ callback you have defined there will run that frequently and must be highly optimized. Your onSceneReady will run once. So, typically you would do all of your scene setup and then the onRender you will see often for things like rotation on a spinning mesh. If you are not interacting with the screen with input (like rotation, zoom) then you could explicitly call render instead having the runRenderLoop call it for you ( Engine | Babylon.js Documentation (babylonjs.com)), using runRenderLoop I think is the most common scenario for interactive scenes. You will need to render whenever anything in your scene changes including camera perspective. I have seen attempts to reduce those render loops and there is a lot of potential there if you can track things like pointer events and camera observables to reduce rendering when the scene is static. You need to be careful that when your component unmounts that the scene and engine are disposed and that component you have does it for you if you got it from the examples. You can also do an observable and then your SceneComponent is off screen you could stop rendering using something like an Intersection Observer, which needs polyfills ( Intersection Observer API - Web APIs | MDN (mozilla.org)).

So, you have 2 render loops that essentially work completely irrespective of each other (react and babylon) and they don’t directly affect each other’s performance - one is in the DOM and one is on the canvas. In your onSceneReady you would need to get references to any objects you need to be able to interact with them from React (like the material color changes you mention would need a reference). Hope that is helpful. Please ask more questions - it is a really interesting topic for sure and would be great to hear how you are improving performance.

2 Likes

@Drigax @Deltakosh @brianzinn Thank you all for your help!!

@brianzinn What you’re saying is that

  1. All my scene setups (declaration, defining for scene, camera, materials, meshes, etc.) goes under “onSceneReady”
  2. and “onSceneReady” runs ONCE only
  3. and everything that I would want to update(such as rotation, color change, material change, etc.) goes into “onRender,”

If this is what you meant, it makes everything just way much clearer!

Looking at documentation on Babylon website,

it appears to me that with useEffect, if there is a change on “reactCanvas,” wouldn’t engine, scene, resize, or even eventListener, and especially “onSceneReady” are all declared inside be repeating? and wouldn’t this be unnecessary or bad for performance?

You said that if everything is well designed and optimized, onSceneReady should run once only, but I am not quite sure how this works :frowning:

I feel a few steps closer to Babylon though!! :+1: :+1:

1 Like

The useEffect has as a second parameter a dependency array. So, what is inside the useEffect will run every time that the dependency array changes. The only item in the dependency array is the DOM canvas and that should only change once when the canvas element is initially rendered to the page.

When the useEffect returns a function that function will run when the effect is re-ran/unmounted. In this case when your Scene Component is unmounted (removed) then the engine will dispose automatically.

You can verify that yourself just by looking in the developer console, since every time an engines is created the details are logged to console.

So, with all of that in hand then hope that is all clear. In terms of actual performance improvements you will probably be able to get the biggest improvements by reducing draw calls, instancing/merging meshes, octrees, etc. or some of the many different optimizations that are not React specific. Once you get your application further along then you will be able to ask more questions that are babylon specific.

1 Like

Let me apologize. I think I said it wrong.

Other than constant changes like rotation, updates like color change shouldn’t be inside “onRender.”

but in that case, where should I update my materials/meshes then?

  1. All my scene setups (declaration, defining for scene, camera, materials, meshes, etc.) goes under “onSceneReady”
  2. and “onSceneReady” runs ONCE only
  3. and everything that I would want to update(such as rotation, color change, material change, etc.) goes into “onRender,”

Also, I just noticed that @brianzinn you’re from Vancouver, BC. I am currently working in Korea, but kinda grew up in Vancouver too! Nice to see you! :slight_smile:

2 Likes

Sounds like you’re getting it :slight_smile: Typically you want initialization related things like mesh creation/loading to happen at scene initialization/onSceneReady, or asychronously happen via callbacks you setup for certain events.

Then your render loop should be as lean as possible. Grab user input (of you are polling instead of setting up input callbacks :slight_smile: ), update the scene in response to the input/things you want to change from frame to frame. Depending on how you want to change your color, you may want this to happen in your onRender callback as well. Updating material uniforms like baseColor isn’t terribly expensive.

Its key that we minimize the amount of additional initialization you need to do each frame as well. If we block the JS thread synchronously doing expensive things like synchronously loading meshes/other assets onRender, you’ll notice drops in framerate.

Thanks @brianzinn :slight_smile:

1 Like

You can do that any time and outside of scene initialization and render loop - even from React or from button clicks in React or babylon events like actions - You need to track the reference to your object if you are doing so from React. Then on the next babylonjs scene render your changes will take effect automatically, which is basically immediately. If you only have access to your scene then you can search through the meshes or use methods like getMeshByName ( Scene | Babylon.js Documentation (babylonjs.com)). That may start to get cumbersome if you are tracking lots of objects. Depending on what you are building it can be convenient to use a reconciler (react-babylonjs) that will track your objects and send changes through automatically and allow some composition.

I’m actually moving from Vancouver this month, so maybe I should change my profile!

@brianzinn hope you enjoyed Vancouver!
@Drigax

I just made some changes in my code and wanted to share! I would love some feedbacks on it.

Thanks!!

App.js

import React from "react";
import { FreeCamera, Vector3, HemisphericLight, MeshBuilder } from "@babylonjs/core";
import SceneComponent from "./SceneComponent"; // uses above component in same directory
function App() {
  return (
    <div className="App">
      <SceneComponent id="my-canvas" />
      <img height={100} width={100} src="https://www.hackingwithswift.com/uploads/matrix.jpg" />
      <img height={100} width={100} src="http://www.gregorybufithis.com/wp-content/uploads/2016/05/random-numbers.jpg" />
      <img height={100} width={100} src="https://upload.wikimedia.org/wikipedia/commons/thumb/3/36/Two_red_dice_01.svg/1200px-Two_red_dice_01.svg.png" />
    </div>
  );
}
export default App;

SceneComponent.jsx

import {
  Engine,
  FreeCamera,
  HemisphericLight, MeshBuilder,
  Scene,
  Vector3,
  Color3,
  StandardMaterial, Texture
} from '@babylonjs/core'
import React, { useEffect, useRef, useState } from 'react'
let box;
let myTexture;
let imageTexture;
const onSceneReady = (scene) => {
  // This creates and positions a free camera (non-mesh)
  var camera = new FreeCamera("camera1", new Vector3(0, 5, -10), scene);
  // This targets the camera to scene origin
  camera.setTarget(Vector3.Zero());
  const canvas = scene.getEngine().getRenderingCanvas();
  // This attaches the camera to the canvas
  camera.attachControl(canvas, true);
  // This creates a light, aiming 0,1,0 - to the sky (non-mesh)
  var light = new HemisphericLight("light", new Vector3(0, 1, 0), scene);
  // Default intensity is 1. Let's dim the light a small amount
  light.intensity = 0.7;
  // Our built-in 'box' shape.
  box = MeshBuilder.CreateBox("box", { size: 2 }, scene);
  myTexture = new StandardMaterial("texture", scene);
  myTexture.diffuseColor = new Color3(0,0,0);
  imageTexture = new StandardMaterial("imageTexture", scene);
  box.material = myTexture;
  // Move the box upward 1/2 its height
  box.position.y = 1;
  // Our built-in 'ground' shape.
  MeshBuilder.CreateGround("ground", { width: 6, height: 6 }, scene);
};
/**
 * Will run on every frame render.  We are spinning the box on y-axis.
 */
const onRender = (scene) => {
  if (box !== undefined) {
    var deltaTimeInMillis = scene.getEngine().getDeltaTime();
    const rpm = 10;
    box.rotation.y += (rpm / 60) * Math.PI * 2 * (deltaTimeInMillis / 1000);
  }
};
const SceneComponent = () => {
  const reactCanvas = useRef(null);
  const [scene, setScene] = useState(null);
  const [imgSrc, setImgsrc] = useState("");
  const changeColor = (targetMaterial, targetTexture, keyNum) => {
    if (keyNum == "37") {
      targetTexture.diffuseColor = new Color3(1,1,1);
      targetMaterial.material = targetTexture;
    }
    else {
      targetTexture.diffuseColor = new Color3(0,0,0);
      targetMaterial.material = targetTexture;
    }
  }
  const changeImage = (targetMaterial, targetTexture, src, scene) => {
    targetTexture.diffuseTexture = new Texture(src, scene);
    targetMaterial.material = targetTexture;
  }
  useEffect(() => {
    if (reactCanvas.current) {
      const engine = new Engine(reactCanvas.current);
      const scene = new Scene(engine);
      if (scene.isReady()) {
        onSceneReady(scene);
      } else {
        scene.onReadyObservable.addOnce((scene) => onSceneReady(scene));
      }
      engine.runRenderLoop(() => {
        if (typeof onRender === "function") {
          onRender(scene);
        }
        scene.render();
      });
      const resize = () => {
        scene.getEngine().resize();
      };
      setScene(scene);
      if (window) {
        window.addEventListener("resize", resize);
      }
      window.addEventListener("keydown", (e) => {
        if (e.keyCode == "37") {
          changeColor(box, myTexture, e.keyCode);
        }
        else if (e.keyCode == "39") {
          changeColor(box, myTexture, e.keyCode);
        }
      })
      window.addEventListener("pointerdown", (e) => {
        console.log(e.target.src);
        setImgsrc(e.target.src);
      })
      return () => {
        scene.getEngine().dispose();
        if (window) {
          window.removeEventListener("resize", resize);
        }
      };
    }
  }, [reactCanvas]);
  useEffect(() => {
    changeImage(box, imageTexture, imgSrc, scene)
    console.log(imgSrc)
  }, [imgSrc])
  return <canvas width={600} height={600} ref={reactCanvas} />;
};
export default SceneComponent;

Have a wonderful day, guys!

hi @DOEHOONLEE - looks really good! I see you connected all the concepts together - it’s great to see.

You don’t really need to change anything, but since you are asking for feedback, I’d make small changes, but none are required.

  1. I would leave the (re-usable) SceneComponent alone and try to break apart what makes your application unique into a separate component. This composition will make it clearer and separate the two ideas. You took the two components from (Babylon.js and React | Babylon.js Documentation (babylonjs.com) and merged them, but if you prefer it that way then keep it as-is :slight_smile:
  2. Using let box; won’t work with HMR/fast refresh. Depending on how you want your DX (Developer eXperience), you could instead useRef and see your scene change while you are saving. Some changes still require a refresh, though. Move then all code inside your Component.
  3. Not using base imports can help with tree-shaking (making your bundle size smaller) and make your page load faster. If you are using VSCode then you can get some help from the light bulb
    image. More info here:
    Babylon.js ES6 support with Tree Shaking | Babylon.js Documentation (babylonjs.com)
  4. Make sure to remove those event listeners - they will keep adding if you are using for example a router and navigating back to that page.

Thanks for sharing your progress. I can see actually that I should probably update the example. I think I was trying to get the example to as few lines as possible and to not distract from the concepts, but should have been also more mindful to show best practices.

Thank you for the detailed explanation!!

For #1, yes, I still haven’t decided if I should keep two separate components or just have one merged one.

May I ask what HMR stands for in #2?

#4, wow! I totally missed that part!! Thanks for pointing out!!

I spent last weekend thinking about this and making it into a JS class @_@

having a hard time lol

Hope you guys all enjoyed your weekend!!

1 Like

HMR stands for Hot Module Reloading. Fast Refresh is a more modern enhancement that applies updates and maintains component state. If you declare your “ref” outside of the component then when you change your component and hit save, the framework won’t be able to maintain your state (that’s why I would recommend switching to useRef there).

Thanks for the suggestion! That totally makes sense :slight_smile: