Babylon.JS viewer with React

Hi guys, what is the best way for using the Babylon.JS viewer in React?
I mean i have the following snippet in HTML:

<html>
  <head>
    <title>Babylon.js Viewer - Display a 3D model</title>
    <script src="https://preview.babylonjs.com/viewer/babylon.viewer.js"></script>
  </head>
  <body>
    <babylon
      model="https://raw.githubusercontent.com/ipr0310/testing-babylon/main/wheel.gltf"
      templates.main.params.fill-screen="true"
    ></babylon>
  </body>
</html>

So what i would like to know is “How the Babylon viewer can be used in React”?

A bunch of thanks in advance!

@brianzinn is the king of react :slight_smile:

I personally like to create my own React components which work along-side of BabylonJS. My BabylonJS engine/scene/assetContainer refs are passed through components as a prop. My <canvas> element uses React.createRef(). It’s important to manage your state carefully: I initiate my scene on componentDidMount() at top App level, here we can also load assets and update the state again for any promises (loading etc). I use the FluentUI framework, with the interface aimed for accessibility requirements.

There’s a QueueManager that is good for when my React states update (for controlled components). This is helpful for controls like sliders that want to update frequently but the interval decides when to execute a named function, overwriting any any old/redundant function calls. Here we could also bind observables to a custom actionsManager so BabylonJS can setState without being nuclear…

IE an interval for the queue: setInterval(this.execute, 100);

  execute = () => {
    if (this.queuedFunction && !!this.queuedFunction.func && !!this.queuedFunction.args) {
      this.queuedFunction.func(...this.queuedFunction.args);
      this.queuedFunction = null; // Invalidate it
      this.scene.executeWhenReady(() => {
        this.setState({ loading: false });
        // other actions stuff here
      });
    }
  };

I’d like to hear from others how they work with React but the way I’ve done it has been good for my projects so far, with minimal sugar.

1 Like

To get the viewer running in a React app you just need to render a <babylon ... /> host element.
brianzinn/react-babylon-viewer: Example of integrating babylon viewer in React (github.com)

I was not able to get an element name to render with dots (TS1003 identifier expected). Maybe it works in a JS project @ipr0310?

My personal opinion is that I would build something along the lines of this answer here without the viewer, but it will require detecting the animations and showing controls accordingly - some answers above have different ways as well that do use the viewer:
tag doesn’t work with React - Bugs - Babylon.js (babylonjs.com)

2 Likes

A bunch of thanks!
I will try with the second option

The first option can not render the viewer dynamically!

import { useState } from "react";

const App = () => {
  // templates.main.params.fill-screen="true"

  const [showViewer, updateStatus] = useState<boolean>(false);

  const activateViewer = () => {
    updateStatus(true);
  };

  return (
    <div style={{ width: "100%", height: "100%" }}>
      <button
        onClick={activateViewer}
        style={{ position: "absolute", top: 5, left: 5 }}
      >
        Switch now!
      </button>

      {showViewer && (
        <babylon model="https://raw.githubusercontent.com/ipr0310/testing-babylon/main/wheel.gltf"></babylon>
      )}
    </div>
  );
};

export default App;

1 Like

@brianzinn I tried having the basic functionality of the viewer, is pretty awesome having a scene setup with this tool! ,but i wonder how can i update the camera’s target and rotation once the model is loaded?

This is my current code:

import { Engine, Scene } from "react-babylonjs";
import { Vector3, Color3 } from "@babylonjs/core";
import ScaledModelWithProgress from "./ScaledModelWithProgress";
import "@babylonjs/loaders";
import "@babylonjs/inspector";

export const SceneWithSpinningBoxes = () => {
  return (
    <Engine
      antialias
      adaptToDeviceRatio
      canvasId="babylonJS"
      canvasStyle={{ width: "100%", height: "100%" }}
    >
      <Scene>
        <arcRotateCamera
          name="arc"
          target={Vector3.Zero()}
          position={Vector3.Zero()}
          alpha={-Math.PI / 2}
          beta={0.5 + Math.PI / 4}
          minZ={0.001}
          wheelPrecision={50}
          useAutoRotationBehavior
          allowUpsideDown={false}
          checkCollisions
          radius={2}
          lowerRadiusLimit={0.5}
          upperRadiusLimit={15}
          useFramingBehavior={true}
          wheelDeltaPercentage={0.01}
          pinchDeltaPercentage={0.01}
        />

        <environmentHelper
          options={{
            enableGroundShadow: true,
            groundYBias: 1,
          }}
          setMainColor={[Color3.FromHexString("#ffffff")]}
        />

        <ScaledModelWithProgress
          rootUrl={`3d/`}
          sceneFilename="knight.glb"
          progressBarColor={Color3.FromInts(135, 206, 235)}
          center={Vector3.Zero()}
          modelRotation={Vector3.Zero()}
          scaleTo={1}
          onModelLoaded={() => {
            console.log("Loaded");
          }}
        />
      </Scene>
    </Engine>
  );
};

I set up an enviroment perfectly, but the camera has a target value of Vector3.Zero(), which makes the model looks like this:

So once the model is loaded:

  • How is it possible to target the camera to the loaded model?
    (I mean, how to target the camera to the center of the loaded model)

  • How is it possible to update the camera’s rotation accordingly to the loaded model?

I’m trying to make the camera look like the Sandbox babylon page does:

Looks great what you came up with so quickly! If you want to have it look like the sandbox then I think you need to use the framing behaviour:
Camera Behaviors | Babylon.js Documentation (babylonjs.com)

The sandbox is also written in React, but using the older style of components (instead of hooks). You can see the component that renders the model here (this is how the camera is positioned):
Babylon.js/renderingZone.tsx at master · BabylonJS/Babylon.js (github.com)

Have a look at the onSceneLoaded callback that is triggered when the model loads (that calls that prepareCamera method). If you are going to use that ScaledModelWithProgress component as in your sample code above then you maybe don’t want to pass through scaleTo, etc? Maybe I could make a hooks based Sandbox clone with the react-babylonjs library if that seems useful - as you can see there is not a whole lot of code there. Let me know if you are able to get the camera positioning working as expected. Cheers.

2 Likes

Cool, thanks!. I will try i, the camera i have the useFramingBehavior={true} i’ll share the progress through here! Tbh, i think that would be pretty cool having the sample with Hooks, because a bunch of newcomers to babylon like me would love to find Better ways to integrate tools with React hooks. adding this to the storybook, your source code will be a good reference of good practices in the moment of integrating Babylon+React.

Hi! @brianzinn , i’m still struggling setting up the camera properly and update the rotation/position related to the loaded mesh with the component “ScaleModelWithProgress” even using the framingBehavior to true :cry:

The sandbox even rotates the camera accordingly to the model, mine does not

So i wonder how would you make it with hooks

  • How is it possible to target the camera to the loaded model?
    (I mean, how to target the camera to the center of the loaded model)

  • How is it possible to update the camera’s rotation accordingly to the loaded model?

I have a fallback to update the camera position, but do not know how to access the origin of the model

  const camera: any = useRef(null);

 const onModelLoaded = (e: any) => {
    console.log(e);

// The following properties does not make the camera target the proper mesh
// Like the sandbox does
e.rootMesh._forward
e.rootMesh._position
e.rootMesh.ellipsoid

    if (camera && camera.current && camera.current.setPosition) {
      camera.current.setPosition(new Vector3(2, 2, 2));
    }
  };

Blockquote

Are you calling the same code referenced above:

framingBehavior.zoomOnBoundingInfo(worldExtends.min, worldExtends.max);

From here:
Babylon.js/renderingZone.tsx at master · BabylonJS/Babylon.js (github.com)

That would match the same logic. If you are following that logic then I think we should make a Code Sandbox if you are OK to share your model?

Hi, im’not the owner of the 3d files, i found them from different sources, so i use it for educational purposes!
I did not use that function “zoomOnBoundingInfo”

The file is called “knight.glb”

1 Like

Does this work for you?

export const SceneWithModelComponent = () => {
  const camera = useRef<Nullable<ArcRotateCamera>>(null);

  const onModelLoaded = (e: ILoadedModel) => {
    if (camera && camera.current) {
      if (e.loaderName === "gltf") {
        camera.current.alpha += Math.PI;
      }

      // Enable camera's behaviors (done declaratively)
      // camera.current.useFramingBehavior = true;
      var framingBehavior = camera.current.getBehaviorByName(
        "Framing"
      ) as FramingBehavior;
      framingBehavior.framingTime = 0;
      framingBehavior.elevationReturnTime = -1;

      if (e.rootMesh) {
        camera.current.lowerRadiusLimit = null;

        var worldExtends = e.rootMesh
          .getScene()
          .getWorldExtends(
            (mesh) =>
              mesh.isVisible &&
              mesh.isEnabled() &&
              !mesh.name.startsWith("Background") &&
              !mesh.name.startsWith("box")
          );
        console.log("zoomOnBoundingInfo", worldExtends.min, worldExtends.max);
        framingBehavior.zoomOnBoundingInfo(worldExtends.min, worldExtends.max);
      } else {
        console.warn("no root mesh");
      }

      camera.current.pinchPrecision = 200 / camera.current.radius;
      camera.current.upperRadiusLimit = 5 * camera.current.radius;
    }
  };

  return (
    <Engine>...</Engine>
  )
}

You had already declaratively set many of the ArcRotateCamera options. Note that the main difference with the Sandbox is that I am excluding the “BackgroundHelper”, “BackgroundPlane” and “BackgroundSkybox” - otherwise your bounds will be {-10, -10, -10} and { 10, 10, 10 }. You have the mesh names of the fallback component starting with “box” and they need to be excluded as well. If the Model you are loading happens to match that predicate then you will want to do something more precise.

1 Like

I tried the snippet you recommended, but i do get a bug
the following function “line” is the cause of the bug

 framingBehavior.zoomOnBoundingInfo(worldExtends.min, worldExtends.max);

You can see in the picture that the following console log works nicely!

console.log("zoomOnBoundingInfo", worldExtends.min, worldExtends.max);

The bug:

So if i comment that line

console.log("zoomOnBoundingInfo", worldExtends.min, worldExtends.max);
// framingBehavior.zoomOnBoundingInfo(worldExtends.min, worldExtends.max);

The model is shown like this:

You can watch the snippet here:

It was this part of your code (you need to remove the attribute onMeshTargetChangedObservable or pass in a callback function):

<arcRotateCamera ... onMeshTargetChangedObservable={true} />

What the renderer was doing was calling:

arcRotateCamera.onMeshTargetChangedObservable.add(true);

Then the notifyObservers was calling the callback function on the true primitive (which matches the error you are getting).

I will add a warning if a non-function is added. The whole observable thing from a renderer perspective is a bit odd - there is a new documentation site coming up and I will make sure to note this there as well. Sorry for the late reply - that was a bit of a sinister error.

edit: remove also scale={1} from ScaledModelWithProgress and it looks like this:

Hello! It works!!!
But there is a little problem, the camera has no a proper behavior when loading different 3d files.

You can watch the list of all files here

I will show you the behavior error the camera gets with different models.
All models loads correctly on sandbox.babylonjs.com
You can try my repo, everything is there! GitHub - ipr0310/react-babylon-viewer

Avocado.gltf
(Does not load)

box.glb
(Does not load, weirdly no logs appear)

island.glb
(Camera loads with no proper zoom to model, models flickers a lot)

robo.glb
(Camera loads with no proper zoom/rotation to model)

rims.gltf
(Camera loads with no proper zoom/rotation to model)

selva.glb
(Camera loads with no proper zoom/rotation to model)

soldier.glb
(Camera loads with no proper rotation to model, models flickers a lot)

spider.glb
(Camera loads with no proper rotation to model, models flickers a lot)

star.gltf
(Camera loads with no proper zoom/rotation to model, models flickers a lot)

----In case you are interested with the file which works :clap::clap::clap:

eyes-a.glb
(It works, hoorah!)

knight.glb
(It works, hoorah!)

ny.glb
(It works, hoorah!)

pencil.glb
(It works, hoorah!)

prayer.glb
(It works, hoorah!)

statue.glb
(It works, hoorah!)

hi @ipr0310 - sorry been a bit busy lately, but I did fork your repo and start looking. I will need to look more at the sandbox code and also at some babylonjs code (ie: createDefaultCamera, createDefaultSkybox).
brianzinn/react-babylon-viewer-1 (github.com)

The flicker may be due to how the environment is set. You are using the environment helper, while the sandbox creates a skybox with environment texture:
Babylon.js/renderingZone.tsx at master · BabylonJS/Babylon.js (github.com)

I will try to see if I can add a declarative way to do that. Also, FYI you are using the gltf plugin if you are looking at how lighting is set.

Additionally, for the avocado - it doesn’t load because you are missing the accompanying bin file.

1 Like

Ok! so based on your fork, will it have the same behavior as my GH repo or you made big changes?

my fork has most of the same issues, but you can switch models easy on it with a dropdown in my fork. one thing I had noticed was that the order of changing models changes the result, so I think we need to do some kind of reset between models. the other things to look for will be duplicating (or calling directly the createDefault() helpers). I think there are 2 issues:

  1. flicker - verify environment and check model for PBR materials
  2. camera framing - seems like we need a reset between models.