CameraGizmo with ReactJS

tfmertz · October 5, 2022, 12:12am

Playground example non-reactJS: Babylon.js Playground

Hello,

I’m having trouble getting the CameraGizmo to render using ReactJS-Babylon. As you can see in the playground above it’s a pretty simple example, but using the react-babylonjs library, it just doesn’t seem to take, I’ve tried multiple different versions of this code, but here’s the first iteration that is very similar to my working code for mesh gizmos:

<utilityLayerRenderer>
    {/* add the second render camera */}
    <freeCamera
        name="renderCam"
        position={new Vector3(0, 0, 2)}
        cameraDirection={new Vector3(0,0,-1)}
        scene={scene}
        ref={renderCameraRef}
    >
        <cameraGizmo ref={gizmoRef} camera={renderCameraRef.current} />
    </freeCamera>
</utilityLayerRenderer>

From what I can tell logging out the gizmoRef, the cameraGizmo camera attribute is always null, even when set from the the renderCameraRef.current, which I log out as well. The cameraGizmo.attachedNode is correctly set to the freeCamera, but it never renders.

Any help is appreciated!

Thanks,

Tom

tfmertz · October 5, 2022, 12:21am

I guess I found a solution, it doesn’t feel very reacty, but here’s code that seems to be working, we just brute force it on ever render check if we don’t have a camera and set it to the refs. Is there a more elegant way?

const RenderCamera = (props) => {
  const renderCameraRef = useRef();
  const gizmoRef = useRef();

  if (renderCameraRef.current && gizmoRef.current && !gizmoRef.current.camera) {
    gizmoRef.current.camera = renderCameraRef.current;
  }

  return (
    <>
      {/* add the render camera */}
      <freeCamera
          name="renderCam"
          position={new Vector3(0, 0, 2)}
          cameraDirection={new Vector3(0,0,-1)}
          ref={renderCameraRef}
      >
        {renderCameraRef.current && <cameraGizmo ref={gizmoRef} camera={renderCameraRef.current} />}
      </freeCamera>
    </>
  );
}

brianzinn · October 5, 2022, 12:24am

Looks like a bug in react-babylonjs… maybe you are the first one to try declarative camera gizmo?. The camera prop wouldn’t be set by renderer I don’t think - could use an update to the lifecycle listener to attach.

Maybe similar to how light gizmo connects should work otherwise:

github.com

brianzinn/react-babylonjs/blob/master/packages/storybook/stories/babylonjs/Basic/gizmoManager.stories.js

import { Color3 } from "@babylonjs/core/Maths/math.color";
import { Vector3 } from "@babylonjs/core/Maths/math.vector";
import React, { useEffect, useState } from "react";
import { Engine, Scene, useScene } from "react-babylonjs";
import "../../style.css";

// export default { title: 'Babylon Basic' };

const Inspector = () => {
  const scene = useScene();
  scene.debugLayer.show();
  return null;
};

// broken in 4.2 - works in 5.0
const GizmoManager = () => {
  const [lightRef, setLightRef] = useState(undefined);

  useEffect(() => {
    console.log("light has been set", lightRef);

This file has been truncated. show original

tfmertz · October 5, 2022, 1:31am

Thanks for the quick reply @brianzinn. Maybe a dumb question, but is there a way to view that story in the story book like the Babylon Playground? I’m not seeing it on this page, which has been really helpful for learning React BabylonJS by example.

Another ask, how were you able to determine it was a bug with the camera prop not being set by renderer so quickly. I’ve love a bit of insight into how to track these issues down so that I don’t have to rely solely on the forums. Teach me how to fish so to speak I’m loving the package, but one of the larger points of friction is that some times its hard to translate BabylonJS into React and get it working reliably.

Any insight or code or docs to read that would be greatly appreciated!

Thanks,

Tom

tfmertz · October 5, 2022, 3:23pm

I think I found another bug? Not sure if I’m just using the react components a way they aren’t meant to be used. The onClickedObservable doesn’t seem to be firing no matter how I set it on the component:

const RenderCamera = (props) => {
  const renderCameraRef = useRef();
  const gizmoRef = useRef();

  if (renderCameraRef.current && gizmoRef.current && !gizmoRef.current.camera) {
    gizmoRef.current.camera = renderCameraRef.current;
    const obs = new Observable();
    obs.add((data, state) => {
      console.log(data);
      console.log(state);
    })
    gizmoRef.current.onClickedObservable = obs;
    console.log(gizmoRef.current);
  }


  return (
    <>
      {/* add the render camera */}
      <freeCamera
          name="renderCam"
          position={new Vector3(0, 0, 2)}
          cameraDirection={new Vector3(0,0,-1)}
          ref={renderCameraRef}
      >
        {renderCameraRef.current &&
          <cameraGizmo
            ref={gizmoRef}
            camera={renderCameraRef.current}
            scaleRatio={25}
            updateScale={false}
            // onClickedObservable={observable}
          />
        }
      </freeCamera>
    </>
  );
}

The console.log of gizmoRef.current shows the observable and the observer I added to it. But clicking on the gizmo doesn’t log anything, maybe I missed a setting to make the camera clickable?

I’ve added a camera and a gizmo and observerable through basic BabylonJS in my react app and it works correctly, just like the playground below. Example of this:

export const cameraGizmoInit = (camera) => {
    const cameraGizmo = new CameraGizmo();
    cameraGizmo.scaleRatio = 25;    
    cameraGizmo.updateScale = false;
    cameraGizmo.camera = camera;
    const obs = new Observable();
    obs.add((e, s) => {
        console.log(e);
        console.log(s);
    })
    cameraGizmo.onClickedObservable = obs;
    return cameraGizmo;
}

brianzinn · October 5, 2022, 4:08pm

onClickedObservable I believe works differently than you are expecting. You shouldn’t be assigning directly to an observable property (setter).

// do not do this:
cameraGizmo.onClickedObservable = myObject;

You should be calling the add method it has instead.

 gizmo.onClickedObservable.add(() => console.log);

if you are going the declarative route then it should work with just a function signature:

// you can memoize this or useCallback with dependency array, etc.
onGizmoClick = (e) => console.log;

return (
   ...
   <cameraGizmo ... onClickedObservable={onGizmoClick} />
)

Here is the API ref:
CameraGizmo | Babylon.js Documentation (babylonjs.com)

There is a link “search playground for…” - sometimes following that leads to helpful results as well.

I think that’s not entirely unexpected and the docs section for react-babylonjs is not helping. I do have a new site that will one day come online and it does explain a lot of how things really work and how to properly translate. I just cannot seem to find time to work on it - right now also the newer babylonj.js versions changed their target compatibility, so my CI/CD pipeline broke, so there won’t be any releases until I sort that out as well!

tfmertz · October 6, 2022, 12:12am

No worries! Thanks for all of your time and guidance . It’s a lot of work maintaining a project and keeping up with one as large as BabylonJS.

I was able to get the camera gizmo’s onClickedObservable working, just not through the declarative react component, I ended up removing the gizmo and creating it with regular babylon. Everything seems to be rendering and clicking is working as well. I’ll post what worked for me and mark as solution so others can hopefully benefit:

const RenderCamera = (props) => {
  const renderCameraRef = useRef();

  const [gizmo, setGizmo] = useState(null);

  const onClick = (e, s) => {
    console.log(e);
    console.log(s);
  }

  if (renderCameraRef.current && !gizmo) {
    const cameraGizmo = new CameraGizmo();
    cameraGizmo.camera = renderCameraRef.current;
    if (callback) {
      cameraGizmo.onClickedObservable.add(onClick)
    }
    setGizmo(cameraGizmo);
  }

  return (
    <>
      <freeCamera
        name={"freeCam"}
        position={Vector3.Zero()}
        rotation={Vector3.Zero()}
        ref={renderCameraRef}
      />
    </>
  );
}

brianzinn · October 6, 2022, 3:57pm

Glad that works for you. I can make a couple of small changes to react-babylonjs and this would work (right now the renderer is ignoring the camera prop):

const onGizmoClicked = (e, s) => console.log

return (
  <freeCamera ...>
    <cameraGizmo camera={cameraRef} scaleRatio={25} ... onClickedObservable={onGizmoClicked} />
   </freeCamera>
)

With another couple of small changes I could do a deferred instantiation until there is a camera in the tree graph, so when you declare as a child component of a camera that it would automatically assign the camera/light. Right now that is already done, but only for attaching to node/mesh. For example, this automatically attaches a LightGizmo to a mesh and works on current version:

<utilityLayerRenderer>
    <gizmoManager
      thickness={3}
      positionGizmoEnabled
      rotationGizmoEnabled
    >
      <sphere name="sunMesh" diameter={1}>
        <lightGizmo light={lightRef} updateScale />
      </sphere>
    </gizmoManager>
</utilityLayerRenderer>

Anyway, I’d be happy to work on a declarative solution if you would find it useful. There would still be an opt-out escape hatch skipAutoAttach:
react-babylonjs/CustomProps.ts at master · brianzinn/react-babylonjs (github.com)

cheers.

tfmertz · October 7, 2022, 1:29am

Thanks @brianzinn ! Yes I was referencing that exact lightGizmo example when I was setting up the cameraGizmo and I think that’s why I struggled so much with the cameraGizmo, because there needs to be that deferring step for the camera. (I actually ran into this later with some camera renderTargetTextures, my “temporary” solution was a rather ugly setTimeout)

As for adding it, it’s not urgent for me. I have a workaround solution, though I do love the declarative approach . But I realize you’re busy.

I can give you my perspective as an outside developer that the difference between how the lightGizmo and cameraGizmo worked threw me for a big loop and made me start questioning my sanity . My developer brain couldn’t get past “lightGizmo works like this, cameraGizmo has to be the same!”

If you don’t mind me asking, I thought the lightGizmo and cameraGizmo were generated by Babylon.JS types, it sounds like there is some logic happening to handle the attaching for the lightGizmo onto a mesh, do you mind sharing a link to the github where that code is for me? I wouldn’t mind digging a bit more into the inner workings to gain some more insights.

Thanks for your time!

brianzinn · October 7, 2022, 2:37am

Would be super happy to have more people checking out the codegen parts.

It’s a bit restrictive, but the object properties can be assigned with an allow only list. I actually added the light explicitly for the LightGizmo. Just need two lines:

There are plans for a v4 and I do away with most of the codegen and just connect whatever is passed along. It will allow a way to register the other libraries like declarative @babylonjs/materials:
Extensions - Grid Material Example ⋅ Storybook (brianzinn.github.io)

The other thing is that I probably still want to know the type, so that I can coerce like I do with this ChromaJS example:
Integrations - chroma-js ⋅ Storybook (brianzinn.github.io)

But also there are conversions to numeric arrays for the integration with react-spring, etc. Anyway, adding light is easy, but it’s getting to the point of needing a major refactor…

Topic		Replies	Views
Camera view in React component Questions	2	721	April 7, 2021
Activate two camera with different viewports with react-babylonjs Questions react	5	592	May 3, 2021
Gizmo and Multiviews Questions	16	2030	June 5, 2019
React-babylonjs v2.0 Announcements	4	1814	December 17, 2019
How can I define Gizmo in React and Babylon JS, give me an initial boilerplate Questions react , tutorial , gizmo	12	996	August 16, 2021

CameraGizmo with ReactJS

Related topics