Problem with connect redux useSelector with babylon js

I am now developing a react-based Babylon app, the process is going smoothly and I am able to dispatch my action payload from Babylon world to my store in redux, and use some param from Babylon passing to react component as some props, (like I have an Id Info component in react, and I will dispatch the Id of my mesh in my Babylon world to react, it works perfectly).

So my problem now is when I try useSelector as the way that redux can pass some updated stuff inside Babylon, it always gives me only the initalState from redux, no matter the param from selector I pass is in onSceneReady, or onRender hook (You might’ve telled I am using Scene Component),.

Idk if there is a way I can put my case in the playground, so if my question is lacking some important information, I will add on.

Thanks,
Johnson

@RaananW or @brianzinn could probably help.

I think you might have a stale reference. Can you share more of the code?

export const BabylonMain = () => {
  const dispatch = useAppDispatch();
  const { boxInfo } = useAppSelector((state) => (state) => state.babylon3d);

  const onSceneReady = (scene: any) => {
    // This creates and positions a free camera (non-mesh)
    var camera = new FreeCamera("camera1", new Vector3(0, 5, -5), scene);

    // This targets the camera to scene origin
    camera.setTarget(Vector3.Zero());

    //Prevent too fast zooming
    // camera.wheelPrecision = 50;

    const canvas = scene.getEngine().getRenderingCanvas();

    // This attaches the camera to the canvas
    // camera.upperBetaLimit = Math.PI / 2.2 // limit the z axis no less zero
    camera.attachControl(canvas, true);
    scene.clearColor = new Color4(0.109803922, 0.109803922, 0.117647059, 1);

    // This creates a light, aiming 0,1,0 - to the sky (non-mesh)
    var light = new HemisphericLight("light", new Vector3(0, 1, -1), scene);

    // Default intensity is 1. Let's dim the light a small amount
    light.intensity = 0.7;

    async function CreateModel() {
      const model = await SceneLoader.ImportMeshAsync(
        "",
        "./models/",
        "model.glb"
      );
      // console.log(model)
      model.meshes[0].position = new Vector3(710, 25, 510);

      // mesh.material = materialOne
      // hightLightColor.addMesh(mesh, Color3.White()) // pass hightLightColor to all meshes

      // const gl = new GlowLayer('glow', scene)

      model.meshes.map((mesh) => mesh.addBehavior(pointerDragBehavior2)); // add drag behavior

      scene.createDefaultCameraOrLight(true, true, true);

      const helper = scene.createDefaultEnvironment();
      helper.setMainColor(Color3.Gray());

      return model;
    }

    CreateModel();

    scene.onPointerMove = (e: any) => {
      // e.preventDefault()

      // const pickingInfo = scene.pick(scene.pointerX, scene.pointerY)
      const pickingInfo = scene.pick(
        scene.pointerX,
        scene.pointerY,
        scene.pointerZ
      );

      if (pickingInfo.hit && pickingInfo.pickedMesh.name.includes("Brep")) {
        // console.log(globalCoords)
        const payload = {
          id: pickingInfo.pickedMesh.name,
          mousePos: { globalCoords },
        };

        dispatch(babylon3dSlice.actions.clickBox(payload));
      }
    };
  };

  /**
   * Will run on every frame render.  We are spinning the box on y-axis.
   */
  const onRender = (sence) => {
    console.log(boxInfo);
    //   if (box !== undefined) {
    //     var deltaTimeInMillis = scene.getEngine().getDeltaTime();
    //     const rpm = 10;
    //     box.rotation.y += ((rpm / 60) * Math.PI * 2 * (deltaTimeInMillis / 1000));
  };

  return (
    <>
      <SceneComponent
        antialias
        onSceneReady={onSceneReady}
        onRender={onRender}
        id="my-canvas"
      />
    </>
  );
};

so my code here load a custom model.glb and within it each mesh have an unique id, and I set a onClick event when I click on the mesh, Babylon will dispatch the mesh id to redux.

I also set a boxInfo from useAppSelector, which design for handle some click event from react that I want to bring back to Babylon, as mentioned, it also give me initial state in onRender hook no matter the id keep changing.

The code was written in kind of dirty way for quick demonstrate my thought, I hope it will not affect a lot

Thanks,
Johnson

export const BabylonMain = () => {
  const dispatch = useAppDispatch();
  const { boxInfo } = useAppSelector((state) => (state) => state.babylon3d);

  const onSceneReady = (scene: any) => {
    // This creates and positions a free camera (non-mesh)
    var camera = new FreeCamera("camera1", new Vector3(0, 5, -5), scene);

    // This targets the camera to scene origin
    camera.setTarget(Vector3.Zero());

    //Prevent too fast zooming
    // camera.wheelPrecision = 50;

    const canvas = scene.getEngine().getRenderingCanvas();

    // This attaches the camera to the canvas
    // camera.upperBetaLimit = Math.PI / 2.2 // limit the z axis no less zero
    camera.attachControl(canvas, true);
    scene.clearColor = new Color4(0.109803922, 0.109803922, 0.117647059, 1);

    // This creates a light, aiming 0,1,0 - to the sky (non-mesh)
    var light = new HemisphericLight("light", new Vector3(0, 1, -1), scene);

    // Default intensity is 1. Let's dim the light a small amount
    light.intensity = 0.7;

    async function CreateModel() {
      const model = await SceneLoader.ImportMeshAsync(
        "",
        "./models/",
        "model.glb"
      );
      // console.log(model)
      model.meshes[0].position = new Vector3(710, 25, 510);

      // mesh.material = materialOne
      // hightLightColor.addMesh(mesh, Color3.White()) // pass hightLightColor to all meshes

      // const gl = new GlowLayer('glow', scene)

      model.meshes.map((mesh) => mesh.addBehavior(pointerDragBehavior2)); // add drag behavior

      scene.createDefaultCameraOrLight(true, true, true);

      const helper = scene.createDefaultEnvironment();
      helper.setMainColor(Color3.Gray());

      return model;
    }

    CreateModel();

    scene.onPointerMove = (e: any) => {
      // e.preventDefault()

      // const pickingInfo = scene.pick(scene.pointerX, scene.pointerY)
      const pickingInfo = scene.pick(
        scene.pointerX,
        scene.pointerY,
        scene.pointerZ
      );

      if (pickingInfo.hit && pickingInfo.pickedMesh.name.includes("Brep")) {
        // console.log(globalCoords)
        const payload = {
          id: pickingInfo.pickedMesh.name,
          mousePos: { globalCoords },
        };

        dispatch(babylon3dSlice.actions.clickBox(payload));
      }
    };
  };

  /**
   * Will run on every frame render.  We are spinning the box on y-axis.
   */
  const onRender = (sence) => {
    console.log(boxInfo);
    //   if (box !== undefined) {
    //     var deltaTimeInMillis = scene.getEngine().getDeltaTime();
    //     const rpm = 10;
    //     box.rotation.y += ((rpm / 60) * Math.PI * 2 * (deltaTimeInMillis / 1000));
  };

  return (
    <>
      <SceneComponent
        antialias
        onSceneReady={onSceneReady}
        onRender={onRender}
        id="my-canvas"
      />
    </>
  );
};

edit: I think you want onPointermove.add(…)

are you sure your dispatch is being called? Does it not work with the second pickInfo?
Scene | Babylon.js Documentation (babylonjs.com)

hello @brianzinn , I appreciated your reply.

I can call my dispatch function inside Babylon hook, and my redux logger showed my store did updated for what was changed in Babylon world.

Also inside my BabylonMain component, I am able to console.log the updated id, but it wont work once eI trynna to pass the boxinfo inside either onRender or onSceneReady hook, it always consold.log initalState no matter the boxinfo is updated in redux.

Thanks,
Johnson

Exactly - that’s what I had originally guessed. You have a stale ref. If you switch to a ‘useRef’ and access ‘current’ property will solve it.

If that don’t make sense - look up lexical function scope and closure.

Edit : that’s said without looking at your SceneComponent. I’m assuming it doesn’t update onRender method.

2 Likes

hello @brianzinn, thanks a lot, my problem had been solved by implementing your useRef solution, also I dived into some articles about stale references and gained a lot since I never heard of them before.

I might seek one more piece of advice from you if possible, basically after my redux can transmit between react and Babylon, the next step I will do is build draggable components in react, and I will drop the ‘mesh’ component into my “Babylon” then should generate a corresponding mesh in the dropped location inside my Babylon world.

I am wondering if there is a solution to convert the screen window X, and Y coordinates into vector locations in Babylon world, since the other similar articles I have seen in the forum, haven’t used these react Babylon combo for dragging meshes.

I have learned a lot in the forum and was able to build the first step of my project, really appreciated it!

Thanks,
Johnson

1 Like

hi @johnsonafool
I made a game once in babylon that had it’s own level editor - but I dragged only on the canvas. You can use the x/y co-ordinates to drop, but probably that will be in x/z co-ordinates. Not sure what you will do for Y (see below what a planeNormal is) - I was doing a camera pick on the camera forward ray and highlighting where it would drop. You can tap into the drag events - i’ve used react-dnd project for DOM (only DOM) drag and drop with success - hope the pointer events will pick up on the client and that is a workable solution.
Here is a drag-n-drop live demo using react-babylonjs:
Behaviors - Drag ‘n’ Drop ⋅ Storybook (brianzinn.github.io)

You can probably get some good code from the pointerDragBehavior - here is the declarative code for that, but the original code is a babylon demo (inspiration link in the code):
react-babylonjs/dragNdrop.stories.js at master · brianzinn/react-babylonjs (github.com)

1 Like