React+Colyseus+Banylon camera shake problem,How can I prevent camera shake? Or solve the problem that the camera cannot be rotated during player movement?

I encountered a camera shake issue when using React to develop Babylon and Colyseus multiplayer online games, the effect is as follows:
player4
It can be clearly seen that the camera is shaking, and on other player screens, my character is moving smoothly.
code show as below:

  const playes : GameObject[] = [];
  async function loadGameData(scene: BABYLON.Scene,camera1: BABYLON.ArcRotateCamera) {
        const assetManager = new BABYLON.AssetsManager(scene);
         playerTypes.forEach(type => {
            playes.push(new GameObject("players", type.asset, assetManager));
        })
      //ground  skybox 
       ......
      await assetManager.loadAsync();
    };



   function connect(scene: BABYLON.Scene, camera1: BABYLON.ArcRotateCamera) {
        const client = new Colyseus.Client('ws://localhost:2567');       
        client.joinOrCreate("my_room").then(room => {
        let playerEntities: any = {};
        //Defines the player's position, used when the interpolation loop renders   
        let playerNextPosition: { [key: string]: BABYLON.Vector3 } = {};

        state.players.onAdd=(player: Player, sessionId: string) => {
            if (sessionId) {  
              let transformNode  = new BABYLON.TransformNode(sessionId, scene);
              transformNode.metadata = sessionId;
              let playerMesh: BABYLON.TransformNode = playes[0].assetContainer.instantiateModelsToScene().rootNodes[0];
              playerMesh.setParent(transformNode);
              //Here, the position generated by each player is set, and the position coordinates are generated by the backend
              playerMesh.position.set(player.x, player.y, player.z);               
              playerEntities[sessionId] = playerMesh
                  
            playerNextPosition[sessionId] = playerMesh.position.clone();
            player.onChange = (changes) => {            
                console.log('player.id:',player.id,'   room.sessionId',room.sessionId);          
                //Reset each player position, do not reset your own, because your own displacement is performed separately in keyboard events
                if (player.id != room.sessionId) { 
                    playerNextPosition[player.id].set(player.x,player.y, player.z);
                }
                if (player.id === room.sessionId) {
                    //The camera follows the player, the problem is here, setting the camera to follow here will cause the camera to shake
                    camera1.target = new BABYLON.Vector3(player.x,player.y, player.z)
                }           
            };
            
            //Interpolated loop rendering
            scene.registerBeforeRender(() => {
                for (let sessionId in playerEntities) {
                    if (sessionId != room.sessionId) {
                        var entity = playerEntities[sessionId];
                        var targetPosition = playerNextPosition[sessionId];
                             entity.position = BABYLON.Vector3.Lerp(entity.position, targetPosition, 0.05);
                     }
                }
            });   
         }
        };
        
        //  Keyboard events
        var inputMap:any = {};
        scene.actionManager = new BABYLON.ActionManager(scene);
        scene.actionManager.registerAction(new BABYLON.ExecuteCodeAction(BABYLON.ActionManager.OnKeyDownTrigger, function (evt) {
            inputMap[evt.sourceEvent.key] = evt.sourceEvent.type === "keydown";
        }));
        scene.actionManager.registerAction(new BABYLON.ExecuteCodeAction(BABYLON.ActionManager.OnKeyUpTrigger, function (evt) {
            inputMap[evt.sourceEvent.key] = evt.sourceEvent.type === "keydown";
        }));
        
        //Hero character variables 
        var heroSpeed = 0.1; 
        var heroSpeedBackwards = 0.01; 
        var heroRotationSpeed = 0.1;
        scene.onBeforeRenderObservable.add(() => {
          if (inputMap["w"]) {
             playerEntities[room.sessionId].moveWithCollisions(playerEntities[room.sessionId]
                 .forward.scaleInPlace(heroSpeed));
        }
        if (inputMap["s"]) {
            playerEntities[room.sessionId].moveWithCollisions(playerEntities[room.sessionId]
                 .forward.scaleInPlace(-heroSpeedBackwards));
        }
        if (inputMap["a"]) {
            playerEntities[room.sessionId].rotate(BABYLON.Vector3.Up(), heroRotationSpeed);
        }
        if (inputMap["d"]) {
            playerEntities[room.sessionId].rotate(BABYLON.Vector3.Up(), -heroRotationSpeed);
        }
   
            if (playerEntities[room.sessionId]) { 
                let position = playerEntities[room.sessionId].position
                room.send("updatePosition", {
                    x: position.x,
                    y: position.y,
                    z: position.z,
                });
                //If the code for camera following is written here, the camera shake problem will disappear, but a new problem will arise: I can't rotate the camera's angle when the player moves
                // camera1.target = new BABYLON.Vector3(position.x,position.y,position.z)
            } 
        });
    }).catch(e => { });
}


 const onSceneReady = (scene: BABYLON.Scene) => {
        var camera1 = new BABYLON.ArcRotateCamera("camera1", Math.PI / 2, Math.PI / 4, 10, new BABYLON.Vector3(0, 0, 0), scene);
      //other code  .
      .....
        loadGameData(scene,camera1).then(() => connect(scene,camera1));
    };

When I put the settings for the camera to follow at the very bottom of the code, the camera no longer shakes, but I can’t rotate the camera when the player moves, the effect is as follows:
player7

The above image moves very smoothly, but the camera cannot be rotated no matter how the mouse is dragged.
How can I solve this problem of camera shake or camera not rotating?

Hmmm I don’t have experience with Colyseus but I’d guess the problem is the frequency of updates you’re receiving. You’ll probably need to interpolate the camera targets, possibly even adding some predictive networking to ensure a smoother experience. Hopefully someone more experienced with networking can jump in to correct anything I get wrong :sweat_smile:

1 Like

Hi @yjkhaoyun you could try this package instead GitHub - virbela/buffered-interpolation: A class for interpolation of position, rotation, and scale for networked BabylonJS objects. to handle the interpolation of different networked objects. It’s gotten me very smooth results

2 Likes