I think your issue is related to physicsBody.disablePreStep, which for performance reasons is normally true. If you want to modify the physicsBody position or rotation outside the control of the physics engine, set disablePreStep = false until the next scene.onAfterPhysicsObservable.
You can read about disablePreStep here.