Agreed, this should be possible. A potential blocker is mentioned in computeWorldMatrix gives delayed positions when using Physics in onBeforeStepObservable.
If this is resolved, we can have deterministic physics with accurate position tracking, so you could send correct positions between server and client.