While looking at this post PointerDragBehavior keeps resseting my mesh position I found a few issues the one I haven’t solved occurs in this PG Babylon.js Playground
Mousedown and move on sphere, sphere jumps to y = 3 on mousedown but jumps back to y = 1 on drag
Run PG again
Click on sphere. After release do mousedown and move, sphere stays at y = 3.
Not what I would expect but perhaps intended behavior??