Please check this PG: https://www.babylonjs-playground.com/index.html#UTINNB#16
Lines 48-50: PointerDragBehavior is created to work in the XZ plane.
One lines 138 and 138 the behavior is attached and detached to the clicked/released mesh respectively.
What happens is, that the SECOND (and only the second) time you drag a mesh, the XY plane is used. After the second time it works as expected.
Plus, on the SECOND time, when the object is selected it kind of jumps close to the position where the first drag action has ended.