Ok so doing more research and things get crazy.
I found this post; How to reapply physics after dragging mesh? and in it they do have drag behaviour working BUT they are using ammoJS and imposters. In my Havok example, I am using PhysicsBody and Shape…
See it kinda working with Imposters (but not perfect)