I think @Cedric recently added an option to avoid this issue:
BABYLON.Gizmo.PreserveScaling = true;
Related: Gizmo use tmpParent but do not preserve scaling sign - Bugs - Babylon.js (babylonjs.com)
1 Like