It seems that the drag behaviour stops when the cursor is too far away from the gizmo itself, on ‘big scenes’.
I made a playground to demonstrate this issue, you need to start dragging and, at the same time as you move the mouse upwards, you move it sideways.
You’ll notice that at some point the movement stops (but the observable isn’t triggered), and starts again if you move back closer to the gizmo. (video at the end of the post)
According to my initial research, the problem most likely comes from the _moveDrag function, as the condition calling this function seems to be correct.
But the drag function itself relies on raycasting, which might cause problems for scenes of this size.
I do not know if it is a known issue, or if there are parameters on the gizmo or dragBehavior itselves I can tweak to make it work.