Using again your code from Version 4 brought me a ‘breakthrough’ check:
https://www.babylonjs-playground.com/#F7ZZJF#6 
(need to proof it a bit more, but could not hold it back to share)
I used the code you supplied and just changed the the pointer.X to evt.offsetX!
Will now try it in my project code where the canvas has other dimensions. I hope that will not affect the offsetX values in a way that it breaks the behavoir.
Thank you very much for bringing me here 