So you are trying to achieve an HUD-Style GUI that will follow the camera wherever you go.
You could use a full-screen UI for that. This is our default GUI example with XR support and a bit of changes so that you will see it correctly in XR:
https://playground.babylonjs.com/#J8AGW2
This can get you started with building HUD-style GUI.
It will work better once this is merged - [XR] Babylon calculates the view matrix of the main camera by RaananW · Pull Request #8487 · BabylonJS/Babylon.js · GitHub , but the fullscreen GUI does work correctly