It appears using billboard mode on meshes in VR don’t really work. The issue is as you move your head around the billboard rotates while still in the frustum giving unwanted movement. Reading the docs lookAt seem to be a decent alternative.
const camPos = this.vrHelper.webVRCamera.devicePosition
.clone()
.add(
new Vector3(
0,
this.vrHelper.webVRCamera.deviceDistanceToRoomGround(),
0
)
)
m.lookAt(camPos);
However without the addition to the Y axis its pointing at my feet.
- Why do I need the offset for the lookAt?
- When the headset is on the ground it says 0.55 for this.vrHelper.webVRCamera.deviceDistanceToRoomGround() so this seems odd even though the content is correctly place on ground.
- Could there be a billboard mode for VR where it updates to use the vr camera’s position instead of rotation? Other wise you have to do a lot of onBeforeRenderObservable to emulate the correct behavior.
well technically billboard SHOULD work in VR. Can you repro in the PG with a device orientation camera?
I’ll make sure to look at it next week for sure
Oh, it VR billboard does “work”, but not like you’d expect. I’ll get a PG example up but in the mean time…
Imagine a box(4,0.1,0.1) and you use billboard_Y. When you are in VR you move your head around and when you are not looking at the box its constantly rotating still in your view.
oh so you want some kind of billboard only when you look at it?
https://www.babylonjs-playground.com/#TAFSN0#242 hopefully shows the behavior. In “normal” mode it follows the camera as expected; in VR it “swings” around.
Pinging @sebavan to see if he can have a look
I can not see or understand the issue, the cube is always well locked on the camera position which is what the billboard should do ???
Could you describe a bit more what you are trying to achieve here ?
Turn on VR mode in my example, you’ll see the box swinging around we you are stationary but moving your head/cam. You can emulate a proper billboard using look at but means a lot more event hookups. It just seems the cur behavior in VR explicitly it’s not really helpful. In normal operation outside of VR billboard works great.
Specifically by looking at the camera position instead of copying the cameras view matrix do you get a behavior that makes more sense in VR
I am not observing this in device orientation mode on desktop 
Maybe @trevordev could double check ? or somebody with a device ?
Hopefully we are not talking past each other. Here is a video of the behavior. Especially when zoomed out you can see the issue. If that plane was text it would be even more jarring in VR. From testing in VR billboard mode should be doing a lookat to the camera’s position and not the cameras orientation. Otheriwise as you move your head the text/billboards are constantly moving and only really visible when you are looking directly at them.
simplescreenrecorder-2019-07-30_06.08.53.mp4.zip (1.2 MB)
From the video it looks expected, the cube always point toward the camera position. It does not rotate when the camera rotates The main issue I guess you are highliting is that it is not anymore perpendicular to the camera view direction ???
It is actually the same exact behavior not in VR if you use a free camera. The orientation does not impact the billboard orientation. Only the camera position does.
(just trying to better understand the wished behaviour)
I don’t understand what you’re saying unfortunately. Without changing the position at all and rotating the camera, the box is rotating. It doesn’t seem to be related to position at all but to the orientation of the camera. It means at least in VR when you turn your head the billboards are constantly moving instead of just being aimed towards your position. Also the farther you tilt your head from the box being in the center of the screen the more the box appears to rotate. The farther to the edges of your peripheral the less visible the billboard is, I’m sure that’s not the intended behavior. Also of note if you turn your head to the left the box turns the opposite way which never makes sense with a billboard, right?
Are you watching the whole video @sebavan? I show it working in normal camera operation but then show it rotating when you rotate in VR mode. The camera is rotating but not translating and the box is constantly rotating to match the cameras orientation.
Let me look at the code to better understand what it does I ll come back to you ASAP.
Ok so I was totally wrong 
The billboard tries to always align with the camera direction… Thought of a lookat for some reason… do not ask me why I was thinking this… sorry again for the confusion.
It is not a VR related issue as you can see here: Babylon.js Playground
Is it highlighting your issue ? and is what you want simply the cube facing the position only rotating on Y ?
Thanks for confirming this.
Yes your example shows it 100%. I too expected the billboard mode to be base on position of camera, not orientation. By using look at I get close to what I want but its ignoring the built-in billboard support and adds more events that should be unnecessary. There is still the issue of the the lookAt(this.vrHelper.webVRCamera.devicePosition) pointing at the feet but lets treat that as a separate issue for now.
Actually we can not change the behavior of the billboard as it is widely use for other scenarii but we will definitely help creating a small piece of code you could easily reuse
I might do it a lot later today or somebody from the community might beat me to it, thanks for your patience
I didn’t want to change the current behavior so much as add an extra OR to the billboard mode like USE_POSITION=128 that would modify the billboard calculation to use position in lieu of the view matrix. It would be easy to check for with if(billboardMode & USE_POSITION) and not be a breaking change elsewhere.
Would you fancy doing a PR for it ??? this would be awesome
Sure, I forked BabylonJS but it appears all the playground/examples use the CDN/Preview and not the local codebase. Is there a good “here’s the setup for developers” guide around @sebavan?