Showing a mesh always-on-top in WebXR

Questions
#1

Heya,

I’m currently working on a feature that toggles whether a mesh should always be shown or not. For the desktop implementation, I’m using a second camera with a layerMask, and changing the mesh’s layerMask to that one if it needs to be on top. This is working exactly how I want it to.

Now, I also need to get this working when in WebXR mode and I’ve found this to be a bit of a challenge. The methods I’ve tried are…

  • Copying the camera from xrHelper.baseExperience.camera and setting the layerMask there. That didn’t seem to do much.
  • Manually creating a different StereoscopicUniversalCamera or WebXRCamera, and copying the position/rotation from the main XR Camera. This seemed to do something, but I wasn’t able to actually get it to use the layerMask. I had a feeling that the value was ignored.
  • Create a TargetCamera for each eye, setting the viewport((0, 0, 0.5, 1) for left, (0.5, 0, 0.5, 1) for right)), set the right layerMask, and copying the position / rotation. To my surprise, this actually seemed to do the trick, but it feels a tad hacky and I’d love to hear if there’s a better way to approach it.

I’ve created a playground to demonstrate my current implementation for the desktop which is available here: https://playground.babylonjs.com/#GRQTVW#3. Double-click on the canvas to toggle the alwaysOnTop feature on the box behind the sphere.

If anyone could give me any pointers or tips on how to achieve this in WebXR, it would be appreciated.

#2

Adding @RaananW

#3

Hi,

layer masks (or multiple cameras) are not supported in XR, due to the way WebXR is working. You can instead use the rendering group ID parameter, existing on every mesh:

alwaysOnTop | Babylon.js Playground (babylonjs.com)

Also, WebXR has no support for the dblclick event, and no support of pointer events on the canvas, so the event would never trigger in WebXR (unless you use a mouse while using the emulator). This PR is a clean version, also changing the way the other cameras and meshes are created. I hope this helps!

3 Likes
#4

I just realised I never came back here to thank you for your answer, @RaananW. Your example is exactly what I needed to know on how to get this properly running. Thank you so much :pray:

1 Like
#5

Apologies for bringing up an old thread, but it’s related to this topic, so I thought that might be preferred over starting a new one.

So I’m expanding a bit on the implementation I’ve done earlier, and now need to add some more interaction. Unfortunately, as I had slightly feared, being able to click on a mesh will still have it click on a mesh that is front of it, as demonstrated here: Babylon.js Playground

What I’m currently trying to do is add an option that would let me interact with a mesh that’s on top, even if there is a mesh between it and the pointer. I’ve looked into a number of different approaches here…

#1. The general example I’ve seen is having separate scenes for this. Unfortunately, for my use case, I need to be able to toggle whether a mesh should always be on top, and as far as I could tell, there is no easy way to transfer an object between scenes.
#2. Use the pointerDownPredicate, pointerMovePredicate and pointerUpPredicate functions to determine whether it should be able to interact with a mesh. From what I could tell, I was only able to tell it that a mesh in front wouldn’t be clickable, but it wouldn’t let it click on something behind the mesh.
#3. Temporarily set .isPickable = false on all meshes, aside from the always-on-top meshes. This one seemed to work kinda and is the closest I’ve come yet. It feels a bit like overkill to take this approach, and there were still be blind spots where I wouldn’t be able to interact with the mesh. Really small blind spots where you’d have to know they were there, but still there. So far, the best result I’ve gotten, but I am curious if there is a better way.

Any possible feedback or ideas on how I could approach this?

1 Like