Why does this Highlight Layer not silhouette the mesh? (Or How to display flash messages without breaking highlighting)

First, here is an example playground so you can see the highlight issue I’m talking about in the title question: https://playground.babylonjs.com/#N7F8QI


vs

Now for more explanation of what I’m trying to do:

I want to create a sort of heads up display for important messages. For example, in immersive XR mode, there is no way to look at console.log error messages while wearing the headset, so I put any messages I want into an advanced texture and place them on a plane in front of my camera so I can see them without taking off the headset.

This is working pretty well, except when there are other objects in front of my plane, they block the messages. I solved this by setting the plane’s renderingGroupId to a number higher than 0, which every other object is set to using. This renders my plane on top of other objects so the text is always visible.

The other problem I had was that my own message plane would get in the way of my pointing and picking on other objects in the scene. I fixed this by making my plane isPickable to false.

I discovered a new problem though, and this has to do with an entirely separate feature, which is highlighting objects. Apparently my message wall, when rendered in front of an object I want to highlight will not silhouette the object, but instead illuminate the entire object, See playground example: https://playground.babylonjs.com/#N7F8QI

which… though cool, is not what I want. I don’t want my message wall to break other features.

My question is:

  1. Can I have a brief explanation as to why this is happening and

  2. Is there a better way to do this message wall plane thing?

From reading the docs, Babylon’s 3D Gui “Once instantiated, the manager will create a utility layer which is a specific child scene that will host all the meshes used to render the controls. This way, your main scene won’t get populated by the utility meshes.” That sounds like what I want, to keep my message wall away from interacting with other things in the main scene. But I couldn’t find any examples of using 3D GUI with 2D GUI stuff (advanced Texture on a plane).

Thanks!

Update:

Instead of fussing with a plane mesh, I tried using a full-screen dynamic texture as suggested by @RaananW here https://forum.babylonjs.com/t/how-to-make-a-gui-to-follow-the-camera-in-xr-vr/12259/5?u=owen

This works pretty well on desktop and I didn’t have to do anything to make the GUI stick to the camera.

However when I put on the headset, (Oculus Quest 1) the GUI is not usable… the left eye and right eye show the GUI elements overlay way too far apart for my brain to converge the overlay into a single image.

1 Like

Adding @RaananW regarding the Oculus problem.

For your first post, it does not work because when setting renderingGroupId=1 for the plane, there’s an automatic clearing after rendering objects with renderingGroupId=0 and before rendering objects with renderingGroupId=1. As the highlight pass (which requires the stencil buffer) is coming after regular objects rendering, the stencil is no longer ok as it has been cleared.

So, for this to work you need to explicitely disable the stencil clearing by calling setRenderingAutoClearDepthStencil with the right parameters (clear depth but not stencil). You also need to exclude the plane from the highlight layer:

https://playground.babylonjs.com/#N7F8QI#1

2 Likes

Regarding XR - there seems to be an issue between multi-camera clearing when HL is enabled:

Babylon.js Playground (babylonjs.com)

(Lines 43-47 were added)

@Evgeni_Popov - any idea?

1 Like

You need to enable the stencil buffer for the RTT. Also, as you render two times the highlight layer in the same RTT, you should clear the depth/stencil between each render else you will get artifacts:

https://playground.babylonjs.com/#BCYE7J#24

2 Likes

We should probably add that as the default behavior in XR scenarios (when an HL is being used at least), as we provide no easy access to the rtt of the xr camera

2 Likes