Layermask showing as topmost layer

Hello Babylon experts!
I’m having issues with layerMasks, most likely because of limited understanding.

I have successfully created a screenshot function by pressing “P” on the keyboard. This creates screenshots that only captures meshes that I have set as layermask = 2, and the scene.clearColor as transparent.

But it does not behave exactly as I would like it since all “layerMask = 2”-meshes are displayed on top of everything else.

Any ideas how I can solve this?

PS. I have also experimented with “setRenderingOrder” of the meshes and also “RenderTargetTexture” for the screenshots, but I’ve had less luck with those.

LayerMask/renderingGroupId testing | Babylon.js Playground (babylonjs.com)

You should indicate that it is using XR, as it probably changes how things work (I tried to hit “P” without XR enabled and could not see any screenshot generated).

cc @RaananW

Do you mean that i should indicate for the screenshot function that I’m using XR?
Because it is not important if pressing P when in VR works or not. It is perhaps better if it is disabled in VR mode allt together.
The screeshot function is only really meant to be used in desktop mode.

All right so I think I solved it by setting camera 2 first in the scene.activecameras on line 50.
And also setting the sphere layermasks to “sphere1.layerMask = camera2.layerMask | camera1.layerMask;”
And then set “scene.activeCameras = [camera2];” on line 408.
And then back to scene.activeCameras = [camera2, camera1] on function end on line 428.

LayerMask/renderingGroupId testing | Babylon.js Playground (babylonjs.com)

But I also stumbled across another issue that I cannot quite grasp. How do I give “Button 1” a layermask value of 1 (lines 262-292) so that is hidden when I take the screenshot?

I have tried “var but1mask = scene.getMeshByName(“reset1_rootMesh”)” and then but1mask.layerMask = 1;
But it doesn’t work since but1mask returns “null”…

If being in VR/XR mode does not impact the problem, I think you should remove it from the PG to reduce the possible source of problems.

GUI elements don’t have a layerMask property. You can hide button1 by doing button1.isVisible = false;

3 Likes

Thank you so much! .isVisible solved it!

Here is a working PG. WASD controls and Space to jump. “P” for screenshot with only Spheres included.
LayerMask/renderingGroupId testing | Babylon.js Playground (babylonjs.com)

Ps. Only desktop mode. VR-mode is broken because of camera2.

cc @RaananW for the VR mode.

VR doesn’t render to the canvas so it is a little more complicated to do that. It also uses 2 different cameras for each eye, which will make it even more problematic! :slight_smile:

You can turn on spectator mode on the session manager, which will then render the left eye to the canvas. Another solution (which is the preferred solution IMO) is to use the device’s screenshot capabilities to take screenshots. You can’t use it in javascript, but it is the only solution that will provide you with a simple screenshot of the entire VR scene.

1 Like

Thanks for the explanation!
The screenshot functionality is not important in VR mode. Or let’s say that I’d rather just disable the screenshot functionality all together when in VR. But I’m not sure how to achieve that since it seems like as long as the second camera is active, I’m unable to even launch the scene in VR mode in my Quest 2. I only see black…

It would be really sweet if the same scene could be made work for both desktop and VR, but without the screenshot functionality when in VR mode. Is there a simple way of disabling camera 2 only when the browser senses that a VR headset is trying to run the scene?

Oh, that happens because you are using the activeCameras array instead of setting a base active camera. That’s an interesting case that I believe we don’t take into account in XR. that is because we assume that no other cameras should be rendered when in XR (at least not active by the scene).

One thing you can do is empty the array when entering XR, and refill it when leaving. Would that work?

Sure, that would be perfect! But I’m not sure exactly how to acheive that when entering XR and refill upon exit. Are we talking “scene.activeCameras = ” to empty and then “scene.activeCameras = [camera2, camera1]” to refill?

Should it be placed inside
const xrHelper = await scene.createDefaultXRExperienceAsync({ floorMeshes: [ground, pubg] });
somehow?

The session manager has a few observables you can use:

onXRSessionInit - WebXRSessionManager | Babylon.js Documentation
and
WebXRSessionManager | Babylon.js Documentation

Would be the first i would check :slight_smile: The Base experience has a state observable that triggers when the state changed. These are the supported states:

1 Like

Super thanks again! Now it works in VR as well :slight_smile:

LayerMask/renderingGroupId testing | Babylon.js Playground (babylonjs.com)

But somehow the grabbing of the spheres got broken when replacing the camera with an xrCamera (I think…) I can not see anything about cameras in the xr.input lines, so not sure what went wrong…
Well, I’ll have to keep on investigating things tomorrow.

Once again, thank you for your help and expertise.

Edit: Solved after a couple of hours… I had leftover references to deleted meshes in the xrHelper.Input.

Now working also in VR and the spheres are grabbable again…
LayerMask/renderingGroupId testing | Babylon.js Playground (babylonjs.com)

2 Likes