I think I’m gonna need a hand/explanation for the handling of the 3D GUI with a multi-camera approach.
My scene features a BJS 2D GUI & 3D GUI. I am just introducing post-processing and in order not to apply my rendering pipelines effects to my GUI, I have been using LayerMask (set to advancedTexture.layer.layerMask = 0x10000000 and coupled with a BG camera with same layer mask.
All works fine but then I just realized that my 3D GUI controls are not accessible anymore. Thought I would simply set the manager or utilityLayer to the same layerMask, but no, this didn’t work.
Only when I set the layerMask to a value of 2, can I get the advanced texture (2D GUI) and 3D GUI controls back in the scene but it creates problems with the other meshes.
I did set up a quick and ugly PG to showcase this.
Could you explain what does not work in your PG / what you expect? What are the 3 cameras used for? If I keep only camera the PG still renders in the same way.
If I uncomment line 37 and comment line 38 the 3D buttons are still working.
Press < C > to change camera. Side-note: Although here only one camera could suffice by just changing the target, I had imagined I could have a second or more different type of camera. This should not impact the overall problem. The problem remains with just one (scene) camera.
It has indeed something to do with the handling of the cameras and layerMask.
Here are my findings:
When I use ‘scene.activeCameras’, the last camera in array is actually considered the ‘scene.activeCamera’. The one I can act on when I call ‘activeCamera’.
There must be more to the layerMask then just the values of 0,1,2 (as just quickly stated in the doc).
I get different results and I suppose I’d like to get a better understanding of how this works. And how this works with RenderingGroupID and post-process…
The goal:
Have my GUI (advanced texture) and 3D GUI visible and with active controls. This part I already had.
Have my post effect (or rendering pipeline) on my scene but NOT on my 2D GUI (advanced texture)
++ Still have control of my camera (active or scene camera - not the GUI camera)
I thought it would be easy and everyone would want to do that… though I’m struggling…
Thanks anyone who can give me a light on this…
I think what you are missing is that layerMask also applies to meshes. By default, meshes have a layerMask = 0x0FFFFFFF so will be seen by your cameras when you set their layerMask to 2 (because 0x0FFFFFFF & 0x10 != 0).
Setting 0x10000000 explains why your 3D GUI meshes disappear, as the AND operation between 0x10000000 and 0x0FFFFFFF is 0.
To make the meshes from the utility layer scene visible, you should do something like this after you created the 3D buttons:
Thank you for this explanation. I will try this right away but it already makes sense to me.
Just out of curiosity can you also explain what the values of ‘…FFFFFF’ or ‘000000’ (the second part after 0x0 or 0x1) is here for? Does this have something to do with alpha, alpha combine for underlying layers?
Also, I was not sure I could manage ‘layerMask’ through the utility layer. I was attempting to apply the instruction to the manager or utilityLayer itself but figured this couldn’t be right because I did not find this in the API for utilityLayer. Applying it to the meshes attached to the utilityLayer indeed makes much more sense. Sometimes, I feel I’m just so stupid…
I’ll let you know about the outcome.
Thanks a lot.
It is simply the whole layerMask value displayed in hexadecimal for better visualizing. The only two things to understand with layerMask are that:
it can be any number from 0 to 0xFFFFFFFF (or 4294967295, but it’s easier to write 0xFFFFFFFF)
when computing a visibility flag, two values of layerMask are ANDed and if the result is different from 0 the flag is true
So, for eg, if camera.layerMask = 35 and mesh.layerMask = 2, 35 & 2 == 2 != 0 so the mesh is visible from this camera. We often show these numbers in hexadecimal because doing the AND is easier that way when doing a mental calculation: 35=0x23 and 2=0x02 and 35 & 2 == 0x23 & 0x02 and from this latter representation it’s easier to see that the result is 2 (well, at least when you are a bit comfortable with hexadecimal).
In Babylon, by default the layerMask properties are 0x0FFFFFFF (for Camera, Mesh, Layer, …). So, when it comes to determine a visibility (between a camera and a mesh for eg), 0x0FFFFFFF & 0x0FFFFFFF != 0 and everything is always visible (at least from the standpoint of the layerMask).
As you can see, there are 4 bits not set by default in the layerMask (the leading 0 before the first F). It’s to ease the use of layerMask, as a common usage is to make some meshes hidden from a camera. To achieve that, you need to set a layerMask on the meshes so that the AND operation with the layerMask of the camera (0x0FFFFFFF by default) is 0: you can use 0x10000000, 0x20000000, 0x40000000 or 0x80000000. Now you see why showing these numbers in hexadecimal help a lot: it’s a lot easier to see that 0x10000000 & 0x0FFFFFFF = 0 than 268435456 & 268435455!
If the default value for layerMask was 0xFFFFFFFF, you would also need to update the layerMask of your camera to have your meshes invisible from it because 0xFFFFFFFF is a all-one 32 bit value. So the 0x0FFFFFFF value is simply a default value that can help make your life a little easier, but in all generality you can put any value in the layerMask property of cameras, meshes, layers, etc (as in the 35 & 2 example above) and simply understand how it is used to compute visibility.
Awesome, simply awesome.
Thanks so much for taking the time (your valuable time) to write all this.
I will read it through a second time but this is really VERY GOOD support.
I luv it. Have an awesome day
Yup, it sure does… and I was expecting so from BJS… AND I have not been deceived
Not sure I’d appreciated having to manage the layers for all meshes in my scene, with all effect, particles and alpha blends and combine (in hexadecimal or ‘any decimals’
I am back with the subject. Thought I got an understanding of this (and I did but just for a part of it, obviously). I’m still struggling with I believe the handling of my camera together with LayerMask and rendering pipeline. I simply cannot get the result I need. When I activate my rendering pipeline on activeCamera, the post process still applies to the advancedTexture AND my 3D controls (that are now back thanks to your input:) also render on top of my meshes, which is not something I want. The camera not using the rendering pipeline does render my 3D control together with my scene meshes correctly. This is while both camera (camera and camera1) are set the same. The only difference is I apply the (lenseffect) rendering pipeline on camera (or camera1). This does the same no matter if I set a layerMask to the camera (and/or camera1). It seems like activating the rendering pipeline on the camera changes something to the (default) layerMask?!
You can still change camera with the < C > key press in this updated PG example:
Would it be too much to ask you to actually demonstrate how this works, based on my PG example? (or any similar example)
The scene and 3D controls on a same layer/level (not the 3D controls on top of the meshes)
The post effect on the scene (and eventually 3D GUI) BUT NOT on the advancedTexture.
I feel I’m just moving around the topic, eventually getting close to it but never really correct. This is making me feel really stupid. I think I have just been fiddling with the case for too long. I cannot get a clear vision of this anymore…
Again thanks for your awesome support,
When you create several cameras, the first created one is the active camera and the scene will only render with this camera if you don’t setup explicity scene.activeCameras with all of your cameras. So I think the first thing you need to do is populate this property so that the scene displays correctly when run (meaning, before we hit the “C” key).
If you want to render the GUI above the regular scene, you should:
add the bgcamera (which is the GUI camera) after the regular camera in the scene.activeCameras property
set the layerMask property to the same value for the bgcamera and for the advanced texture
One last thing to know: the 3D GUI manager (in fact the utility layer inside the manager) will by default render itself after the last camera of the scene. It’s not what you want, as you need the 3D buttons to be rendered before the 2D GUI (so the bgcamera). You can use the setRenderCamera method of the utility layer to instruct the layer to render after the given camera.
You have teaching skills my friend, clearly. May be you are a teacher?
Turns out it was exactly what I ‘sensed’ (but couldn’t resolve;)
In fact, I had it all right in the first place, except for my 3D controls and next started mess it all up simply because I wasn’t really knowing what I was doing.
This is exactly the last part I was missing (of 3 in total). But what I like even more than the solution is the explanation of 1) LayerMask 2) Camera 3) GUI and post-process… the last one for the 3D GUI, I believe is mostly a ‘personal’ tweak of mine;) But I bet many users (and especially newcomers) would like to have these explanation (in the doc) and eventually also a PG.
May be you have noticed that the PG featured in the 4.2 official (with the lovely, cute, i luv it, little pirate fort) does not have the GUI button on top of PP? I found it would have been nice to make 1) a nicer button 2) display the button on top… don’t you think? Same as I did not find a PG example featured in the doc for this (to which you could have pointed me directly instead of doing all this (again thanks so much).
So, as a conclusion, is there or is there not an example and mention of the above (short version) in the doc? And if no, would you say that parts of the above would be worth a mention. If so, I would also be open to take some time to provide with (the base) of a nicer and more comprehensive PG…
Again thanks so much for your time and all the above.
Have an awesome day,
French would be perfectly fine with me
Sure, 'will certainly try set up an example ‘in the spirit of’ and using ‘the base of’ the other ‘basic examples’. I should be able to do that.
May be next you could quickly review and in case, amend it…
I finally did my work this morning which also helped me realize that I’m just a rookie here I’m not really confident yet, even with this simple example. So, may I ask you to review/amend it?
I based it on the simple examples from the doc/PG for gui3d GUI layermask and #Post-process. I guess we could add more (like your example of detaching/attaching controls when switching camera or how to handle other rendering pipelines (like SSAO and a skybox)… but I have the feeling that in the spirit of the PG examples, it should remain simple. What do you think?
I think you can remove the alert, however, I’m not sure we need them, we can easily see the changes when we click the “switch camera” / “change post fx” buttons.
Also, as you now have the “switch camera” button, you can remove the onKeyboard observable (will make the PG shorter).
!).
which also helped me realize that I’m just a rookie here 
I’m not really confident yet, even with this simple example. So, may I ask you to review/amend it?