Hi everybody, so i’m trying to create an online game using Babylon.js but have run into a problem thats got me a little stumped so hoping someone here would be willing to help me out. Please bear with me on this one, i’m a complete newbie with babylon as i’ve only every worked with THREE.js. Right now my game consists of a scene compromising of multiple meshes with multiple users represented as avatars (created from basic circle geometry for the moment) loaded into an environment. What I want to do is highlight the outline of these avatars ONLY when they are occluded by any other object, meaning that when they are not occluded they look normal with no highlight but when behind an object their highlighted silhouette can be seen by others (including yourself as you can see your own avatar). This is very akin to effects used in many other video games (see example below).
Thus far, based on some googling and forum browsing (Babylonjs outline through walls - Stack Overflow & Highlight through objects - #4 by Aymeric_LS) I’ve figured out how to highlight the outline of objects using Babylon.HighlighLayer and I know that i can render objects above others via RenderingGroups but I can’t seem to figure out how to use them in conjunction to create the effect I want. The best i’ve managed to do is get the highlighted avatar render above everything but I need just the silhouette not the entire mesh. I’m also constrained by the fact that my scene has many meshes in it that are loaded dynamically and i’m also trying to keep things as optimal as possible. Can’t afford to use very computationally expensive procedures.
Anybody know of the best way to approach this? Would greatly appreciate any advice or assistance you can provide.Thanks!
What you could is drawing the object at the end (after all other objects) this way:
draw the object with zTest=Greater / disable z writing / use the shader with the x-ray effect. This pass will draw the parts of the object using the x-ray shader where the object is occluded by some other objects from the scene
draw the object normally with its normal shader. This pass will draw the parts of the object that are not occluded.
In Babylon.js, you will need to clone the object to be able to draw it two times with a different shader each time. To make sure the objects are drawn in the right order, you can set Mesh.renderingGroupId=1 for the cloned objects and Mesh.renderingGroupId=2 for the regular object. That way, all normal objects will be drawn first (because their renderingGroupId is 0 by default), then the objects with the x-ray shader will be drawn, then the objects without the x-ray shader.
This is not as cool as @Evgeni_Popov 's solution but might give you an idea. If I can recall the correct behavior in games the highlight is displayed only when the whole mesh is hidden so the idea is to cast a ray from the camera towards the occluded object and put it’s clone to the position where the ray hits the wall actually a bit closer to the camera. The clone visibility can be set to a very small number
sphereClone.visibility = 0.000001
so it’s not viewable but the highlight is still rendered. This is not a complete solution. First of all you have to scale the clone. I am sorry, I don’t have time to do it but is as easy as getting the distance to the original sphere and to the pickedPoint and do the math.
Another idea to check whether the whole mesh is hidden is to cast rays to the 4 corners of the mesh (should be enough). If all 4 hits the same obstacle the mesh is hidden. So you can drop the occlusion functionality provided by BabylonJS and customize this to your needs.
You can avoid the visibility=0.00001 thing by setting disableColorWrite=true on the material of the cloned sphere. That will also be a tiny perf improvement because the cloned sphere won’t be considered as transparent and won’t be sorted with the other transparent objects.
There’s a small bug in your PG because the occlusion query is also applied on the cloned sphere as you clone after having set the occlusion query on the original sphere: you should enable the occlusion query after the cloning (or disable occlusion query on the clone).
Finally you can avoid using ray casting by simply drawing the clone at the original sphere position but without depth testing, so that the clone is always drawn even if there are obstacles. That way, you don’t have problems with distance/size of object:
[EDIT] I forgot one important thing with occlusion queries: you must make sure that all the objects that must be “occlusion query tested” are drawn last, at least after the objects that can occlude them! In the PG it works because the wall is created before the sphere and Babylon.js draws the meshes in the order they are created. In real projects you may not be able to control the creation order of meshes: in that case, you can use the renderingGroupId property and set a greater value for objects that must be occlusion query tested than for the other objects.
I am still lacking knowledge of these little hacks that’s why I was proposing ( You know where ) to have more docs on stuff like this.
Don’t even mention it! It took me some time to realize this. I was tearing my hair already LOL The PG from the docs was working but my code was not because the mesh creation was in reverse order. I’ve stripped down my code, removed every extra line and still not working. I forgot to mention it in my first response though.
Thanks a lot for the pro advices! Have a great day!
However there is one thing which still bothers me. The occlusion is not really precise. You can easily end up by not seeing the mesh but still not marked as occluded. Wouldn’t casting of 4 rays from the camera to the corners of the bounding box of the mesh solve the problem and make it more precise?
I don’t think using 4 rays would be better because you could get some false positive: the 4 rays could tell the mesh is occluded but we should still draw it normally (imagine two walls separated by 1m and a character standing between the walls). You could use more rays to improve things, but you could always have some cases where it would not work.
Occlusion queries are not 100% accurate because they are using the bounding box of the mesh to know if it is occluded or not: in the PG, sometimes the mesh is not silhouetted because the bounding box is still visible whereas the mesh inside is not. It’s something that can be mitigated if the bounding box is thin compared to the width/height dimensions, which is normally the case with characters. Also, you could shrink the bounding box a little (but this time you could display the silhouette a bit too early - it’s always a trade off).
Well, in that case imagine that the two walls are “attached” (distance = 0m) but for some reason they are two different walls and not a single one: 2 rays will tell it’s wall A, the other two that it’s wall B and the silhouette won’t be drawn.
Hey guys! Sorry to hop on my own thread so late (I was camping over the weekend) but I super appreciate the answers and discourse you two have provided and i’m definitely going to give the solution a go.
One more thing I wanted to ask your opinion on is regarding a potential solution I was given on stackoverflow where someone said that it might be possible to change the drawing layer of HighlightLayer such that it is drawn above everything. You could then just enable/disable the highlight when the object is occluded or not. Is thus even possible though? I can’t find anything in the docs that say you can do that. Seems too simple to be true would love to hear a more experienced opinion on the matter. Thanks again!
After some fiddling, seems like assigning a renderingGroupId to the highlight layer makes it invisible unless the object it is added to is on the same renderingGroupId. This won’t work for me as then the whole avatar is rendered above everything. Wonder if there is a way around this?
So I went ahead and went with the clone solution you guys came up with. The only difference is that instead of setting the position of the clone when the avatar is occluded, i’ve just made both the regular avatar and clone avatar children of the same TransformNode. That way the transform (rotation, position, scale) will both be the same for the both of them at any given time. Its working wonderfully, I wish that it would highlight the occluded part of the avatar when it is partially occluded but I can live with what I have for now. Just want to say thanks again guys! Really appreciate the help!
I was just typing an answer to your post regarding the renderGroupId issue but your other post has just popped out suddenly so I am not going to elaborate on it anymore
If you’re keen to elaborate on the renderGroupId issue, would be keen to hear it just out of interest/curiosity. Always keen to learn more! Might help me out on something else in the future, who knows! but if its a mission to explain then no worries
Basically, we simply add the sphere in the highlight layer when it is occluded and we remove it from the layer when it is visible.
However, doing only that won’t work as expected, you will get:
instead of:
That’s because having the right highlight requires the object to be drawn in the stencil buffer. But when it is occluded, the mesh is not displayed anymore and the stencil is not updated. So we need the object to be drawn even if the occlusion query states that it is occluded.
To do that, we can override the AbstactMesh._checkOcclusionQuery function which is the function that updates the isOccluded property and which returns that value: when true is returned, the mesh is not displayed. So, by simply calling the old function but always returning false we still update the isOccluded property correctly and the mesh is always rendered.
The last caveat is that the mesh being occluded, even if the engine displays it it will be entirely culled and the stencil won’t be updated. So we need to disable the z-testing (depthFunction=ALWAYS) to force the rendering and we also need to disable writing to the color buffer as we don’t want the sphere itself to be visible, we only want the stencil buffer to be updated (disableColorWrite=true).
@Evgeni_Popov I’ve managed to successfully implement your method without using clones. It’s working nicely. I do have some questions for you though.
Does the AbstractMesh._checkOcclusionQuery override only apply to meshes set to use occlusionQueryAlgorithmType = BABYLON.AbstractMesh.OCCLUSION_ALGORITHM_TYPE_ACCURATE or does this apply to all meshes?
I might be paranoid but I’m concerned about the performance overhead I might get if we are overriding the checkOcclusionQuery to always render meshes even when they are occluded. I’m aiming to have plenty of avatar’s in my scenes. Perhaps you can clarify what this means exactly?
It took me some time to realize this. I was tearing my hair already LOL The PG from the docs was working but my code was not because the mesh creation was in reverse order. I’ve stripped down my code, removed every extra line and still not working. I forgot to mention it in my first response though.
Have a great day! 
would love to hear a more experienced opinion on the matter. Thanks again!
