Showing highlighted silhouette of mesh ONLY when it is occluded

Welcome aboard!

What you could is drawing the object at the end (after all other objects) this way:

  • draw the object with zTest=Greater / disable z writing / use the shader with the x-ray effect. This pass will draw the parts of the object using the x-ray shader where the object is occluded by some other objects from the scene
  • draw the object normally with its normal shader. This pass will draw the parts of the object that are not occluded.

In Babylon.js, you will need to clone the object to be able to draw it two times with a different shader each time. To make sure the objects are drawn in the right order, you can set Mesh.renderingGroupId=1 for the cloned objects and Mesh.renderingGroupId=2 for the regular object. That way, all normal objects will be drawn first (because their renderingGroupId is 0 by default), then the objects with the x-ray shader will be drawn, then the objects without the x-ray shader.

Here’s an example:

I have used a node material for the x-ray shader. Feel free to use whatever you want for this shader.

Note that you will need those lines to avoid having the zbuffer cleared between the render groups:

scene.setRenderingAutoClearDepthStencil(1, false, false, false);
scene.setRenderingAutoClearDepthStencil(2, false, false, false);

Note also that you could use a simplified mesh for the x-ray pass, you don’t have to use a clone of the original mesh.

6 Likes

Hey! Welcome to the forum!

Warning redneck solution ahead! :disguised_face:

This is not as cool as @Evgeni_Popov 's solution but might give you an idea. If I can recall the correct behavior in games the highlight is displayed only when the whole mesh is hidden so the idea is to cast a ray from the camera towards the occluded object and put it’s clone to the position where the ray hits the wall actually a bit closer to the camera. The clone visibility can be set to a very small number

sphereClone.visibility = 0.000001

so it’s not viewable but the highlight is still rendered. This is not a complete solution. First of all you have to scale the clone. I am sorry, I don’t have time to do it but is as easy as getting the distance to the original sphere and to the pickedPoint and do the math.

Another idea to check whether the whole mesh is hidden is to cast rays to the 4 corners of the mesh (should be enough). If all 4 hits the same obstacle the mesh is hidden. So you can drop the occlusion functionality provided by BabylonJS and customize this to your needs.

:vulcan_salute:

2 Likes

Great solution!

You can avoid the visibility=0.00001 thing by setting disableColorWrite=true on the material of the cloned sphere. That will also be a tiny perf improvement because the cloned sphere won’t be considered as transparent and won’t be sorted with the other transparent objects.

There’s a small bug in your PG because the occlusion query is also applied on the cloned sphere as you clone after having set the occlusion query on the original sphere: you should enable the occlusion query after the cloning (or disable occlusion query on the clone).

Finally you can avoid using ray casting by simply drawing the clone at the original sphere position but without depth testing, so that the clone is always drawn even if there are obstacles. That way, you don’t have problems with distance/size of object:

[EDIT] I forgot one important thing with occlusion queries: you must make sure that all the objects that must be “occlusion query tested” are drawn last, at least after the objects that can occlude them! In the PG it works because the wall is created before the sphere and Babylon.js draws the meshes in the order they are created. In real projects you may not be able to control the creation order of meshes: in that case, you can use the renderingGroupId property and set a greater value for objects that must be occlusion query tested than for the other objects.

4 Likes

Thank you! :slight_smile:

I am still lacking knowledge of these little hacks that’s why I was proposing ( You know where :slight_smile: ) to have more docs on stuff like this.

Don’t even mention it! :hear_no_evil: It took me some time to realize this. I was tearing my hair already LOL The PG from the docs was working but my code was not because the mesh creation was in reverse order. I’ve stripped down my code, removed every extra line and still not working. I forgot to mention it in my first response though.

Thanks a lot for the pro advices! :heart_decoration: Have a great day! :muscle:

@Evgeni_Popov

However there is one thing which still bothers me. The occlusion is not really precise. You can easily end up by not seeing the mesh but still not marked as occluded. Wouldn’t casting of 4 rays from the camera to the corners of the bounding box of the mesh solve the problem and make it more precise?

I don’t think using 4 rays would be better because you could get some false positive: the 4 rays could tell the mesh is occluded but we should still draw it normally (imagine two walls separated by 1m and a character standing between the walls). You could use more rays to improve things, but you could always have some cases where it would not work.

Occlusion queries are not 100% accurate because they are using the bounding box of the mesh to know if it is occluded or not: in the PG, sometimes the mesh is not silhouetted because the bounding box is still visible whereas the mesh inside is not. It’s something that can be mitigated if the bounding box is thin compared to the width/height dimensions, which is normally the case with characters. Also, you could shrink the bounding box a little (but this time you could display the silhouette a bit too early - it’s always a trade off).

1 Like

I meant hitting the same obstacle.

ACK!

Thanks!

Well, in that case imagine that the two walls are “attached” (distance = 0m) but for some reason they are two different walls and not a single one: 2 rays will tell it’s wall A, the other two that it’s wall B and the silhouette won’t be drawn.

1 Like

image

Hey guys! Sorry to hop on my own thread so late (I was camping over the weekend) but I super appreciate the answers and discourse you two have provided and i’m definitely going to give the solution a go.

One more thing I wanted to ask your opinion on is regarding a potential solution I was given on stackoverflow where someone said that it might be possible to change the drawing layer of HighlightLayer such that it is drawn above everything. You could then just enable/disable the highlight when the object is occluded or not. Is thus even possible though? I can’t find anything in the docs that say you can do that. Seems too simple to be true :sweat_smile: would love to hear a more experienced opinion on the matter. Thanks again!

2 Likes

Hi!
Did you try renderingGroupId?
Sry for the short response. I am on phone.
:vulcan_salute:

All good. No I haven’t. I wasn’t aware you could use renderingGroupId on things that aren’t meshes

After some fiddling, seems like assigning a renderingGroupId to the highlight layer makes it invisible unless the object it is added to is on the same renderingGroupId. This won’t work for me as then the whole avatar is rendered above everything. Wonder if there is a way around this?

So I went ahead and went with the clone solution you guys came up with. The only difference is that instead of setting the position of the clone when the avatar is occluded, i’ve just made both the regular avatar and clone avatar children of the same TransformNode. That way the transform (rotation, position, scale) will both be the same for the both of them at any given time. Its working wonderfully, I wish that it would highlight the occluded part of the avatar when it is partially occluded but I can live with what I have for now. Just want to say thanks again guys! Really appreciate the help!

1 Like

I was just typing an answer to your post regarding the renderGroupId issue but your other post has just popped out suddenly so I am not going to elaborate on it anymore :slight_smile:

You’re welcome! Looking forward to see your game!

If you’re keen to elaborate on the renderGroupId issue, would be keen to hear it just out of interest/curiosity. Always keen to learn more! Might help me out on something else in the future, who knows! but if its a mission to explain then no worries :grin:

1 Like

There’s a way to do it without using a clone:

Basically, we simply add the sphere in the highlight layer when it is occluded and we remove it from the layer when it is visible.

However, doing only that won’t work as expected, you will get:
image
instead of:
image

That’s because having the right highlight requires the object to be drawn in the stencil buffer. But when it is occluded, the mesh is not displayed anymore and the stencil is not updated. So we need the object to be drawn even if the occlusion query states that it is occluded.

To do that, we can override the AbstactMesh._checkOcclusionQuery function which is the function that updates the isOccluded property and which returns that value: when true is returned, the mesh is not displayed. So, by simply calling the old function but always returning false we still update the isOccluded property correctly and the mesh is always rendered.

The last caveat is that the mesh being occluded, even if the engine displays it it will be entirely culled and the stencil won’t be updated. So we need to disable the z-testing (depthFunction=ALWAYS) to force the rendering and we also need to disable writing to the color buffer as we don’t want the sphere itself to be visible, we only want the stencil buffer to be updated (disableColorWrite=true).

2 Likes

oh nice! I’m assuming that the removal of the clone will also serve as a minor performance improvement too. Thanks Evgeni! Will give it a go.

1 Like

@Evgeni_Popov I’ve managed to successfully implement your method without using clones. It’s working nicely. I do have some questions for you though.

  1. Does the AbstractMesh._checkOcclusionQuery override only apply to meshes set to use occlusionQueryAlgorithmType = BABYLON.AbstractMesh.OCCLUSION_ALGORITHM_TYPE_ACCURATE or does this apply to all meshes?

  2. I might be paranoid but I’m concerned about the performance overhead I might get if we are overriding the checkOcclusionQuery to always render meshes even when they are occluded. I’m aiming to have plenty of avatar’s in my scenes. Perhaps you can clarify what this means exactly?

1 Like

OK! I might create a PG these days but we have @Evgeni_Popov, the mastermind here to help you and it’ll be hard to introduce something new but NGL I have an idea :smiley: