Mesh intersection by using stencil buffer

Hi all,

I need help to do mesh intersection using engine stencil. Currently, I can make a sphere intersect with the box and create a recessed hole. When I try to add more spheres to make the same hole like that for the other side, I still see the sphere inside of the box and I don’t want that. How can I make a similar hole like that for other sides without seeing the sphere inside of the box?

Here is my playground: Babylon.js Playground

Thank all

The problem is that the sphere intersect of the other side of the cube is still drawn in the stencil buffer because the depth test is “greater or equal”, screwing up the stencil buffer.

One way to do it is to create 6 planes and handle the 6 planes with different values of the stencil buffer, so that handling a (plane + sphere intersect + sphere) of one side does not temper with any other sides.

In the PG below I have set the stencil values to 3/5/7/9/11/13 for the 6 planes because I have set the “stencil operation pass” function to “DECR”, meaning a value of 2/4/6/8/10/12 will be written to the stencil buffer where the “sphere intersect” is drawn. The important thing is that those values are different from the plane values, so it’s one way to do it. I had first tried to set the stencil mask to “1” when writing the “sphere intersect” objects, in the hope to get 1 into the stencil buffer (because 3 & 1=1, 5 & 1=1, etc) but it did not work for some reasons…

Last thing, I used hemispheres instead of full spheres as you don’t really need full spheres here (it saves some polys).

…and welcome aboard!


Forgot to say you could get a similar result with CSG, I don’t know if you considered this possibility?

Thank you so much for your help! My Playground was just a simplify idea of making intersect for my house building tool. There are some recessed windows need to be done by this way. I also considered using CSG but I think it’s not good for performance and I have to deal with new mesh creation from CSG.

Also, in your PG, can it be done without setting the material with depth function for the sphere?

Yes, but then you need to do the same thing in onBeforeRenderObservable / onAfterRenderObservable (like you did in your first PG). I thought it was easier to set the corresponding properties (depthFunction and colorWrite) in a material instead of changing the global state of the engine in those methods.

1 Like

Since my meshes are not created by MeshBuilder. So, Can I set corresponding properties (depthFunction and colorWrite) for materials of imported meshes?

Yes, you can update the materials of imported meshes.

Can you help me out with using onBeforeRenderObservable / onAfterRenderObservable instead of using material? I am trying to understanding both ways. Sorry that I am a newbie in Babylonjs and I am confused with many stencil functions. Thank you so much. I really appreciate your help!

Never mind. I was able to make it work. Thank you so much!

1 Like