Exclude transparency when writing to stencil buffer

Questions
1

Hello, I would like to produce an effect where a mesh masks another mesh. See this image to understand clearly:

stencil
stencil1418×1072 188 KB
.

For external reasons, it needs to be done with 2 meshes, not one mesh with 2 textures.

I feel this can definitely be done using the stencil buffer.

I am getting very close by following [SOLVED] How to display an object only on a different object - Questions & Answers - HTML5 Game Devs Forum .

My issue is that the transparency in the mask image is not affecting the stencil buffer which makes the mask write out it’s entire rectangle to stencil buffer,
rather than only the opaque parts of the mesh.

Here is a Playground with my progress: https://www.babylonjs-playground.com/#T32V41#31

I feel like I’m really close, I’m just wondering if someone has insight into what I’m missing.

2

pinging @sebavan

3

are you just trying to mask out ?

if yes in this case instead you could simply disable color write during the rendering of the mask ???

4

Hi @sebavan . I’m trying to only show the background mesh when it falls inside of the mask meshes opaque region. I just tried disabling color write in the playground above, but it didn’t yield the proper result. The proper result would look like:

Annotation 2020-11-12 164033
Annotation 2020-11-12 1640331236×765 43 KB

5

Is it possible the solution involves depth? Does a mesh pass stencil/depth in the transparent regions?

6

Hi, I was actually able to overcome the problem using transparency mode. I set the mask material transparency mode like

maskMaterial.transparencyMode = BABYLON.Material.MATERIAL_ALPHATESTANDBLEND;

and that solved the problem. Here is an updated playground for anyone in the future who wants to know how it is solved: https://www.babylonjs-playground.com/#T32V41#33 .

Thank you for the comments so far as this was a very difficult problem for me and it took over a day to discover this trick. Hopefully this thread can help people in the future.

7

Oh nice glad you found a way :slight_smile: and now I better understand the goal.