AR Camera Occlusion Material

Questions
1

I have been playing around with AR and was wondering if BabylonJS had a material I was not aware of (or a hack of an existing material) that may be able to help me.

What I am looking for is some sort of camera occlusion material (Yes, I know there is an Occlusion Material in BJS, but I don’t think that is what I need, enlighten me if so!)… So a material that blocks the visibility of objects generated in AR, but does not block the visibly of “real” things in the scene?

For example, I would love to apply it to the collision mesh created by the table so the cubes that fall on the floor are visibly hidden until the camera moves around to put them in the foreground.

In the video you can see the cubes that fall on the ground “through” the table… I want to prevent that.

Bonus if this solution still allowed shadows to be cast onto the material optionally - maybe even pick up and blend in virtual-object reflections… This is needed to help integrate real and virtual objects into the same scene.

2

cc @RaananW

3

For any non-transparent material

    mat.disableColorWrite = true
    mat.needDepthPrePass = true // I don't know why, but it seems to be necessary for certain materials.

The principle is to use the depth test, even if the color writing is disabled, the depth of the model can still be used to mask other models

Example:

image
image787×460 12.7 KB

1 Like
4

For shadows, consider copying a mesh and using ShadowOnlyMaterial

image
image803×574 14 KB

5

You nailed the basic occlusion material hack. Thank you.

I still want to try and find a more robust solution than duplicating meshes / separate materials for shadow casting though. Its fine for short objects, but anything furniture size is likely going to run into an occlusion plane before a shadow stretches its full length. That would cut the shadow, rather than projecting it say across a floor and up a wall. Any suggestions from any takers are welcome.

1 Like
6

Got a lot closer with the shadowOnlyMaterial… Occlusion of objects AND shadows on the surface. I switch over to it at 1:06 into the video, after mapping is done.

I completely forgot that material existed, but it is well suited. Amazing how reliant (and how flexible) the Standard Material and PBR materials are, you tend to forget about all the other amazing options we have :slight_smile: .

Now if only it would catch reflections, and allowed for more than one lightsource it would be perfect.

May have to build something custom in the future as this seems like a super useful material for any sort of AR work to blend objects in the scene without tricks like shadow catching meshes attached to the base of models (UE, Unity, all do this approach). Currently don’t know if I am up for the challenge of that one though.Thoughts on how to go about doing this would be appreciated.

Edit:
Hmm, looks like @Evgeni_Popov may have been working on a simular challenge a few years ago… Custom transparent ground that receives shadows, show mirror and use diffuseTexture - #18 by Evgeni_Popov

1 Like