SelectionOutlineLayer: How to disable transparency entirely?

Questions
1

In SelectionOutlineLayer the selected mesh can be seen through meshes in front of it. Is there a way to disable this effect entirely to save some computations?

outlineLayer.occlusionStrength = 1; (or any value >=1) gives me the desired visual effect, but does it cut off the computations? Playing with occlusionThreshold doesn’t disable the effect visually.

Having just 21 highlighted low-poly meshes drops my FPS from 130 to 110. Of course, it depends on many factors, but it feels like it’s too heavy, so I am wondering if I can cut some computations off.

2

occlusionStrength = 1 is the right setting if you want the outline to be fully hidden by meshes in front of it, but it only changes the final visual result. It does not skip the occlusion work internally: the layer still uses the depth renderer and the compose shader still samples/compares the depth texture.

occlusionThreshold only changes the depth comparison sensitivity, so it won’t disable the effect either.

For performance, you can try reducing the layer render target size, for example:

const outlineLayer = new BABYLON.SelectionOutlineLayer("selection", scene, {
    mainTextureRatio: 0.5,
});
outlineLayer.occlusionStrength = 1;

or use mainTextureFixedSize if you want a fixed budget. Also make sure to clear/dispose the layer when nothing is selected.

If you only need a simple per-mesh outline and don’t need the screen-space/grouped behavior of SelectionOutlineLayer, mesh.renderOutline = true may be a cheaper alternative.

3

I was using mesh.renderOutline before, but it draws lines along every edge, while the outline layer draws lines only along the most outward edges.

They both drag the performance down, but with meshes in my case I would say that outline layer is slightly more performant (1-2 frames) and looks much nicer (considering that I already applied occlusionStrength = 1).

4

Thanks for the report! I opened Add selection outline depth occlusion toggle by Popov72 · Pull Request #18498 · BabylonJS/Babylon.js · GitHub to add a useDepthOcclusion option to SelectionOutlineLayer.

This does not implement hidden-outline removal for non-visible parts of selected meshes. Instead, when useDepthOcclusion is set to false, all selected outlines are visible, including outlines behind other scene or selected geometry. We added this mode because it can still be useful for some scenarios, and it is more performant than the default useDepthOcclusion = true mode because it skips the scene depth renderer and compiles out the depth-occlusion shader path.

The default remains true, so existing behavior is preserved.

5

Yeah, but it keeps outline visible through other meshes, basically occlusionStrength = 1 gets ignored. I would prefer to have both at the same time. Nonvisible mesh and border + computations savings.

6

Unfortunately, that’s not possible; we have to generate the depth buffer to hide the edges of invisible meshes.