Trying to get outlines on every object added to a HighlightLayer

Questions
1

My scene has a ton of objects so I couldn’t take the approach of multiple HighlightLayers.

I’m also interested in other possible approaches. I tried using the SSAO post process to some success making outlines, but it didn’t quite capture the style I’m going for.

Playground link - I don’t want the outline of the two objects to merge together

2

I got some results I’m pretty happy with from modifying the sharpen post process. :smiley:

Screenshot from 2022-07-24 09-39-13
Screenshot from 2022-07-24 09-39-131920×1219 323 KB

Here’s the glsl code:

#ifdef GL_ES
precision highp float;
#endif

// Samplers
varying vec2 vUV;
uniform sampler2D textureSampler;
uniform sampler2D depthSampler;
uniform vec2 screenSize;
uniform vec4 sharpnessAmounts; // x = amount, y = offset, z = color threshold, w = depth threshold

void main(void)
{
  vec2 onePixel = vec2(1.0, 1.0) / screenSize * sharpnessAmounts.w;
  vec4 color = texture2D(textureSampler, vUV);
  vec4 depth = texture2D(depthSampler, vUV);
    
  vec4 edgeDetection =
    texture2D(textureSampler, vUV + onePixel * vec2(0, -1)) +
    texture2D(textureSampler, vUV + onePixel * vec2(-1, 0)) +
    texture2D(textureSampler, vUV + onePixel * vec2(1, 0)) +
    texture2D(textureSampler, vUV + onePixel * vec2(0, 1)) -
        color * 4.0;
        
  vec4 depthEdgeDetection =
    texture2D(depthSampler, vUV + onePixel * vec2(0, -1)) +
    texture2D(depthSampler, vUV + onePixel * vec2(-1, 0)) +
    texture2D(depthSampler, vUV + onePixel * vec2(1, 0)) +
    texture2D(depthSampler, vUV + onePixel * vec2(0, 1)) -
        depth * 4.0;

  float amount = dot(edgeDetection.rgb, vec3(1.)) / 3.;
  float amountDepth = depthEdgeDetection.r;
  vec4 edge = (amount > sharpnessAmounts.y || amountDepth > sharpnessAmounts.z) ? vec4(vec3(clamp(1. - amount - amountDepth, 0., 1.)) / sharpnessAmounts.x, 1.) : vec4(1.);
  gl_FragColor = color * edge;
}
3 Likes
3

I believe your approach of choosing an alternative is good (although I do not know what result you wanted to achieve). Sadly, the highlight layer has multiple issues with the rendering order and is also heavily impacting performance. I hope some day we will be able to improve it but it doesn’t look like something easy.

1 Like
4

I haven’t looked into exactly how the HighlightLayer is working, however I recall seeing one method where each object is rendered with a different color before processing that achieved individually outlined objects.

5

Indeed, there are other methods such as i.e. the edge rendering

You can also achieve outline based on vertices, using shaders (such as the toons shader) and/or combined with a node material. It all depends on the type of design/fx you want to achieve.
But none really have the same ‘glowing’ fx as the highlight layer. So I believe an improved highlightLayer would certainly be welcomed.

1 Like
6

Here is PG of OP’s shader if anyone’s interested.

image
image1230×1036 114 KB

and here’s edge for normal instead of color

image
image1270×1040 123 KB

2 Likes
7

Cooool !!!

8

Hi all sorry for necrobumping this thread!

I’ve been trying to create a post process with the NME using this shader. This is as far as I am right now, but I think I’m not understanding some aspects of it, or how to translate the raw shader code into the NME. Can you guys help a bit or can you see what is blatantly wrong?
Babylon.js Node Material Editor (babylonjs.com)
Simple PG with this:
Custom postprocess demo | Babylon.js Playground (babylonjs.com)

EDIT: Updated NME

9

Sorry for late response.

I don’t think normal buffer is straight forwardly available in NME.

here’s my take on color edge detection

1 Like