Creating an outline shader in NME. Curious about which technique to employ with the current available nodes

Greetings Babylon,
I am currently creating an outline effect for my toon shader in NME. It’s still discovery for me and I am not exactly sure about the limitations NME has with regards to the nodes currently available.

I found this short & sweet blog post listing different outline solutions:

I am motivated to try to do them all and thus have a few questions.
Only using NME…

  1. Is it possible to clone a mesh and set the render order of each of them? (inverted hull method)
  2. Is it possible to reach the stencil buffer?
  3. Is it possible to “convert normal to view space”? (For the matcap method)
  4. For this technique, is it possible to reach clip space? (See “Working in clip space method”)

I guess more simply put my question is, which of these think are feasible in NME?

1 Like

@Evgeni_Popov could you have a look ?

#1 is not possible because meshes are rendered in the order they are created (at least for opaque meshes). However, you could draw the same mesh two times and change the material to use for each draw. It’s a more advanced usage of Babylon but it is doable. That’s what the outline renderer is doing, for eg => you would probably want to use it as an example (it is located in src/Rendering/outlineRenderer.ts).

#2 is possible, the stencil buffer can be used in Babylon (the outline renderer is using it to handle transparent meshes)

#3 and #4 are only a matter of multiplying the coords with a specific matrix to get coords in view or clip space

2 Likes

Actually for #1, you can use a trick by providing your own sorting function: Babylon.js/renderingGroup.ts at master · BabylonJS/Babylon.js · GitHub

2 Likes

Of course I knew these effects were possible in Babylon, my question has more to do with them being achieved in NME.

Therefore, it is not possible in NME to duplicate the vertices of the input mesh and composite the order in which theyre rendered in the vertex shader? There is a node called “instances”, I am not sure of how to use it but could not this be a way?

for #2 I know we can interact with the stencil buffer, my question once again is wether there is a node that makes me able to manipulate it in NME yet?

Regarding #1, @Deltakosh gave a solution but I’m not sure it will work with instances.

You can’t composite two (or more) instances in the same shader. Instances exist to minimize the number of draw calls: with a single draw call you can draw the same mesh multiple times, but each instance has its own execution of the shader.

You can’t interact with the stencil buffer from the shader. The stencil buffer is a global state of the engine, you have to interact with it through the suitable engine functions. You have an example of stencil buffer usage here: Mesh intersection by using stencil buffer

3 Likes