Semantic render pass possible?

Hi, I was wondering if it’s possible to implement a secondary render pass offscreen where some objects render themselves with unique solid colors, and others just render as black. This is used in machine learning applications to generate a semantic label map of a scene.

1 Like

Hello and welcome!

Seems that you may use Offscreen Canvas | Babylon.js Documentation

1 Like

You can create a custom shader or material and then once perFrame apply it to all the activeMeshes replacing their current material. Capture the RTT data, then reapply the old material back.

https://playground.babylonjs.com/#TCFPG3#62
This demo is broken now unless you go back to version 4.1

But it uses a process like that, starting at like like 628

You can do something like that:
https://playground.babylonjs.com/#SYQW69#868

However, it’s slow because the material of each mesh is constantly flipped between the material for the RTT and the material for the regular rendering and each time the engine must recreate the corresponding effect (in fact the effect is not recreated because there’s a cache, but preparing everything to be able to query that cache takes time).

You can use the technic describes here to improve the performances:
https://playground.babylonjs.com/#SYQW69#866

1 Like