Video Background Layer Apply Post Process

Questions
1

Hello, and I have been looking at using a web cam in a scene which requires post effects (specifically tone mapping) to the 3d content. I have so far been unable to rig the default rendering pipeline so that it excludes the background layer (with webcam texture) from post processing.

The following playground shows a scene using a post effect on a textured background layer successfully: https://www.babylonjs-playground.com/#08A2BS#381

Here is a scene using the same playground content and rig, but with a webcam texture attached to the background: https://www.babylonjs-playground.com/#971060#2

I have tried a variety of order of operations attempts (creating the nodes in different series), and auditing the rendering pipeline to see where it attaches or what alpha values effects are using, but found nothing which would suggest a fix.

Let me know if I can clarify anything further, and thanks for any help you might lend,
r

*edited to enable hdr in default pipeline

2

cc @amoebachant

3

Here it is, the mask needs to exclude the layer: https://www.babylonjs-playground.com/#971060#3

4

Thanks, although it seems that changing the layer mask from (0x100fffff to 0x10000000) was just as important as setting alpha mode on the last pass.

Can I ask what is the rule on layerMask ids (value range, etc)?

r

5

they are like a flag/binary mask : Babylon.js docs

6

Thanks, although the 0x100fffff value did not work (vs 0x10000000, 0x20000000, 0x11000000, etc).

The getLastProcess loop, which assigns the ALPHA_COMBINE required the following additional settings in order to correctly clear the buffer before redraw (outside of the playground it was redrawing over each new frame).

const lastPostProcess = getLastPostProcess(defaultPipeline);
if (lastPostProcess) {
  lastPostProcess.alphaMode = BABYLON_CONSTANTS.ALPHA_COMBINE;
  lastPostProcess.autoClear = true;
  // do not inherit scene clear color
  lastPostProcess.clearColor = new Color4(0, 0, 0, 0);
  lastPostProcess.forceAutoClearInAlphaMode = true;
}

The code change appears to clear the buffer between frames, but is causing new issues with particles. Is this the intended use? And are you doing anything additional in the Playground to ensure that buffers are being correctly refilled between frames?

Thanks again for the help,
r

7

Can you share a repro of what is not working? @sebavan PG seems to work but because we have no particles we cannot see the issue.

Other alternatives: 2 scenes (one for the webcam, one for the foreground)

8

Hey, and it’s deeply nested within a react project. The fix @sebavan suggested works within the project as it does within the playground, but redraws each frame over the previous without clearing it. The code above remedies the issue, but looks like it is clearing particles which have been rendered before the new frame begins.

Using two cameras is an option, but I have deferred looking into it while I debug the redraw problem.

9

Let me know if we can help further then!

1 Like