How to blur parts of a camera/canvas as an UI-background

[Blake]

It seems that texture.readPixels returns a promise now, so using await or then fixes the error when using the current BJS version.
https://www.babylonjs-playground.com/#10KKS5#12

For the other playground thou, that does what you want, the external script would need to be updated to wait for the promise to resolve as well I think. Here I copied the script onto the playground and used .then() to wait for readPixels to finish.
https://www.babylonjs-playground.com/#5E96NN#6

This fixes the error, but IDK how good of a solution it is. Probably it would be better to call readPixels and putImageData less often than once per frame I think.

[jelster]

@Blake

texture.readPixels returns a promise

This is a change to support WebGL2 and WebGPU

[sebavan]

@fab

I’m not quite sure about the second-camera-approch you guys described. How could I attach the RTT or the camera to the GUI this way? Or would I not do this and use the RTT as texture on a plane and set it as a “background” for the second camera, putting the GUI on top?

yup it would be a background for the GUI but a foreground of the scene. not passing it to the gui is the best option to prevent expensive GPU → CPU → GPU resources sharing operations.

@fab

When setting up a plane and giving it a rtt-material it isn’t more performant, than the approach I used based on this playground https://www.babylonjs-playground.com/#KG6SGU#13 by @Gijs, where I am able to paint the RTT directly into the GUI. (Via: How to show a RenderTargetTexture on GUI?)

Yes it is normal cause in this case the GUI is rendered through WebGL not through the Canvas2D. The approach is actually fully similar to the other one.

@fab

Even more sadly it appears I’m not competent enough to remove the whole greenscreen-custom-shader-thingie and just display the RTT without any shader

Displaying the RTT without shader would mean introducing back to huge perf hit of sharing the resources across different contexts.

About the aliasing it is probably cause you are using a post process and you should turn on MSAA on your RTT by doin rtt.samples = 4;

[fab]

Thanks a lot for all the replies!

So, again I did some experimenting…

@labris: I tried the render only nth frame with screenshot, but then the blurry area is (of course) async, which doesn’t look nice. This gave me a nice idea though: Whenever I don’t show the RTT in the GUI, I set the renderTarget.refreshRate to 1000, to save FPS and it works great. Thanks.

@blake: I tested it and the performance of the canvas-approach in the fixed playground is not very good, as you suggested… (and I think I slowly begin to understand why this is and the differences between the two approaches… Thanks a lot for fixing the playground anyways! (Once my elevator and the controls are fully ready I’ll invite you for a ride

So I looked a bit more into the other (shader-based) approach and stripped it of all the parts I don’t need, removing the greenscreen-part and converting it into a position-based approach. Which works rather well, I’d say… I’m even able to raise the brightness of the affected area… Webgl is fun!

Here is a PG —> https://playground.babylonjs.com/#P8HK81#10

However with every bit I understand, there are a lot more things I don’t unterstand

This part should make sure, that the RTT (and the shader?) is only displayed within the borders of the „image“.
if(insideBox(orgPixel.xy) < insideBox(vUV)){}

But somehow the shader affects also the surrounding scene in a way that (as far as i understand) i didn’t put in anywhere. It makes the edges pixelated (which should be fixable via MSAA as @sebavan suggested) but also messes somehow with the emissive parts(?) of images/meshes. Here is a pic of how it affects my scene.

One can also observe the behavior in the playground (on top of the sphere). It goes away when switching off the guiTexturePostProcess in the inspector.
Can someone please explain to me why this is happening and maybe how to fix it?

Furthermore I lose the opacity of the GUI elements on top of the shader affected area and the button-text gets a pixelated black outline. How is this possible when they are on top of the shader area, or am I thinking CSS here?

When I also plug in my DefaultRenderingPipeline I get even more strange results. Again in combination with the postProcess. (the Pipeline worked fine before)

The DefaultRenderingPipeline affects the whole Scene, the postProcess only camera or RTT? Jet I can recreate all the DefaultRenderingPipeline effects with postProcesses? What is the difference between the two? Why use DefaultRenderingPipeline?

Sorry for the (again) long post, I would offer a potato, but I don’t know if it’s customary around here…

I hope at least my questions are getting slightly less stupid

/////

I really enjoy this forum. It reminds me of the good old (intenet) days - before the invention of social media - when people just met online and discussed topics. Without screaming at each other… Thanks for that!

[Deltakosh]

@fab

Without screaming at each othe

I’m personally really attached to that as well