So i encounters a problem and i’m not totally sure how to solve it ideally and most performant: I want to be able to blur out parts of my camera/canvas. This parts will be used as background for HTML/CSS UI-overlays, so i need to be able to animate this areas (opacity/width&height)
Here is a screenshot of an UI illustrating what i want to achieve:
My first approach was to simply clone my camera (arc-rotate, following a player), blur it and crop it via viewport. However when i do this the cloned camera is also centered and doesn’t show only a part of the original one.
—> Is there a way to clone a camera, blur it and mask off parts of it?
My next idea was to use a mesh and parent it to my camera, giving it a „blur-everything-behind-material“
—> Is there such a material? Or more correctly asked: Which properties would i have to assign to my material to achieve such an effect
Or maybe I’m completely wrong and there is a way easier solution to achieve blurry UI-backgrounds?
Can someone point me in the right direction please?
Hey @labris
thanks for your reply. i found backdrop-filter when i researched the problem, but unfortunately the compatibility (CSS Backdrop Filter | Can I use... Support tables for HTML5, CSS3, etc) isn’t so nice jet. I think it is less supported than bjs itself hence my plan to use a more “native” solution
You may try th old CSS way.
Before backdrop-filter , the only way to add a filtered background was to add a separate “background” element, position it behind the foreground element(s) and filter it - backdrop-filter | CSS-Tricks
I don’t think there is a native Babylon solution to pass CSS properties to the HTML element. Also please note that CSS blurs are heavy for performance and may slow down the Babylon scene.
then i would need to doublicate the canvas running the bjs, crop it and blur it in css. that seems a bit costly to me… there must be an easier way to mask/crop a camera in babylon? (i only want the ui-content q.e. texts, images and videos in html/css)
Have you considered a post-process to do the filtering?
Alternatively, With a multi-camera setup, you’ll want to specify layerMasks for cameras and meshes. I will typically have a scene camera and a gui cam on different masks, with the gui cam in orthographic mode looking down on a similarly masked plane mesh with an AdvancedDynamicTexture that hosts the GUI controls. Full control over alpha and blending would provide you with an easy blur effect.
For your purposes, you could simply use CSS static or absolute positioning to position your UI elements in the appropriate places (cool UI design BTW!)
You may render the main canvas as screenshots (blobs), blur this image and pass for HTML element background. (Or browser will do with CSS basically the same with its own way).
In order not to kill performance I would make just in CSS something like opacity = 0.8 and very subtle noise overlay to imitate blur effect.
Hey @jelster
thanks for your reply. Yes, i’m using a post-process to achieve the blur. my fist approch was
this.camera_overlay = this.camera.clone();
var left = .7;
var bottom = 0;
var width = .3;
var height = 1;
var viewport = new BABYLON.Viewport(left, bottom, width, height);
this.camera_overlay.viewport = viewport;
var kernel = 128.0;
var postProcess0 = new BABYLON.BlurPostProcess("Horizontal blur", new BABYLON.Vector2(2.0, 0), kernel, 1.0, this.camera_overlay);
var postProcess1 = new BABYLON.BlurPostProcess("Vertical blur", new BABYLON.Vector2(0, 2.0), kernel, 1.0, this.camera_overlay);
this.scene.activeCameras.push(this.camera_overlay);
however this sadly dosn’t give me the desired effect… The viewport only “narrows” the shown picture, while still centering the viewpoint of the camera. I want the exact same(!) picture as my original cam, only crop stuff away from it, that i dont want blured.
so i think i’m searching for a possibility to either crop/mask my cloned camera or my post-process… or something completly different…
@labris i also thought about it. but it seams rather expensive to me. specially when i’m aming at 60fps and want to grad the user the ability to move while the ui is visible… there must be a way to achive this effect in “native” babylon?!?
Thanks for the clarification. Would it be ok to only blur meshes in the affected area? Instead of multiCam and such you could make a material (using NME or shaders) to accomplish the blurring, perhaps
The Babylon GUI system is worth looking at for decent perf in ‘native Babylon’ - ignite help get where you’re trying to go too
sry, i’m afraid i’m not quite sure, what you mean. Put an extra material on the meshes I want to blur? Then probably not. What if a mesh is only partly in the affected area? Then this approach wouldn’t
work, or would it? It would either blur the whole mesh or not, but not half a mesh…?
I thought about using a box or plane with the size of the affected area, parent it to the camera and give it some „lense“-material, but i’m not sure if I could realize a nice gaussian blur effect this way like with BABYLON.BlurPostProcess…
In CSS I would just put the second camera or the post-process-„layer“(?) into a box, position and size it and then set the box to overflow: hidden; This is what I’m searching for, basically
When logging the object generated by new BABYLON.BlurPostProcess I saw _parentContainer, but I couldn’t find anything on it in the API-Documentation…
Of course I’m open to using the BabylonGUI System, if it get’s me my box with overflow:hidden;
(I just think, since the UI-„Content“ in my App is rather complex (Texts,Images,Videos, etc.), I probably better display the Content with HTML/CSS)
When looking at the AdvancedDynamicTexture documentation I saw, that one can define a background - but only with colors? Is there a possibility to use a camera-output as the background – and can I set overflow:hidden; ?
In every path-based-graphics programm there is the possibility to subtract a form from another… Is there maybe something similar in Babylon? Something where I can subtract a form from the camera-output?
Sry, if i ask stupid questions, i’m fairly new to the 3d-world and this amazing framework.
I have done this by passing an ADT to a PostProccess along with the scene RTT. I did a big no no and made two ADTs one that was a mask and one that was the UI, but you can get away with one I found out later.
Once you have all your data in the Post you blur you main scene and then use the mask ADT to mix in the blurred data to the unblurred.
Thanks for your reply. Before i asked my first question regarding the blur-ui topic i also came across RenderTargetTexture but it seamed way to complex for my trivial(?) problem…
But… it seams soooo wasteful. And my FPS went down 20%…
Am i understanding correctly what i’m doing?
By using renderTarget = new BABYLON.RenderTargetTexture(); and pushing every mesh in my scene into it I basically render my scene twice?
Then i use new BABYLON.BlurPostProcess() on my renderTarget to blur the whole, redundant scene.
Next i set um a BABYLON.GUI.Rectangle with a width of 0.333 and put my renderTarget as an image into it, basically throwing away two thirds of the just computed image.
There has to be a better, more performant way to achieve this! Somewhere i red, that the postprocess is like a pixel-layer? Then it must be possible to mask or crop it somehow…?
I already have the image generated by my camera. And it should be more performant to blur only part of the image rather then “clone” my complete scene, blur it an then crop it?
How would someone achieve something like this for example?
Not gonna lie, I have no clue what’s going on in the PG you posted… but the process you described is about right, and you could always make a variance of what ever post process you want and include a mask/crop addition.
Basically you need a full res none blurred texture (your rtt)
Two half res texture for a nice bidirectional blur (or only one texture for a kernel based one).
Then on the canvas you would first render the shape of the ui with planes at the position of your UI skinned with the blur at a position close to the camera (use an orthographic to simplify matching your ui position) then you render a fullscreen plane with the none blurred background on the back to prevent overdraw of the blurred area. This is about the most efficient you can go.
Here is a quick draft - https://playground.babylonjs.com/#WTJ7L1#1 (click on button twice to see effect).
Basically, the second camera should have the same properties as the main camera and render screenshots with blur postprocess for the whole canvas/viewport. The screenshot image with blur (or portion of it) will be the background of UI element.
Thanks for your replies! I did a lot of testing and experimenting, but sadly I’m not quite there jet…
@labris. Thanks for the playground! I also tested your approach, but unfortunately when I want the blurry parts to update in realtime it seems to be not performant enough. (same when I tryed DumpFramebuffer instead of Screenshot)
I’m not quite sure about the second-camera-approch you guys described. How could I attach the RTT or the camera to the GUI this way? Or would I not do this and use the RTT as texture on a plane and set it as a “background” for the second camera, putting the GUI on top?
So my conclusion so far: it is possible to display a RTT within a GUI.
It also should be possible to set the image.source to a base64 output of the RTT (similar to what @labris suggested), but probably it is not very performant as a base64 would be rendert for every frame, correct?
As far as I understand @Gijs approach, the RTT doesn’t get rendert, rather transformed into a 2d-representation an integrated into the GUI?
Because the image is blurred you may A) use lower resolution for RTT; B) render it, for example, every 4th frame to save performance with no harm for user experience.
Works with firefox as well when switching to BJS 4.2… I encounter the same problem on other playgrounds too… → https://www.babylonjs-playground.com/#10KKS5#5 (BJS4.2,ok // BJS5.0,broken)
can it be a BJS version problem with new ImageData() ?
The viewport only “narrows” the shown picture, while still centering the viewpoint of the camera. I want the exact same(!) picture as my original cam, only crop stuff away from it, that i dont want blured.
