Babylon GUI Editor Responsive Mode

[MackeyK24]

Hey guys. I am having trouble with response mode using the gui editor. Our art guy made a demo user interface for me. It is set to Responsive in the top tool bar.

And is loaded into babylon.js at runtime using:

advancedTexture.parseContent(json, true)

Looks great so far with the initial scale to fit option used when parsing content. But when i change the window size its not response. it cuts off the gui:

Is there anything special i need to do get the gui to repaint/redraw on window resize ?

Is there some special container i was suppose to use to get it be response and redraw on window resize ?

@sebavan or @msDestiny14 … got any clues ?

[carolhmj]

Hi, is it possible for you to share the GUI with me or @RaananW ? Preferentially me as Raanan is OOF

[carolhmj]

@MackeyK24

It is set to Responsive in the top tool bar.

Oh shoot, I just looked at this again and realized, the position you showed on the top tool bar is not the responsive one… the responsive mode is supposed to look like this:

I can see why you thought the opposite, we could do a better job of differentiating the on/off states here @PatrickRyan is this something you can look at?

[msDestiny14]

Hey @MackeyK24

Some top of mind things I’m thinking without seeing gui editor snippet:

1. Check if your items are in pixels vs percentage values.

In an earlier iteration of the editor having the mode set to responsive would automatically ensure that when you scale or move a control it would do so in % values unless you manually specified otherwise. Now I see in the editor you can toggle between pixel and percentage for each value using the button next to the value. I’m not sure what going on there at the moment with the responsive mode specifically.

2. We can also try

    advancedTexture.idealWidth = <size here>;  
    advancedTexture.idealHeight = <size here>;
    advancedTexture.renderAtIdealSize;

I believe the true in the parseContent(json, true) sets the size for a texture (if it was saved at a random size in non responsive mode for example) but does not handles the scaling post load.

I think the best bet would be to see it as well.

P.S. Also as a side not I’m currently not a developer for the GUI Editor anymore BUT I still love answering questions based on past knowledge. Just hope it’s still relevant and I’m not completely out of date

[PatrickRyan]

@MackeyK24 there is one bit of complexity here that has a big impact on how the GUI behaves. The responsive flag on the GUI Editor gives you the ability to design as responsive and see how the GUI behaves across different screen sizes or as a set size with the responsive flag turned off. The goal here is to let the user see how their choices in the design will be impacted by how the ADT is displayed.

In either case, a responsive design or not, there are good reasons to mix both percentage and pixel units on control parameters. For example you may always want a button to be 10% up from the bottom of the screen, and always be 60 pixels in height. It’s very common to choose the unit that meets the needs of the control or the screen in individual cases.

But there is one big caveat here and that is how the ADT is created in code. It doesn’t really matter if the json saved from the GUI editor is set to responsive or not because how the ADT is created determines if it’s full screen or not. If you use AdvancedDynamicTexture.CreateFullscreenUI you will always get a full screen ADT whether the json you load was saved as responsive or not. If you use AdvancedDynamicTexture.CreateForMesh you will always get an ADT set at the size set in the constructor. So the responsive toggle in the editor is mainly there so users can preview what the GUI may look like in a particular situation but does not have a lot of weight outside of the editor.

Best practice here would be to always use responsive when designing UI that is supposed to be full screen and change the target size whenever you need to see how the UI is behaving. If you need to create to project on mesh, you will likely know what dimension you need to target so disabling responsive will allow you to enter a specific size. And then for either path, you will mix percent and pixel units as the need arises.

You may even want to swap ADTs based on screen size. So if you get into a situation where the canvas width is below a threshold that suggests it’s on a mobile device, you may want to load a different ADT json that was designed to work on a mobile screen. This could also happen dynamically if the user changes the size of their browser window on desktop and so you may want to have multiple layouts based on the size or proportion the UI has available due to the browser window.

I hope this helps for workflow, but let me know if you have any questions.

[MackeyK24]

Im sorry @PatrickRyan i dont quite understand.

I am using CreateFullScreen in code to render the GUI JSON.

Are you saying i need a separate json for different browser window sizes ?

What if they just change the window size a little… DO i have to have a separate JSON for every PIXEL combo. I dont think that sounds right and would be extremely counter productive and really UNUSABLE

Is there NOT an advancedTexture redraw or resize or something i can call on window.resize or something.

How is everyone else in the community doing full screen GUI and dealing with window resize ?

[MackeyK24]

So i tried every combo of responsive mode and pixel/Percent toggles for position, width and height

I load the json using Create FullScreen

BABYLON.GUI.AdvancedDynamicTexture.CreateFullscreenUI("GUI", true, this.scene, BABYLON.Texture.BILINEAR_SAMPLINGMODE);

and parse content like so

advancedTexture.parseContent(parsedJson, true);

But still does work when i resize the browser window.

Is this a bug ?

The gui editor seems to not be usable like this, if you cant resize the browser window.

Am I missing something or just plain doing something wrong ?

[PatrickRyan]

@MackeyK24, I’m sorry I was unclear. I assumed you were using CreateFullScreen in code since your GUI was rendering full screen even though the GUI editor was set to not be responsive. I was pointing out that designing the GUI in editor can differ from how it is constructed in code so even if you intend a design to be projected and this disable responsive in the GUI editor, the ADT can be constructed with CreateFullScreen which will render it full screen. So this is something to consider in design.

When the window resizes, the ADT resizes automatically and will render the controls as they were designed. So if you design a rectangle to be 50% of the width of the ADT, it will render at 50% width of any size of the browser window. If you design it to be 960px (half the width of a 1920px screen) and the window is reduced to smaller than 1920px wide, the rectangle will not render at half the screen width and will eventually cut off when the window is narrower than the rectangle.

When I was speaking about different JSON files based on screen size, it was because there is a cutoff when browser window could be considered a mobile format. An example would be to look at http://doc.babylonjs.com and then reduce the width of the window. At some point the sidebar disappears because we consider the window to be a mobile format which is just using different CSS. For Babylon GUI, you could use the window resize event and determine if the width falls beneath a certain measure and then load a different ADT json with a different layout.

For the question you just asked, there is one, possibly two, other things to consider to make your layout stick to the sides. The first is alignment. Right above your red box around the position and width/height are the alignment buttons. Right now you are aligned center horizontally and vertically and any offset is taken from there. This means that whatever you are using, percent or pixel is measured from the middle of the ADT. If you want your controls to stick to the sides of the window, it’s better to left align controls on the left and right align controls on the right. This way you are taking offsets from the sides and not the center which will more accurately position the controls. Same with top and bottom.

The other thing that you may want to consider is the pivot point for a control. We default pivots to the center of the control, but if you want to use controls of different widths all aligned on the left side of the screen, you may want to move the pivot point of each control to the left edge of the control like (0, 0.5) to set left edge and center vertically. All offsets are measured from the pivot point, so if you have controls of differing widths, there offsets from the left edge would be different if their pivot points were all set to the center of the control.

So you may want your top left control:

To have parameters that look something like this:

Note alignment left/top and pivot point (0,0) with position and size both in pixels. This means that control will hang at the same place from left and top no matter what size the screen is like below:

And having the GUI Editor in responsive mode allows me to change the size of the ADT in the editor to check out how the GUI will respond in different proportions.

[MackeyK24]

So your saying we should design the layout better, using features like described above and then when rendered in full screen, it should resize.

YO @Pryme8 … what do you think about all this ?

I am trying to keep our workflow… Where our artist can go from creating the FIGMA right to the babylon GUI Editor and produce a JSON we can load into babylon at runtime and then wire up the controls.

[Dad72]

What PatrickRayan describes is how I build the GUI. I only use pixels to place and size the controls and with right, left, top and bottom alignments in pixels as well. I don’t use the editor though, everything is hard coded.

Result, I have no problem resizing regardless of the screen size. There’s only the minimap where I resize the window to fit the minimap GUI image (which you’ll probably need to do.)

[MackeyK24]

Yo @Dad72

What do you mean resize the window to fit the minimap ?

Resize the actual gui container the minimap is rendered in on window.resize event or something ?

[MackeyK24]

But in general, i think i get what your saying.

First… the Responsive control in the top tool bar of the gui editor really has NOTHING to do with the render of the gui at runtime. Its a design time preview of the content in the gui editor

Second… We need to actual design it to be responsive by using the alignment properties in the gui editor… TOP LEFT in PIXELS. Kind like old school HTML-CSS:

position: absolute;
left: 10px;
top 20px:

[PatrickRyan]

That is a very apt comparison as Babylon GUI was modeled after many of the aspects of CSS when it was first created.

[MackeyK24]

Yeah… I thought the Response Switch was just auto-magic that set your designed gui to be in Response Mode… Which implies you can resize the window and the content would resize as well.

We should make that a bit more clear in the gui editor… Maybe call that component in the top toolbar Response Preview … or something

[mawa]

Yes. I do the same. Old school mode for old people like us It might not be very ‘sexy’ for an eng always looking to break things by adding more features but at least, it works

[mawa]

@MackeyK24

Maybe call that component in the top toolbar Response Preview … or something

I kind of agree but then we would need to explain what the ‘response preview’ or ‘response mode’ is.
It’s actually kind of the same with hooking meshes from the editor. It’s not all that ez to understand at first since the editor is only part of the app build. I think there will still remain a level of confusion no matter the words or explanations (at least until you can access all parts from the editor, which I believe won’t happen anytime soon)

[mawa]

@MackeyK24

We need to actual design it to be responsive by using the alignment properties in the gui editor… TOP LEFT in PIXELS. Kind like old school HTML-CSS:

You could say that; but then remember it took a quarter of a century for css to evolve to v3.
If you ask me, the people at BJS are doing a smashing job. And, at this pace, it won’t be long before we get there.

[MackeyK24]

4 sho… I am an old school programmer… That is where the reference comes. Not knocking ANYTHING about BJS or its GUI/Editor.

Everything i do in the WebGL is done using my Babylon Toolkit… I am the biggest fan of BJS

[Dad72]

@MackeyK24

What do you mean resize the window to fit the minimap ?

I was talking about the BABYLON.Viewport() to display the contents of the mini-map. Then above the viewport, you have your graphical interface (the image of the mini-map). When you resize the browser window, it resizes the GUI, but not the BABYLON.Viewport() which distorts and becomes oval.

I do something like this, to resize the viewport()

     resizeViewport()
	{
		const width = this.engine.getRenderWidth();
		const height = this.engine.getRenderHeight();
		const staticValue = {Width: 230, Height: 237, Left: (width - 213), Top: (height - 220) };	
		this.camera.viewport = new BABYLON.Viewport((1 - (staticValue.Width + staticValue.Left) / width), (1 - (staticValue.Height + staticValue.Top) / height), (staticValue.Width / width), (staticValue.Height / height));
	}

[Pryme8]

I believe if you do “BABYLON.GUI.AdvancedDynamicTexture.CreateFullscreenUI”, it will handle the resize all on its own and keep the adt fully covering the screen.

Now to get it responsive, the components just need to be setup with the correct top, bottom, left, right numbers instead of width and height and you can essentially “anchor” them to positions that stay the same as the size changes.

Now if you are looking for something more robust where the layout changes when certain sizes are seen I think the solution would be to have containers in the adt that are associated with the different screen ranges.

We can talk more about it in slack on Monday if you would like?