GUI Editor, JSON output (is barely useable)

[mawa]

Dears,

As much as I hate doing so, I’m here today to complain about the GUI Editor’s JSON export.
I guess yours know how much I like this tool and I’m also advocating for it, so this is why I cannot let this through .

While struggling to port my sketched GUI in the editor to production (because I cannot replace the links from my dropbox with my production links within the editor - a topic posted separetely and a discussion I quickly had with @deltakosh here – Gui Editor (how to) move to production? - #12 by Deltakosh)… it gave me the opportunity to save to json and edit the export for manual change of the production links for my images.

And this is where I nearly fell from my chair when opening the file:

Believe it or don’t (although it will be hard to deny with the samples attached below), my fairly crude not to say ‘basic’ GUI created in the GUI Editor and saved to json totalizes about 45k lines of properties!!!
That’s about twice what I have in my entire ‘Space Museum’ scene.

While it took me about 20min only to be able to reformat the JSON to push each property to a new row within my text editor, I realized that the file contains at least 90 to 95% of completely useless passed-on properties and parameters (which are defaults). It looks to me like generating the json takes just each and every single parameter and property available in the API and passes it single-handedly for each container and control. In fact, there are some in there that, honestly, I didn’t even know they exist

Fact is this JSON output from the Editor is 1) nothing even close to optimized 2) barely manageable/useable. I don’t know if the snippet version does the same?

Fact is, even with my shit-dev skills, if I was to reproduce this GUI in code, I would likely have something like 500 lines (while being careless). Here, I have 45k, seriously? I don’t even want to check out my FS GUI which is about 5 times the complexity would look like in the JSON.

So, below is the link to this GUI in the Editor:

Babylon.js Gui Editor

And here for the text file, pushed to lines, from the saved JSON:
elevgui_json_lines.txt.zip (24.0 KB)

It should basically look like this (screenshot of the first 65 lines… out of 45000):

Now, @carolhmj @PatrickRyan , what I am expecting following the above is that you would tell me something like “Yes, mawa, this is clearly not what it should be and we are going to take care of this ”… I mean, I believe I can understand the complexity behind it and eventually why it is like this at this moment… but honestly, anyone out there who thinks we can really leave it this way? Just asking and of course, as always, my opinion only.

Meanwhile, have a great day

[Deltakosh]

Haha:) I like when people are passionate!

The json export is auto.atic and as you said is going through all properties and export them.

The alternative would be to only ecoor what changed but it would force us to track each property and add a dirty flag to them so it will make the framework buffer. Furthermore outside of babylonjs (like in the figma loader/Exporter) they would have to catalog what is default and what is not

[PatrickRyan]

@mawa, I would add to the practical reasoning of @Deltakosh that we would never want users to open any of the json files for GUI, NME, NGE, Particles, ACE, or any other tool and need to go in and make changes to the json manually. That kind of defeats the purpose of having the editor. All of these tools can have their output written directly in code as well and owing to the engine defaults, you will always write fewer lines of code than the number of parameters on any element. But the idea is to have a graphical editor to either see the artifact you are creating instantaneously or to help abstract the sometimes-difficult parts like shader optimization.

Instead, it makes more sense to address difficulties you may be having with the tool in updates to the tool itself. The goal should always be to never open the json file. That said, if you do have updates to make in the GUI like the links you mentioned, you can certainly do that in code. Let’s say you have your GUI json with links from your development environment and need to push them to production. If you aren’t talking about a lot of elements, you can certainly just change the link in code.

However, there may be a feature request hiding here. It seems logical to have the ability to change the base URL address from development to production, so maybe what we need to do is change things up slightly so that you can change the base URL string to point to a production server while leaving the relative link to images the same. This would have solved your issue of migration and prevented needing to either go into the json or manually update the GUI in editor. @Deltakosh, what do you think about something like that?

[Deltakosh]

I would be totally happy to! I can add a callback at load time so the user can edit the loaded urls

[mawa]

Thanks Both,
I was thinking I might get this kind of answer. I’ll be happy not having to edit the json.
However, I’d still like to ask what kind of impact such a large file for a GUI can have on performance / loading time?

[Deltakosh]

You would be impressed by how far the json parsers are

[mawa]

OK, then…impress me …Just joking, of course. In fact, I just replaced all my ADT for mesh with the cloned version and I am now parsing from the JSON hosted on my server (with edited links only). And I have to admit: the performance is there So, I guess it’s all good. Just waiting for the candy now that will let me replace my path for all images in my FS GUI with the production link. Because, honestly, I don’t really feel like editing this one. May be just ping me when done, if by any means possible? Meanwhile, have a great day

Edit: Forgot to mention that, in case of dropbox (my case), when using the absolute path, you will also need to remove the ‘rlkey’ part which I believe is still included and extending the source name. Read everything that comes after the file extension (I.E.: https://dl.dropbox.com/s/rANdoMGeneR4tedLink/my-file.glb?rlkey=rANdoMGeneR4tedValUe)

[Deltakosh]

Please create an issue for me on the repo and I will add the feature for you

[PatrickRyan]

Ask and ye shall receive. GUI JSON parse - provide callback to allow updating link URLs from development to production servers · Issue #15064 · BabylonJS/Babylon.js (github.com)

[mawa]

If I would be gay, I would ask you if you want to marry me You rock Thanks so much and have an awesome weekend

[Joe_Kerr]

@mawa

I would ask you if you want to marry me

You could at least name your next child after him

@mawa

While it took me about 20min only to be able to reformat the JSON to push each property to a new row within my text editor

I realise the issue is resolved. But just want to mention: never do stuff like that by hand; especially if it is structured data. There is very likely always an automated solution. In this case of just formatting, there are some online tools or, if privacy/performance issues, Notepad++ comes with a json formatter, too, for instance.

[mawa]

@Joe_Kerr

name your next child

Haha…No faen chance. I’m a child, so I know how much of a bother it is Best I can deal with would be a dog

@Joe_Kerr

never do stuff like that by hand

Never tell me NOT to do something… or rest assured I will try anyway
Jokes apart, I know about the tools and i.e. notepad++ (though I’m committed to sublime text, which is sublime,… at least, visually ) and no, it doesn’t have the json reformatting function. But then, just between you and me… may be, just may be, I slightly modified the story around it May be it didn’t take me 20min to reformat it, may be I was just doing things while my editor was processing in the background and may be I was editing videos at the same time and may be I was doing all that on my 13 years old test rig But, shhh no need to tell anyone… In the end, looks like I got what I wanted. For whatever happened or will come out of it, just let the others know that 'it wasn’t me’

[Joe_Kerr]

Oh, oops, sorry. I got that wrong.

[Deltakosh]

Alright! PR done: Fix #15064 by deltakosh · Pull Request #15079 · BabylonJS/Babylon.js (github.com)

This PG will work when merged: Babylon.js Playground (babylonjs.com)

Relevant code:

advancedTexture = BABYLON.GUI.AdvancedDynamicTexture.CreateFullscreenUI("UI");
    advancedTexture.parseSerializedObject(json, undefined, (url) => {
        console.log(url);
        url = url.replace("grass", "crate");
        return url;
    })

[mawa]

Thanks, that’s awesome timing. I was just about to finally release the update for myproject and was more or less just waiting on migrating the FS gui.

However, I do have a question: I’m not 100% sure about how to use this properly.
I mean, honestly, I was more thinking about a solution that would allow us to enter a path for production right within the editor settings (or within an option panel opening when clicking ‘save’ from the editor), so the JSON would already have the new links.

Here it looks like either I am missing something or I will have to split or slice or whatever js method to replace my string, is it? Like I don’t know about the generated by dropbox link and I also have to remove the suffix (using a js method, is it?). Or is there anything I’m missing from the parameters I can set?

Here’s a basic example PG from the PG you created:

Babylon.js Playground

[Deltakosh]

I wanted a really flexible solution and not something to address that only issue (what if for example you have multiple servers for your data based on business requirements ? We see that quite a lot)

So you have a rewriter now but then it is quite easy to use if you really on regex. But if you don’t like regex here is an example without

function rewriteUrl(newServerAddress, currentUrl) {
// Parse the current URL
const url = new URL(currentUrl);

// Extract protocol and port if present in the new server address
const [newProtocolAndAddress, newPort] = newServerAddress.split(‘:’);
const [newProtocol, newAddress] = newProtocolAndAddress.split(‘//’);

// Update the protocol if specified
if (newProtocol) {
url.protocol = newProtocol + ‘:’;
}

// Update the hostname
url.hostname = newAddress || newProtocolAndAddress;

// Update the port if specified
if (newPort) {
url.port = newPort;
} else {
url.port = ‘’;
}

// Return the updated URL as a string
return url.toString();
}

// Example usage:
const newServerAddress = ‘https://new-server.com:8080’;
const currentUrl = ‘http://old-server.com:3000/path/to/resource?query=param#hash’;
const newUrl = rewriteUrl(newServerAddress, currentUrl);
console.log(newUrl); // Outputs: ‘https://new-server.com:8080/path/to/resource?query=param#hash’

[PatrickRyan]

@mawa, there are a couple of issues with the production server URL set in the tool user flow. The first is that when we are loading images through the browser’s loader, we get the full URL for the image and there’s nothing to stop anyone from using images from multiple servers. Pulling images from Dropbox and GitHub in the same GUI is completely valid, so the idea of having one production server address to handle all image links isn’t as flexible as needed. Trying to handle multiple server addresses is complicated at best in the GUI Editor.

Additionally, having this available outside the GUI editor can help users who aren’t using the editor but need a callback to be a central place to change the server URL. Lastly, this is the best method we have to do things at both ends of the URL like adding or removing an rlkey property suffix in the address for Dropbox.

Your example looks good in terms of using the callback correctly. In terms of using the feature with GUI Editor files, you can just load your GUI json as text using something like Asset Manager which gets the json loaded as text. Then use JSON.parse() on the text file and pass this as the JSON to advancedTexture.parseSerializedObject(). You can then do anything you need to with the string with replace, splice, split, add, etc. and then return the correct URL. While this may seem a little more involved, it really does solve most of the problems we identified in the user flow.

Hope this explanation helps clear things up a bit. Happy GUI-ing!

[jamessimo]

I wanted to drop in since I work a lot with outputted Babylon GUI json due to my work with the Figma to BabylonJS plugin.

Yes the files are massive, but as other members said, these json files don’t need to be edited directly.

However I agree that they maybe too large, i notice some properties which I never see utilized or changed. Examples off the top of my head is “metadata”,”max layout cycles” and “render to imitate texture” to name a few.

I do think that the serialization step can skip items if it knows it’s got a default or unchanged value from the type.

Cool to see the URL callback though, may help me in my debugging.

[mawa]

Thanks EVERYONE for all your input, efforts and insight on this. I have to admit I was looking at it only from a user/user experience perspective. Obviously, I did miss some parts and use cases, such as using different path for the server or a change in protocol. I suppose that’s why yours do what you do and I… well, I’m just a simple digital artist

So, in fine, and to I guess close this case - The solutions stated above, I believe would work and are fine by me. Although, I have to admit, from my ‘digital artist’ perspective, using an editor and still having to go through a process that is, as @PatrickRyan 's quote is “involved”, feels a bit odd.

I suppose it is opposite views: designer VS dev. As a designer, when I get a source that’s not appropriate, I have a natural tendancy of trying to replace/normalize the source. Whereas, a DEV is happy just processing it to apply second or even third level modifications to it

Well, I sort of always have been something like “the man in the middle”. I suppose I need to continue to work on finding the best balance, working out my dev orientated mindset just a little bit more

Again, thanks everyone and I think it’s a good thing we added this. And with this, I shall be soon back now with my latest and biggest project update (with everything pushed to production) and that, kind of makes me happy

Meanwhile, have ALL a great day and an awesome weekend

[HiGreg]

I solved a similar problem with material serialization by writing merge and unmerge functions for json objects. I’m sure this will break if the reference object changes during a Babylon update, but this is working for me at the moment. It’s not been reviewed by any javascript experts, so there may be major flaws. unmerge() includes some console.log() statements to indicate which properties are "d"eleted and which are "k"ept.

// MODIFY to0 to an object needed to merge with j to obtain toO
// simplistic. works for base type properties and nested key/value (properties).
function unmerge (toO, j) {
    console.log("u",toO,j)
    Object.keys(j).forEach(p => {
    // https://stackoverflow.com/a/2673229
        if (j.hasOwnProperty(p)) {
            if (!!j[p] && j[p].constructor === Object) {
                unmerge(toO[p], j[p]);
                if (Object.getOwnPropertyNames(toO[p]).length === 0) delete toO[p]
            }
            else if (toO[p] == j[p]) { console.log("d",p); delete toO[p] }
            else if (Array. isArray(toO[p]) && Array. isArray(j[p])){
                if (toO[p].every((v,i) => v === j[p][i])) delete toO[p]
            }
            else {console.log("k",p);}
        }
    })
    // a.every((val, idx) => val === b[idx])
    // return modified object to enable compound statements (with embedded "new")
    return toO;
}

// simplistic merge. works for standard properties and nested key/value (properties)
// adds/replaces properties on toO from fromObjs
function merge (toO, ...fromObjs) {
    fromObjs.forEach((j)=>
        Object.keys(j).forEach(p => {if (j.hasOwnProperty(p))
            !!j[p] && j[p].constructor === Object ? merge(toO[p], j[p]) : toO[p] = j[p]
        })
    )
    // return modified object to enable compound statements (with embedded "new")
    return toO;
}

Here’s how I use it to output the difference between the default StandardMaterial and a current material in the scene:

        const cm = mesh.material.serialize();
        const sm = new BABYLON.StandardMaterial("").serialize();
        const dm = unmerge(cm,sm);

        // show (as a string)  the difference between current material and Standard Material           
        popup(JSON.stringify(dm)); 

I’m intending to use it like this:

        const sm = new BABYLON.StandardMaterial("").serialize();
        merge(sm,dm)

But I haven’t tested it yet. Note that the unmerged delta includes name, id, and uniqueId properties, so I’m not sure how well it actually works. It won’t work for properties that require method calls (are there any?)

I also use merge for more tersely setting object values and function options:

    const editButtonDone = new BABYLON.GUI.TextBlock("edone","Done");
    merge(editButtonDone,{heightInPixels:40, color:"black", background:"yellow", widthInPixels:20 })

Or

    const main = merge(new BABYLON.GUI.StackPanel("mainEdit"),{height:"50%",isVertical:true})

And for function options:

function myFunction(param,options={}) {
    var defaultOptions = { // a whole bunch of default options
    }   
    merge(defaultOptions, options );
    options = defaultOptions
    // ... the rest of myFunction