Boosting BABYLONJSGUI Development Speed Tenfold

[MILIFIRE]

Repository link: GitHub - MILIFIRE/isin-ui (Seeking stars)

npm link: isin-ui - npm

Background In WebGL projects, we often require UI elements such as confirmation dialogs, prompts, loading bars, etc. These can be implemented in two ways:

Overlaying DOM elements on canvas and Pure canvas implementation

Points to Address

1. With the advent of VR, clients may want to use webXR technology for interactive projects. However, it’s impossible to use DOM elements in immersive webXR mode.
   
   

In the first mode, it’s feasible for mobile (M端), but not for VR immersion, resulting in double development effort (cost reduction and efficiency improvement needed).

2. WebGLGUI code development is outdated, resembling the era of JQuery. Using component-based development significantly enhances efficiency and reusability. Below is a code comparison:

// Native GUI development
var advancedTexture = BABYLON.GUI.AdvancedDynamicTexture.CreateFullscreenUI("UI");
var panel = new BABYLON.GUI.StackPanel();    
advancedTexture.addControl(panel);   

var rect1 = new BABYLON.GUI.Rectangle();
rect1.width = 0.2;
rect1.height = "40px";
rect1.cornerRadius = 20;
rect1.color = "Orange";
rect1.thickness = 4;
rect1.background = "green";
panel.addControl(rect1);   

var rect2 = new BABYLON.GUI.Rectangle();
rect2.width = 0.1;
rect2.height = "40px";
rect2.cornerRadius = 20;
rect2.color = "Orange";
rect2.thickness = 4;
rect2.background = "green";
panel.addControl(rect2);            

// Component-based development
<AdvancedDynamicTexture v-if="textureRef" :texture="textureRef">
  <StackPanel :isHighlighted="true" :rowIndex="2" :columnIndex="2">
    <Rectangle
      :thickness="4"
      color="Orange"
      :cornerRadius="20"
      background="red"
      width="40px"
      height="40px"
    ></Rectangle>
    <Rectangle
      :thickness="4"
      color="Orange"
      :cornerRadius="20"
      background="red"
      width="40px"
      height="40px"
    ></Rectangle>
    <Rectangle
      :thickness="4"
      :cornerRadius="0"
      background="red"
      width="40px"
      height="40px"
      :onPointerOutObservable="log"
      :scaleX="0.5"
    >
    </Rectangle>
  </StackPanel>
</AdvancedDynamicTexture>

Advantages and Disadvantages

1. Limited number of components currently available.
2. Lightweight, based on VUE3 virtual nodes.
3. Effectively reduces UI development costs in WebGL projects and improves reusability.
4. Automatic recycling of component lifecycles is advantageous.

[jabulanikid]

I was just about to ask a question about UI Programming in WebXR. Thanks for this. I will check out and let you know if I have any questions

[brianzinn]

hi @MILIFIRE,

Looks like a great start! I have been programming in vue for around 4 years now. I did make an attempt similar to what you are doing - I agree with your point:
“Limited number of components currently available”

I addressed that on one project using code-gen with ts-morph. There is basically 1 file that generates pretty decent support for Babylon.js GUI and core. There is a bunch of glue and the benefit of a reconciler to do a lot of the heavy lifting:
…/generate-code.ts

There still a lot of wrapper code and extra things going on, but you may find that useful.

Not sure, but you may find something useful an this abandoned project to create declarative Babylon.js in vue (I agree with you about declarative speed):
vite-babylonjs/src/App.vue at main · brianzinn/vite-babylonjs (github.com)

[MILIFIRE]

Looking forward to communicating with you.

[MILIFIRE]

Thank you for sharing your experience and insights! Since I have less experience with ts-morph, I will definitely take a look at the generate-code.ts file you mentioned. It seems like it could offer valuable insights into streamlining the component creation process. Additionally, that abandoned project for creating declarative Babylon.js in Vue looks intriguing; I will explore it further to see if there are any nuggets of wisdom to glean from it. Thanks again for sharing your projects with me. Currently, my idea is to gradually establish a component library for convenient and quick use in projects.

[mawa]

Just stumbled over this topic. Found it interesting/intriguing. Since I have been advocating for the BJS GUI since the beginning, all efforts to improve/enhance it already has my full support . To be honest, I’m not too much into VR so I cannot tell (or help) here.
In fact, I was just wondering why someone like @RaananW from the Team hasn’t kicked in yet, to either discard or encourage your research and efforts…

[Pryme8]

@MILIFIRE

WebGLGUI code development is outdated, resembling the era of JQuery. Using component-based development significantly enhances efficiency and reusability. Below is a code comparison:

I think this is a matter of opinion and depends on how your mind works.

buuut this does look pretty cool! Hopefully you are able to get full functionality on it going.

I’m not a big fan of Vue style components but a lot of people are so this might get some traction.

[brianzinn]

@mawa

all efforts to improve/enhance it already has my full support

my interpretation was that it wouldn’t change Babylon.js GUI at all, but just provide a different way to integrate it in a Vue project.

[mawa]

Yes, I guess so. But I have seen a number of posts in this forum from people considering using the BJS GUI in a vue project and at some point, quit (likely because it was too ‘cumbersome’ for them). If we could get these people back, I suppose that would be a good thing, wouldn’t it?

[brianzinn]

@mawa

If we could get these people back, I suppose that would be a good thing, wouldn’t it?

Yes, agree 100%. More options/examples across different development stacks is a good thing.

[RaananW]

What an awesome project

Though I don’t fully agree with

@MILIFIRE

WebGLGUI code development is outdated, resembling the era of JQuery.

I think many people do look for a different way of designing their UI. We try not to hook our users to a specific framework (in this case vue, if I understand correctly), but having this as an extension for users is an amazing addition. Especially in XR scenarios! Are you planning on documenting different scenarios? This could be a great addition to our documentation page in the community extensions section!

[MILIFIRE]

Thank you so much for your kind words

I apologize for my comment about “WebGLGUI code development being outdated, resembling the era of jQuery.” I regret any misunderstanding this might have caused.

I completely agree that many people are indeed looking for different ways to design their UIs. We certainly do not intend to lock our users into a specific framework (in this case, Vue, if I understand correctly). However, offering this as an extension is a fantastic addition, especially in XR scenarios!

Regarding documenting different scenarios, we would be very happy to work on that. It would be a great addition to the community extensions section of our documentation page!