MRTK and Babylon.js

After seeing a demo of a WebXR export of MRTK running in-browser, I suggested that what we really need is an effort to bring the MRTK to Babylon.js. @PirateJC suggest I post this here.

The current Babylon.js GUI and 3D GUI classes are quite nice; I used them for my 3DUI class in the spring, and plan on using Babylon again in the fall. BUT, while they share a lot of visual and functional similarities to MRTK, they are not really as complete or elaborate (see GitHub - microsoft/MixedRealityToolkit-Unity: Mixed Reality Toolkit (MRTK) provides a set of components and features to accelerate cross-platform MR app development in Unity.).

Since both are being driven out of Microsoft, and we can even run Babylon.js on VR and AR (including Hololens2!) devices, it would make a lot of sense to have a version of MRTK that is implemented directly in Babylon. MRTK is MIT licensed as well.

The big challenge, of course, is that much of it would need to be rewritten, and once that happens, keeping it up-to-date feels like a full time job.

But, I’m curious if others see the value here.


We kickstarted the port of some of the controls but as you said it is a huge investment to maintain all of them but we are gladly encouraging any external contribs.

@RaananW will also love this one !!!


I suspect most of us are in the same boat; would love to see it, but can’t afford the time/resources. I mean, if I didn’t have anything else to do, I’d love to do it. :smile: but that’s a massive “if” …ha. I suspect this might be too much for students to do.


The think is there are more and more stuff happening around the XR area so I have a pretty good feeling this will quickly become a requirement and this will be good for everybody :slight_smile:

1 Like

It would be interesting if we could divide and conquer the problem. Perhaps get different folks contributing different controls. Rather than putting a lot of work on one person, a bunch of people all do a small amount of work? Just an idea.

It would be great if we could divide and conquer, yep!

Might there be a way for someone deep into Babylon to help kickstart it? I can imagine there are many people for whom just starting this is overwhelming.

1 Like

Yeah the dev team can definitely look into it.

Looking at @RaananW to lend us his opinion/input on the idea.

As with everyone else, the only thing I’m missing is time. You don’t have to convince me it’s needed, I totally agree :blush:
We have some implemented already. The bounding box, holographic buttons, slate, but we have different names for them (and maybe a slightly different design). They are also not only used for mixed reality, which made the interaction with them a bit different.

I think it’ll be the best of we decide which elements are the most important ones and which are optional, and most importantly which ones are missing and take it from there


What about going a step further and adding this to the babylonjs unity export?