WebXR Compatible HTML Texture

A demo of rendering HTML as a texture. Doesn’t use CSS, so it’s compatible with WebXR but is probably slower and compatible with less HTML content than the CSS-based Solution.

Demo renders a dat.gui / lil-gui chunk of HTML but isn’t interactive (yet) so you can’t actually click on anything. Code is largely lifted from three.js.

Any advice about how to do the lighting or materials differently so it doesn’t appear so dark would be appreciated. :slight_smile:

The resolution and speed would be greatly enhanced by using WebXR Layers, but currently that would only benefit Meta Quest users.


Please don’t read that as a sarcasm but I wonder if you would not be better suited with a GUI render?

At least from a perf standpoint

Nice! is that using a similar approach of the HTMLMesh from three? (read your text again - yes, it is the three.js code :slight_smile: )

What elements are supported? how performant are heavy pages? forms?

Good questions. Basic inputs (range, checkbox, etc) are supported. Text, images, and at least header tags. Not sure beyond that – if people have ideas for how to curate a list of tests I’d be happy to run them.

There are some timing tests in the code that I can enable too. There was a comment buried deep in three.js that they stop using PNG and start using JPEG for images >2048x2048 for performance reasons. Again, I’m open minded to ideas for how to test out performance (probably based on the range and quantity of elements listed above).

Deltakosh – not sure what you mean about a GUI render?

My ask is: for an UI related work with webxr, how does it compare with Babylon.js docs

Your question is “why do this instead of the existing BJS GUI options”? Here’s what the docs say about VR compatibility with BJS GUI flavors:

Comparison of GUI Options


  • Can’t be used for fullscreen/native VR

Babylon 2D GUI

  • Can’t be used for fullscreen/native VR

Babylon 3D GUI

  • Supports fullscreen/native VR

Basically, only 3D GUI supports VR. The 3D GUI docs spell out a bunch of ways to arrange and style buttons but not much else. Not sure if there are other 3D widgets I’ve overlooked, but taking the docs at face value this seems like a great compliment – having a collection of familiar HTML inputs and styling techniques seems like a very useful feature to have.

Of course that depends on getting inputs and eventing to work… off to try that out next. :wink:

1 Like

Oh we need to fix that doc (cc @PatrickRyan )

GUI is meant for XR:)

1 Like

2D GUI works well in XR. Our basic example has the 2D GUI color picker. However, only GUI attached to a mesh, or fullscreen with layers are supported

Oh, but i think the idea here is to be able to embed HTML, and less support for simple UI. How can you view an external website or an iframe in XR. This is currently not easily done.

Not sure this would work for a full website as imagined – it’s not going to interpret JavaScript which will immediately break functionality on lots of websites.

I’ll try out 2D GUI to see if that does what I was looking for. Thanks for the heads up. :slight_smile:

1 Like

Can we update the cons for the HTML UI to mention that using HtmlMesh extenson the 3D scene integration isn’t an issue, though it still doesn’t work in XR? I can take a stab and do a PR for docs if you want.

please do! I’ll merge

1 Like

After playing with 2D GUI it looks like it has a lot of great and performant features, but also isn’t as fully fleshed as many web frameworks. For example – no date picker; accordions / collapsable; menuing; tooltips; chips; etc.

I’m sure they can all be created but there isn’t a library that already exists for those components, right?

Proof of concept demo with interactive HTML:

Apparently there are some approaches that offer fairly broad ability to render HTML (with some cross-domain security limitations):

Rumor has it that these can render modern web component libraries like DaisyUI.


No, there isn’t. At least, not as far as I would know about :thinking:. But then, it’s as you mentioned: The GUI (although still young and perfectible - and continuously improved) is versatile enough to allow you to do everything you listed. Also note that (from my experience with BJS) delivering templates or components is not really something that’s in the DNA of the framework. What is however is that, for most of these, you can find similar samples, studies and playgrounds (starting with this forum). And then let’s face it, for all these things you mention, 90% of the time I find one (in any framework or app) it just doesn’t match 100% what I want to do anyway (except may be, the date picker - but it’s ez enough to do one). And if you want to share you version with us you are very welcome. I’m sure everyone here will gladly take the contribution :smiley:

This looks amazing!

I’ll tell you what my problem is with all of those libraries and implementations - A web renderer is a complex piece of software. a 45kb javascript library (minified, i know) won’t be able to cover all of the use-cases.
But it does look like they are working quite well! :slight_smile:

1 Like

I used a different approach - server-side rendering with selenium.
Demo and explanation here: https://www.youtube.com/watch?v=06VWBi8OJmQ&t=455s
Major issue that I’ve encountered is likely common with what you’re doing - we never know for sure when a web page is ready. I’ve tried different approaches, pretty much everything I found on stackoverflow, but every now and then I render a blank page.
I hate it when security gets in the way :slight_smile: Like, in order to prevent us from stealing youtube videos, we’re struggling with basic functionality (that’s what an iframe was supposed to do). I’m afraid that whatever we come up with to circumvent it, they’re going to figure out new way to break it again in new browser version.
Anyway, keep up the good work! I’ll gladly borrow it eventually, with thanks :wink:

1 Like

Correct. It is far more limited but more performant

PR submitted. The PG that I included for HTML Mesh is not as clean as most of your example PGs are. I could do a very minimalistic one that just shows the Babylon.JS site if that is preferred. Let me know what you think and I can adjust and update my PR.