Possible to use Babylon to render HTML divs and CSS with PBR materials and shadows?

Hi all, I’ve been looking for a way to have PBR materials, lighting and shadows in my web apps. Kind of like this screenshot at the top of this article from Fluent Design by Microsoft.

It seems like the first and most basic step would be to apply a PBR material to a rectangle that represents a div. E.g. A rectangle made of wood. But how would I go about integrating ordinary CSS and HTML with Babylon Engine?

Welcome to the community @TheAussieStew (and fellow Aussie :wink: )

If you search for rendering HTML to texture or variations thereof, there are a few discussions about this already.

My understanding is that what you’re asking isn’t possible. You can fake it with dynamically positioned iframes or html2canvas but rendering fully responsive and interactive HTML & CSS as a dynamic texture applied to a PBR material is beyond what’s currently feasible.

You might be better off trying to do what you need to do using Babylon GUI or layered custom 3D elements.

3 Likes

Thank you! Sydney-sider here :slight_smile:

Could you elaborate on the technique of dynamically positioned iframes?

I stumbled across this feature released in Edge, it visualises the DOM in 3D using bjs. I think this would prove to be a good foundation to start with. Definitely not interactive though.

I haven’t tried it myself and given the screenshot of what you’re aiming to achieve I don’t think it’ll achieve exactly that, as it won’t receive real-time lighting etc.

But I think the general idea is you have a rectangle in the 3D scene which represents the surface for the HTML page, then convert the 4 corners into screen coordinates or somehow convert the rectangle’s transformations into CSS 3D transformation of the iframe containing the HTML. Then when camera is moved, update the CSS 3D transformation to match.