HTML in 3D Space Part 2 - BJS Way to Matrix Transform

Hi again,

Quick summary, I’ve been playing around with getting an HTML view to show up with correct Z-ordering in a BJS scene. I’m trying to do this with CSS 3D transforms so that eventually it’s interactive.

Last time with help from the forum I got the right integrated “hole” punched through the 3d scene that allows objects to correctly cover in front and others to correctly fall behind the hole in appropriate and complex ways: (Forum Link.) Awesome!

Next on my todo list was arguably the hard part, port the relevant parts of a ThreeJS CSS renderer to Babylon so that I can create a matrix3d css transform to match the BJS camera and plane I’m trying to map to. I’ve started down that path with this playground: https://playground.babylonjs.com/#5CGWLN#4

I started to port the relevant parts of CSS3DRenderer, but am finding more differences than I expected. First problem figuring out why the BJS Matrix returned from the camera appeared to be a 3x3 instead of a 4x4? Then I ran into a copyPosition function from Three that doesn’t seem to have an equivalent in BJS?

I can continue down the rabbit hole and see if I can port each of these pieces correctly, but I also wondered if this is something common and if there’s a better BJS way of doing it than how I’m approaching it?

So ultimately my question is: given my goal, am I approaching this in a way the community would recommend? My follow up question is, am I reinventing the wheel for BJS? Is there something else out there doing this I just didn’t find?

Thanks!

1 Like

Hey!

so camera.getProjectionmatrix() is definitely a 4x4 :slight_smile:

The main difference is that we are working in left handed mode by default

(you can get it to work with right handed with scene.useRightHandedSystem = true)

On your question: I see no reason why it should not work and no you are not reinventing the wheel :smiley:

1 Like

Thanks, I was experimenting with the projection and had somehow converted it to a 3x3, thanks for pointing out that it is returned originally 4x4! I think that’s why I wanted to get a gut check on the forum, I’m not very strong on using matrices and figuring out projections for visual effects - so I’m trying to port without necessarily understanding all the steps in the port. It’s easy to make something look right independently but to make it work exactly like BJS’s projections will be… an interesting challenge, I’m sure!

Working on it on the side here and there but will post back if I get progress for anyone interested.

Thanks for the inspiration @tinman I’ve built a full port of CSS3DRenderer. You can see it here https://playground.babylonjs.com/#1DX9UE#2

7 Likes

Wow, very impressive, thanks for sharing @ozRocker!

For some reason I’m getting the error “Can’t find CSS3DRenderer” when I try your sandbox, even though it’s clearly a class in your code. Any ideas?

Weird. I’m not experiencing that. What browser / OS / device are you using?

Using Safari, that might explain it. macOS Big Sur

Wow, that is awesome.

Thanks for sharing that

@ozRocker, I’d love to use this CSS3DRenderer code in my projects. Are you willing to license it under a permissive license like MIT-0? Can you reply to me with confirmation (if you’re comfortable with that)?

The CSS3DRenderer is based off this https://github.com/mrdoob/three.js/blob/master/examples/jsm/renderers/CSS3DRenderer.js so I’d say it has the same license restrictions for three.js

hi @Kris - Welcome to the forum!

Drei Html component from pmndrs was ported to babylonjs via react-babylonjs and has an MIT license. You can see a port of it here - not sure of the differences entirely with CSS3DRenderer.js, but at a minimum you may find some snippets useful:
react-babylonjs/Html.tsx at master · brianzinn/react-babylonjs (github.com)

Cheers.
edit: wrong link posted!

Hi, is it possible to support scene.useRightHandedSystem = true as well?

@ozRocker
Is it possible to get Input passed into the HTML page from your Bayblon CSS3DRenderer port? I noticed input works as expected in the original Three.js example.

Edit: I located this other thread, which has working examples of input. It looks like there’s a limitation of a single iframe and this technique won’t work in VR, but very cool!