Youtube videos on a mesh (port of CSS3DRenderer.js)

Do these work on the oculus quest browser? I tried loading some of the playgrounds in this thread in the Oculus Quest 1 browser and I hear the youtube video playing music but I just see a black texture on the mesh and not streaming video content.

This will not work in VR 'cos its using CSS which is not VR compatible

I found this Typescript example - https://playground.babylonjs.com/#L05ZHQ#1 - but seems it doesn’t work correctly, probably somebody has a better example of CSS3DRenderer with Typescript?

1 Like

I’ve made this TypeScript adaptation Babylon.js Playground with a bug fix for the ‘css-container’ div that was being duplicated sometimes.

The original code was the https://playground.babylonjs.com/#1DX9UE#2 from the 1st post.

3 Likes

@ozRocker and @bnolan: Oh man, this is fantastic! The project I’m working on relies heavily on embedded Vimeo video as well as YouTube, but I display them only in the 2D (Construct 3) parts of my game. I use Babylonjs for the 3D parts of the game and was hoping there would be an effective 3D way of displaying the embeds. Thanks for sharing it!

Great song, by the way! Never heard of The Prodigy before.

Man, I just realized this is an old post. I did searches for “babylonjs embedded video” months ago, but somehow missed this.

1 Like

Lol, thanks for the bug fix. I noticed that the screen duplicated when I “re-ran” the original script.

2 Likes

Not sure why, but it doesn’t seem to work for me on edge (chromium). The background changes (single color), and the audio is playing, but i can’t see the video. Is that a known issue?

maybe, seems to be related to message :
Tracking Prevention blocked access to storage for https://www.youtube.com/.
i also have this problem

I’m also noticing the same issue, whereas a few months ago this was working. Tested in Edge (Chromium)

Decided to do some more testing. The Fish tank 3D iframe demo posted earlier works on Chrome, Brave, and Firefox, but fails to render properly in Edge.

With the YouTube tests, audio plays back on Edge but visual rendering is bugged.


Edge does not have support for elements inside WebGl maybe that is the reason. Works fine on Chrome for me.

Edge is chromium and should render just like chrome. If it renders incorrectly, it is (probably) something in the code. As in the example here. Here, this is working in edge:

CSS3DRenderer | Babylon.js Playground (babylonjs.com)

The code included IE detection (which i assume was done for IE11’s wonderful rendering abilities). But it used ua-sniffing, reading Edge from the ua string. That caused the incorrect rendering. Removing this fixed it (see the playground from before).

4 Likes

very nice and thanks. :slight_smile:
its the greatest solution and working like a charm on babylonjs 5.18.0 version.
any others got problems with latest version( > 4.2.1) .
ex. rendering grounp id / layer, not clickable…

Has anyone modified the demo yet so rotation works properly? I noticed there’s something wrong with the Quaternion → Matrix4 conversion that is breaking rotation.

EDIT: I’ve opened a separate thread to request aid on this issue, but for anyone following this thread check out the demo link below.

Try changing the rotation of the ‘no name’ node in the inspector to rotate the plane.

–WORKING–
RotateX
RotateY

–NOT WORKING–
Rotate X & Y at the same time
RotateZ

So I figured it out, and the answer is… you can’t.

I found this great Matrix4 visualizer, and it turns out you can’t represent a full 3D scene in CSS3D (it’s can only skew the perspective on a flat image). So the limitations to the rotation system in the playground I posted are the limits of the CSS3D technique.

EDIT:
After further research, I think the limitations are still possible to workaround. Check out this CSS3D demo with threejs (Github Link), it has correct rotations for all three Euler angles (X,Y,Z) and can rotate all axis at the same time.

The CSS3DRenderer script in the playground I posted earlier was a port of this project from the Babylonjs community, so it’s entirely possible that there’s a simple problem with the code port that’s preventing it from operating equivalently. I’ll look into seeing if there’s a noticeable difference next.

1 Like

I found a CSS3D demo that shows that if you used the rotateX() rotateY() rotateZ() CSS3D transform styles, you can indeed achieve full three-dimensional rotation.

So the solution involves converting from setting the Matrix4 values of the camera and plane mesh directly to instead using their rotateX/Y/Z CSS3D style string functions.

If anyone wants to help convert the current latest Babylonjs playground, be my guest. I’m attempting it on my side.

Here’s where my headspace is right now on this problem…

  • My assumption is that if we step away from directly setting the Matrix4 for Postion/Rotation/Scale, then this will make it easier to debug.

  • I’ve been reading about how we can convert from world units of Threejs position to the 0-1 normalized units of the DOM for positioning objects. So something similar should be possible for Babylonjs.

  • However, the CSS3D demo I linked to doesn’t have to deal with a parent->child relationship. So converting the camera’s transform node to positionX/Y/Z scaleX/Y rotationX/Y/Z style functions may or may not work out.

1 Like

I have updated this so it’ll take into account the mesh rotation https://playground.babylonjs.com/#1DX9UE#523

4 Likes

Awesome work @ozRocker! Having rotation fully working is a huge win.

Next up, I noticed that the mouse input in the demo will be consumed by the iframe and occasionally the logic to give input control back to the parent webpage doesn’t fire. I’ll be tackling that bug next.

2 Likes

I uploaded a new version of the snippet that shows using a large invisible background sphere mesh to detect focus being lost, which seems to work better than detecting the mouseout event.

In other notes, I found a bug with Chromium based browsers. Mouse drag events have an incorrect X/Y position (it seems to always start at the top-left). This seems to only occur with CSS3D iframes (2D iframes don’t have this bug).

Firefox browsers have another really bad bug, mouse input within CSS3D iframes doesn’t work at all when the iframe src points to a url with a different origin. From what I found this is due to a security limitation with the browser.

1 Like

So a couple of new notes…

  • The iframe-bg-sphere I added for improving detection of onmouseout events has a limit of around 100 in-engine units in scale due to Babylon’s raycasting system, which makes it a useful idea, but with limitations. I would recommend sticking with onmouseout event detection as it is and perhaps adding a list of meshes you want to detect that will disable the onfocus boolean.

  • The PointerEvent in the playground uses event.offsetX and event.offsetY for the mouse position. However depending on your own use case you should consider using event.x and event.y or alternatively event.pageX and event.pageY, as all of these different X/Y coordinates are useful in different scenarios. In my own repo I added an enum option when creating an iframe for selecting which mouse event coordinates to use for the Babylonjs raycast.

[EDIT]
Hey everyone!

After looking around, I found that the CSS3D-iFrame input position bug on Chrome has already been officially reported. However the issue isn’t get much attention (reported on June 21, last comment from a Chromium contributor was on June 30th).

I would appreciate it if you could favorite the bug in the Chromium tracker and leave a comment saying you would also like to see the bug resolved. Thanks!

3 Likes