Youtube videos on a mesh (port of CSS3DRenderer.js)

ozRocker · August 10, 2021, 10:23am

This will not work in VR 'cos its using CSS which is not VR compatible

labris · May 19, 2022, 3:03am

I found this Typescript example - https://playground.babylonjs.com/#L05ZHQ#1 - but seems it doesn’t work correctly, probably somebody has a better example of CSS3DRenderer with Typescript?

jayypluss · June 7, 2022, 3:18am

I’ve made this TypeScript adaptation Babylon.js Playground with a bug fix for the ‘css-container’ div that was being duplicated sometimes.

The original code was the https://playground.babylonjs.com/#1DX9UE#2 from the 1st post.

terransage · June 8, 2022, 3:42am

@ozRocker and @bnolan: Oh man, this is fantastic! The project I’m working on relies heavily on embedded Vimeo video as well as YouTube, but I display them only in the 2D (Construct 3) parts of my game. I use Babylonjs for the 3D parts of the game and was hoping there would be an effective 3D way of displaying the embeds. Thanks for sharing it!

Great song, by the way! Never heard of The Prodigy before.

Man, I just realized this is an old post. I did searches for “babylonjs embedded video” months ago, but somehow missed this.

terransage · June 8, 2022, 3:45am

Lol, thanks for the bug fix. I noticed that the screen duplicated when I “re-ran” the original script.

RaananW · July 20, 2022, 1:33pm

Not sure why, but it doesn’t seem to work for me on edge (chromium). The background changes (single color), and the audio is playing, but i can’t see the video. Is that a known issue?

11128 · July 21, 2022, 5:21am

maybe, seems to be related to message :
Tracking Prevention blocked access to storage for https://www.youtube.com/.
i also have this problem

DerrickBarra · July 21, 2022, 2:05pm

I’m also noticing the same issue, whereas a few months ago this was working. Tested in Edge (Chromium)

DerrickBarra · July 26, 2022, 7:18pm

Decided to do some more testing. The Fish tank 3D iframe demo posted earlier works on Chrome, Brave, and Firefox, but fails to render properly in Edge.

With the YouTube tests, audio plays back on Edge but visual rendering is bugged.

Deni_Klen · August 1, 2022, 7:43am

Edge does not have support for elements inside WebGl maybe that is the reason. Works fine on Chrome for me.

RaananW · August 1, 2022, 11:54am

Edge is chromium and should render just like chrome. If it renders incorrectly, it is (probably) something in the code. As in the example here. Here, this is working in edge:

CSS3DRenderer | Babylon.js Playground (babylonjs.com)

The code included IE detection (which i assume was done for IE11’s wonderful rendering abilities). But it used ua-sniffing, reading Edge from the ua string. That caused the incorrect rendering. Removing this fixed it (see the playground from before).

NeeeeT · August 10, 2022, 7:43am

very nice and thanks.
its the greatest solution and working like a charm on babylonjs 5.18.0 version.
any others got problems with latest version( > 4.2.1) .
ex. rendering grounp id / layer, not clickable…

DerrickBarra · August 22, 2022, 10:01pm

Has anyone modified the demo yet so rotation works properly? I noticed there’s something wrong with the Quaternion → Matrix4 conversion that is breaking rotation.

EDIT: I’ve opened a separate thread to request aid on this issue, but for anyone following this thread check out the demo link below.

Try changing the rotation of the ‘no name’ node in the inspector to rotate the plane.

–WORKING–
RotateX
RotateY

–NOT WORKING–
Rotate X & Y at the same time
RotateZ

DerrickBarra · August 24, 2022, 6:43pm

So I figured it out, and the answer is… you can’t.

I found this great Matrix4 visualizer, and it turns out you can’t represent a full 3D scene in CSS3D (it’s can only skew the perspective on a flat image). So the limitations to the rotation system in the playground I posted are the limits of the CSS3D technique.

EDIT:
After further research, I think the limitations are still possible to workaround. Check out this CSS3D demo with threejs (Github Link), it has correct rotations for all three Euler angles (X,Y,Z) and can rotate all axis at the same time.

The CSS3DRenderer script in the playground I posted earlier was a port of this project from the Babylonjs community, so it’s entirely possible that there’s a simple problem with the code port that’s preventing it from operating equivalently. I’ll look into seeing if there’s a noticeable difference next.

DerrickBarra · September 6, 2022, 9:52pm

I found a CSS3D demo that shows that if you used the rotateX() rotateY() rotateZ() CSS3D transform styles, you can indeed achieve full three-dimensional rotation.

So the solution involves converting from setting the Matrix4 values of the camera and plane mesh directly to instead using their rotateX/Y/Z CSS3D style string functions.

If anyone wants to help convert the current latest Babylonjs playground, be my guest. I’m attempting it on my side.

Here’s where my headspace is right now on this problem…

My assumption is that if we step away from directly setting the Matrix4 for Postion/Rotation/Scale, then this will make it easier to debug.
I’ve been reading about how we can convert from world units of Threejs position to the 0-1 normalized units of the DOM for positioning objects. So something similar should be possible for Babylonjs.
However, the CSS3D demo I linked to doesn’t have to deal with a parent->child relationship. So converting the camera’s transform node to positionX/Y/Z scaleX/Y rotationX/Y/Z style functions may or may not work out.

ozRocker · September 8, 2022, 1:55am

I have updated this so it’ll take into account the mesh rotation https://playground.babylonjs.com/#1DX9UE#523

DerrickBarra · September 8, 2022, 7:11pm

Awesome work @ozRocker! Having rotation fully working is a huge win.

Next up, I noticed that the mouse input in the demo will be consumed by the iframe and occasionally the logic to give input control back to the parent webpage doesn’t fire. I’ll be tackling that bug next.

DerrickBarra · September 12, 2022, 8:27pm

I uploaded a new version of the snippet that shows using a large invisible background sphere mesh to detect focus being lost, which seems to work better than detecting the mouseout event.

In other notes, I found a bug with Chromium based browsers. Mouse drag events have an incorrect X/Y position (it seems to always start at the top-left). This seems to only occur with CSS3D iframes (2D iframes don’t have this bug).

Firefox browsers have another really bad bug, mouse input within CSS3D iframes doesn’t work at all when the iframe src points to a url with a different origin. From what I found this is due to a security limitation with the browser.

DerrickBarra · September 17, 2022, 2:08am

So a couple of new notes…

The iframe-bg-sphere I added for improving detection of onmouseout events has a limit of around 100 in-engine units in scale due to Babylon’s raycasting system, which makes it a useful idea, but with limitations. I would recommend sticking with onmouseout event detection as it is and perhaps adding a list of meshes you want to detect that will disable the onfocus boolean.
The PointerEvent in the playground uses event.offsetX and event.offsetY for the mouse position. However depending on your own use case you should consider using event.x and event.y or alternatively event.pageX and event.pageY, as all of these different X/Y coordinates are useful in different scenarios. In my own repo I added an enum option when creating an iframe for selecting which mouse event coordinates to use for the Babylonjs raycast.

[EDIT]
Hey everyone!

After looking around, I found that the CSS3D-iFrame input position bug on Chrome has already been officially reported. However the issue isn’t get much attention (reported on June 21, last comment from a Chromium contributor was on June 30th).

I would appreciate it if you could favorite the bug in the Chromium tracker and leave a comment saying you would also like to see the bug resolved. Thanks!

nicoraf · December 13, 2022, 11:02pm

Hi @ozRocker , I have one question regarding this implementation of CSS3DRenderer
I notice that it has some hardcoded variables

			const youtubeVideoWidth = 4.8
			const youtubeVideoHeight = 3.6

I don’t see in the ThreeJS implementation similar constants, or there are ?

github.com

mrdoob/three.js/blob/master/examples/jsm/renderers/CSS3DRenderer.js

import {
	Matrix4,
	Object3D,
	Quaternion,
	Vector3
} from 'three';

/**
 * Based on http://www.emagix.net/academic/mscs-project/item/camera-sync-with-css3-and-webgl-threejs
 */

const _position = new Vector3();
const _quaternion = new Quaternion();
const _scale = new Vector3();

class CSS3DObject extends Object3D {

	constructor( element = document.createElement( 'div' ) ) {

		super();

This file has been truncated. show original

I am curious why they are needed for the Babylon implementation, and if that can limit the aspect ratio ?

Thanks

Topic		Replies	Views
Guys something i need to do in BABYLONJS? Questions	4	663	July 27, 2020
Trying to get a simple instance of a YouTube video on a wall in my scene Questions	3	1086	August 6, 2021
Trying to put a YouTube stream iFrame into little world Questions	3	1173	August 10, 2021
Moving from THREEjs to BABYLONjs Demos and projects	23	2713	June 9, 2023
Rainbow spiral, strange 3D presentation Demos and projects	3	501	April 26, 2022

Youtube videos on a mesh (port of CSS3DRenderer.js)

Related Topics