Enable pointer events for the YouTube div when you hover over it, then you can click on the YouTube buttons. Disable pointer events on leave.
Edit: I mean set canvas { pointer-events: none } when hovering over the YouTube divs. You’d need to code this in Javascript
There is no need for this event manipulation in the threejs example, but maybe BabylonJS handles this a bit different. The only ‘hack’ needed is when a 3D iframe covers another, you have to manipulate the z-index to correctly show the one that is in front.
Yes, its different because in the ThreeJS examples they are not interesting with 3D objects. If you do it like that you can have z-index in front for the YouTube containers. The BabylonJS implementation can interest with 3D mesh
Do you perhaps mean intersecting? In the Space-X example above you can actually drag the moon to intersect with some of the frames.
It does however becomes difficult when the moon is dragged in front of a frame and you try to redrag it: then the ray casting stops working.
My point is that again that probably the same should be possible using BabylonJS, but I have also see it stop working with ThreeJS, still not always sure why.
When I have time I’m going to investigate this further and see how I can improve it. Also needs to be a way to add multiple YouTube videos to the scene. Thank for @awonnink for testing this
Although it seems counter-intuitive, it might be that this is enough to do the trick (works for me in ThreejS, style can also instead be applied to the canvas):
<style>
body {
pointer-events: none;
}
</style>
I really like the possibility to have these active Html panels in 3D!
That can work but you would need to only apply body { pointer-events: none; } only when mouse is hovering over a YouTube div. If its permanent then you can’t interact with the 3D canvas
I’ve inspected the SpaceX site and it looks like you cannot directly interact with the canvas. They are roughly translating drag movements on the screen into 3D movements, which is why when you drag the moon it is not accurate, because you are not directly dragging the moon mesh.
So their workaround is to have no canvas interaction at all but use some other tricks to move the moon object around.
Edit: you can prove this by going to the canvas element and adding “pointer-events: none” and see that it still behaves the same way
Just to be clear, I created this Space-X scene. There are no tricks for dragging the moon, just a dragging implementation that doesn’t exactly follow the mouse position. The interaction works with normal ray-casting. I chose to try this with ThreeJS instead of BabylonJS because of the early examples I found. Only a few days ago I found your implementation for Babylon (I would like to have this available in SharePoint Spaces).
Raycasting from the screen point yes? That’s a good workaround. You could even apply that to camera movement if you also want orbit controls.
Just the normal threejs code:
var mouse = new THREE.Vector2();
mouse.x = (mousePos.x / window.innerWidth) * 2 - 1;
mouse.y = - (mousePos.y / window.innerHeight) * 2 + 1;
raycaster.setFromCamera(mouse, camera);
var intersects = raycaster.intersectObjects(allObjects);If you want to use that technique for Babylon.js you can use:
var pickResult = scene.pick(screenX, screenY, function(mesh)
{
return mesh.isPickable && mesh.isEnabled();
});
var mesh = pickResult.pickedMesh;Not sure if that is an improvement over the BABYLON.ActionManager.
ActionManager needs pointer events on canvas. If you want to go from screen to world (workaround for not directly manipulating canvas) you have to use scene picking
If you’re able to get it working, I think it opens up some nice extra opportunities. For me it works not only with YouTube movies or Html pages, but also like this example with a 3d scene within a 3d scene.
I implemented switching pointer-events on/off for the body here https://playground.babylonjs.com/#1DX9UE#183 Please give this a try and see if it works properly
Wow, yes! Seems to have complete interaction with the web page:
O, and look at this one: CSS3DRenderer | Babylon.js Playground
Great! Looks like its working.
I need to clean up the code some time. It cannot handle more than one iframe in the scene which is a bad limitation, but I can fix this
One of the fun things this can do is having the content of the iFrame act on the 3d environment.
Any idea whether this CSS3D should work in VR? I did an attempt in ThreeJS, but not successful so far.
Do these work on the oculus quest browser? I tried loading some of the playgrounds in this thread in the Oculus Quest 1 browser and I hear the youtube video playing music but I just see a black texture on the mesh and not streaming video content.
