Cant find rendering HTML/iframe in texture

This sounds interesting. I will look at this ready.
Thank you very much for spending time for me. I appreciate your help.

Any time buddy! That’s what I am here for.

I tested with several site. Some site does not pass. I do not really know why.
French site, with accents returns one < ? > in a triangle. There is an encoding problem but I do not know where.
Otherwise the approach is quite interesting. :smiley:

There is some thing that I did not quite understand, the translation does not seem correct to me.
I do not know what means mmmPie UI also. :kissing:

I did not understand also what you mean by converting the UV of the mesh into a working space. I am a lost here.

There too, you lost me. I did not understand to correct some warning. (can be the translation that to be messed up)

It’s really a first for me to do this kind of thing and I have a hard time understanding how it works. Your example makes me understand something, but I do not know how it works, so it’s hard to understand what I’m quoting above.

Thanks again Pryme

Basically saying you need a way to parse user mouse clicks on the mesh, and get them to “do something” on the page.

I bet you could actually come up with a system that you take a mouse click from the meshes UV coordinates (assuming is a rectangle you are showing this in) and then convert that into pxl coordinates. ID the element at those coordinates on the injected webpage (you will have to calculates offsets from scrolling independently) and then simulate the clicks. If you are in control of the context that is being presented it would be best aka a non external website.

You could use this to have html forms and other things embedded in a game I guess if fully worked out.

OK, I understand better. This seems really complex to do.

The idea was for each user of metaverse I built, he could put the link of his website to promote it in his world built in 3D. So I can not use an internal web page server and I do not know the structure of the site inject.

This makes this task complex. I do not think I’ll go further with this idea. So I will just create a link on 3D objects that will redirect to the users’ website. It will be an easier solution to achieve.

In any case, thank you Pryme, it’s very nice. Maybe I would find another use with the function that you have offered me that remains very interesting.

1 Like

Try to see if you can fix the images that are on the pages first.

Do a regex scan over the page and replace the srcs of stuff that are set to non-specific locations.
Let me see if I can whip that up really quick.

1 Like

regexp-munge or XSL-transform “node-gathering”… won’t collect the x and y coords of WHERE that link will be flow-located in browser canvas. You need some measurements for your “areaMap/isMap” image, right? lots of 'em. Click zones and pic zones. :slight_smile:

How do you gather the coordinates of where a link happens on a rendered webpage… ie. origin 23.5% .left, and 61.9% .top… and how wide and tall is the link/button/image? (click zone range/extends) hmm. :slight_smile:

I deleted my post about the html-to-gui2d converter… but I still stand firmly behind it… I think. heh.

1 Like

http://html2canvas.hertzen.com/proxy/

and update the getWebsite.php to:

<?php
$str_json = file_get_contents('php://input'); 
$data = json_decode($str_json, true);
//echo file_get_contents($data['url']);

$content = file_get_contents($data['url']);

preg_match('/<img[^>]+src="([^">]+)"/', $content, $matches, PREG_OFFSET_CAPTURE);

$offsetTill = 0;

for($i=1; $i<count($matches); $i+=2){
	$str = $matches[$i][0];
	$idx = $matches[$i][1];
	$content = substr_replace($content, $data['url'], $idx+$offsetTill, 0);
	$offsetTill+=count($data['url']);
}

echo $content;
?>

Does not fix it on the texture but gets them in the databox at least.

1 Like

Hello and apologies for the extremely noob post, but i can’t seem to figure out how to implement an iframe simply as a fullscreen overlay over the scene render. I tried to configure the PG by @Dad72 on https://www.babylonjs-playground.com/#WADD47#50 but no luck whatsoever.

Would greatly appreciate your help !

I am not the author of this PG.
What do you want to configure?
You can modify the URL line 25 to display another html page
the call ifram line 36
There is not much to configure.

1 Like

hey i start that in old forum
what configuration you need?

https://www.babylonjs-playground.com/#1BUXYF#23

Hi, i made this with threejs Hydra + Threejs - YouTube webgl+css3d, is there any updates on this, on babylon.js? in that example the iframes are loaded as holes through webgl, there is no overlapping, they are fully integrated on the scene.

@Pryme8 Hello chaps!

I have stumbled upon this dusscussion after looking for the same, I have used A-Frame in the past and am now looking to move accross to Babylon as it has advantages over ThreeJS that A-Frame uses.

In my A-Frame projects I used a-frame-htmlembed and I have been trying to port it to TypeScript and Babylon using DynamicTexture.
The main class that does the work seems to be htmlCanvas.js so in a Babylon port we would have HtmlTexture.
So I guess the constructor would be:

constructor(content: string | HTMLElement, with: number, height: number)

Or

constructor({
  content: string | HTMLElement,
  ...
}, scene) // Other properties?

One important and tricky thing will be getting mouse and perhaps even keyboard events working, you would need to use Vector3.Project to project the world units into pixels inside the Scene.OnPointerDown (et al) events.
But it is doable.

Perhaps you chaps will be better than me… I only started using Babylon 4 weeks ago, so I don’t have much exp.
What do you think?

Sounds promising, I don’t see why you could not do that honestly.

You might need to come up with a “hotspot” system to track what points are actually valid interaction areas, but other then that I don’t see much problem with your idea.

Just want to share with you my HtmlTextue experiment on StackBlitz.
I am using Html2Canvas which seems to do the job and is a well maintained project so I am happy with using that.
Need to do some tweaking scale to the mesh perhaps.
Still more work to do.

Note the mouse wheel (zoom) and pan are not working on StackBlitz example.

Update

I am experimenting first with mousedown, I cam consruct and dispatch the MouseEvent pretty easy but I cannot get the coordinates quite right.

I have created an absolute 5X5 px div, you can see here where I click on the pane mesh, but in the HTML element space the div is positioned wrongly.

I wonder if someone might take a look and be able to help or offer ideas.

Thanks

2 Likes

This is pretty interesting. This would be awesome if it could support HTML based menus inside VR. Does it work with other interactions like scrolling or text input?

Adding @RaananW

AR does support it using the DOM Overlays. There is no plan (that i know of) to support this in VR, though it will be a nice feature IMO.

1 Like

I’ve gotten simulated mouse actions from taking the uv location of the pointer on the mesh then converting that to the pixel location on the HTML page.

We were able to get an entire react menu rendering on a plane in the scene that had all it’s HTML/css animations working.

Now the problem was that it was not efficient and did not work in a real life project but we where passing the html data to a texture each frame and that was intensive.

Wow just realize this was a Necro post… The forums had pinged me about this thread so I thought someone brought it back up… Wow smh my bad.

2 Likes

Well, to me, it simply shows how much dedicated you are. :innocent: Replying to an over two years old topic is not something you can find in just every community forum :grin:
Have a great weekend :sunglasses: :sun_with_face:

2 Likes