Cant find rendering HTML/iframe in texture

[Amit_Sharma]

Hi i am unable to render HTML / IFrame in texture. Can you please help me here ?I am using this for now but not working?

mat.useAlphaFromDiffuseTexture = true;
var img = new Image();
img.src = “/textures/banner_0”+index+“.png”;
console.log(img.src);
img.onload = function() {
textureContext.drawImage(this,0, 0,this.width,this.height ,0,0 ,this.width-20,this.height-100 );
texture.update();
texture.drawText(ppname, 50, 250, font_aboutus, “white”, null, true, true);
var a = pexcept.match(/.{1,20}/g);
for(var i = 0 ; i < a.length ; i++ )
{
texture.drawText(a[i], 50, 300+i*40 , font1, “white”, null, true, true);
}

[JohnK]

Is this the sort of thing you are looking for Rendering an iframe on Babylon - Questions & Answers - HTML5 Game Devs Forum?

[Pryme8]

heh… that moment you click on a link wondering how someone did that and them whabam, you are the one who did that… jeeez o_O.

Maybe I need to start writing some of this crap down.

[Dad72]

I am also interested in this subject. I would like to be able to display a simple html page on a 3D plan like the videoTexture.
The idea is to have a web page with a scrool on a 3D plane.

Did you manage something @Pryme8 ?

I have an approach on this PG, but it’s just a “div” over the canvas that is transforming according to the distance/rotation of the camera. It’s not really attaching to a 3D plan. It’s not really what I’m looking for.

https://www.babylonjs-playground.com/#WADD47#50

It would be really great if Babylon in all its greatness could allow to do this a bit like the videoTexture (but instead of a video, a Web page)

I have an ongoing metaverse project and I really need this important feature for this kind of project.

Thank you

[Pryme8]

I did dabble in it a bit, I think the only progress I made was converting it to an SVG then making that the texture.

Which would work for displaying data, but no making it interactable.

Your solution so far is pretty amazing.

[Deltakosh]

@trevordev is pushing hard on the browsers to give us that option but so far it is a hard constraint that we cannot work around easily

[Dad72]

@Pryme8

I did dabble in it a bit, I think the only progress I made was converting it to an SVG then making that the texture.

Do you have an example PG? This could be an interim solution. What you put in your link above, does not have a page link, I did not understand your approach with SVG. Thanks in advance

@Pryme8

Your solution so far is pretty amazing.

This solution is not really good. It does not fit into the 3D world. It’s just a cheat.

@Deltakosh

Trevor is pushing hard on the browsers to give us that option

It would be really great if a solution could be found.
Thanks in advance @trevordev

[Pryme8]

If I can get the script to actually load this should work:

https://www.babylonjs-playground.com/#WNS2YE#2

[trevordev]

Yea, the issue is definitely at the browser level if you are trying to get content from another webpage page you don’t own.

See this issue:

Issue: DOM Overlays (for Phone AR support and more)

Which lead to this:

immersive-web/dom-overlays

But it will likely take a long time before something like this shows up in a browser so hacky workarounds are unfortunately the best option at this time.

DIV over canvas: Works pretty well on 2D screens but can’t present to a VR headset and it always needs to be in front of everything.

Server side webpage rendering: Works in 2D and in VR but is a lot slower, videos wont work and requires a server to exist somewhere.

Cross site browser plugin: With a chrome plugin you can inject JS onto your page and the iframe’s page and send messages back and forth so you can send the page data from the iframe to the parent and then re render the page from the parent using https://html2canvas.hertzen.com/ but this is also pretty slow and I’m not sure if this is against browser plugin TOS. It also doesn’t work very well for videos.

I even have a fork of firefox which disables CORS so I can pull data from an iframe directly but its very insecure. GitHub - TrevorDev/NiftyRealityBrowser: AR/VR first browser that supports multitasking

[Dad72]

Your @Pryme8 solution only works for an internal web element on the server. It does not work with a link from an external site.

Is it possible to create a simple image of an external web page as an alternative. Like with Babylon.CreateScreenShot from an external address and display it as a texture on a clickable plane that redirects to the website.
This could be a solution, but I do not know if such an image can be generated remotely.

Of Ideas @trevordev, @Deltakosh , @Pryme8 with this solution ? (is it possible ?)

[Pryme8]

Yes, if you have a PHP server you create a PHP page to do a CORS request and as long as the responding server allows cross origin requests with the correct header you use that page to grab the information.

Then in your scene you would need to do an AJAX call to the PHP page that actually does the CORS request and then push that to the html2canvas object.

Ive done that method with the PHP CORS thing for web scrappers and other externally driven data stuff…

You can also take a look at this: Heres your CORS load... now make it work - Questions & Answers - HTML5 Game Devs Forum and see if you can make that work with html content.

Let me see if I can find my old PHP scripts for you.

[Pryme8]

Hacking It Out: When CORS won’t let you be great - Netscape - Medium

This might help you.

[Dad72]

Yes, I have a PHP server local and dedicate. I usen PHP perfectly. I will study this idea.

Thank you Pryme

[Dad72]

@Pryme8

Let me see if I can find my old PHP scripts for you.

I want to have an example. thank you so much

[Pryme8]

@Dad72 could not find them bud… damn

It was from my web scraper days so its been quite while. Just look up CORS with PHP and how to do web scrapers then convert that to an AJAX call.

[Dad72]

Ok, no problem Pryme. Thank you

[Dad72]

I find this by search on google. It seems to do the job. It’s written in WebGL (With a webgl2 context), So I guess that can be achievable in BabylonJS ? @Deltakosh what do you think of what is done below? I do not know anything about WebGL code

I have tested, but error : gl.bindTexSource which seemed to work on a local version of Chronium. So bad. It seems really boring to achieve that goal. Some speak of a security problem, others say that it is not possible. Zute, I hit a wall with this thing.

toji/webgl-iframe/blob/master/test.html

---

I saw that TreeJS does this with : WebGLRenderer and CSS3DRenderer and it displays a web page on a planeGeometry :

Mixing HTML pages inside your WebGL - Learning Three.js

There is a demo here where we can see a web page in the 3D TreeJS world : Video Browser For LearningThree.js

But maybe it’s a fake, because on this demo, the link is not clickable. It may be just an image/texture.
I did not test directly with TreeJS, because I do not like this engine very much.

I may have to give up on this idea, I think. Or come back when it will be possible without doing multiple acrobatics and pirouette of all kinds. :sourire:

[Deltakosh]

It is not feasible today. CSSREnderer is a “light” version of three that does not use webgl so yeah you can render an iframe and then play with CSS transform but this is FAR from being what you need

[Pryme8]

Let me see what I can do.

[Pryme8]

@Dad72, cause I love you long time <3

Pryme8/website2GL

Now if you combined this with my mmmPie UI canvas engine, you could add hotspots and click callbacks on the canvas which if you did a conversion of the UV of the mesh to canvas space you could simulate the actual website interactions in the 3d scene (including scroll)

Now there are going to be some caveats that you have to fix, like a regEx scrape over the HTML context prior to injecting into the data div to correct image URLS and other things that might not be appearing correctly due to linkages.