Using WebGL API directly to draw into a babylon RenderTargetTexture

Hi @sebavan,
What is the right way to use WebGL calls to update/draw into an existing babylon RenderTargetTexture? I have an external WASM library (custom version of Skia) that I want to use to render into an existing RenderTargetTexture. I started out thinking that I could use the cutomRenderFunction with the hope that the gl.framebufferTextureLayer(...) would be applied and any gl commands I used would be executed against that texture.

What I found was that the customRenderFunction isn’t being called, so I must not have something setup correctly to begin with.

All that said, if you understand what I am driving at, do you have a recommended workflow for this?

Thanks in advance!
jw

This is quite tricky as you would break most of the cached states from the engine. I would advice to share the WebGL context (initializing engine from the shared one so that on every frame you render to your own FB bound to a webgl texture) then you could call into engine.wipeCaches(true) and render into babylon using this webglTexture wrapped into a ThinTexture for instance.

Interesting idea. You will need to help me along a little bit here. As I understand it ThinTexture's constructor wants an InternalTexture but if it doesn’t have one I assume it will create one at some point later in the cycle (not something we can work with in this context). From the constructor of the InternalTexture it looks like it will create a WebGLTexture that is wrapped in a WebGLHardwareTexture. How would I go about wrapping the existing WebGLTexture? Create an instance of InternalTexture and then call dispose() and then set its _hardwareTexture property to new WebGLHarwareTexture(myWebGLTexture, sharedGL)?

How would I then assign that to a material on a mesh? Seems like the material classes are expecting BaseTexture and so this new ThinTexture won’t work.

Thanks again for your time and comments!
jw

Yup you create an internalTexture, with delayAllocation set to true to not create the webgl one.

Then you would assign _hardwareTexture to a new WebGLHardwareTexture where you pass in your custom texture.

Finally you could create a Texture without url and then set texture._texture to your internalTexture from before.

Also you ll need to set the internalTexture.isReady flag to true.

This should do the trick :slight_smile:

1 Like

Okay, that sounds reasonable, what about assigning that texture to a material? Given those want something of the type BaseMaterial and ThinMaterial isn’t one of those.

jw

You would need to create a material like let s say a pbr Material set to unlit or a Standard Material with lighting disabled and place the texture in respectively the albedoTexture or diffuseTexture slot to see it.

The best option here might be to create your own dead simple material from a shaderMaterial or NME to only display the texture.

I feel like I am missing something here. Here is what I think we have been talking about in code form:

const eng = new ThinEngine(sharedGLContext); // odd as we wouldn't generally need another engine
const i = new InternalTexture(eng, InternalTextureSource.Unknown, true);
i._hardwareTexture = new WebGLHardwareTexture(mySkiaGLTexture, sharedGLContext
const t = new ThinTexture(i);

const m = new StandardMaterial("customSkiaMaterial", scene);
m.disableLighting = true;
m.emissiveTexture = t; // !! Compiler error ThinTexture isn't of type BaseTexture!!

That last line is what is causing me confusion, this would be a mistake wouldn’t it? Or would casting this away still work?

jw

yup instead of ThinTexture, if you intent to use it with material do the following steps:

“Finally you could create a Texture without url and then set texture._texture to your internalTexture from before.”

// PSEUDO CODE
var texture = new Texture(null, scene);
texture._texture = i;
i.isReady = true;

instead of const t = new ThinTexture(i);

1 Like

Ah! Yes, I was stuck on the using ThinTexture. I will proceed along these lines and post my findings here. I will start working within a playground that I can share.

1 Like

@sebavan in the playground I am getting a compile error of image
Not sure how to get around that one. Unfinished playground is here: https://playground.babylonjs.com/#PWAZY2#4

Thanks in advance!
jw

Looks like we did not expose it so far, I ll add it into the next nightly in a couple of hours .

Thanks!

@sebavan I see that a new tag was placed for alpha 41, how long does it take for that to propagate to the playground?

I did not put the changes in 41 I am actually running a new nightly with it as we speak

Should be up in 1 hour .

You can follow the deploy here https://github.com/BabylonJS/Babylon.js/pull/10993 allow ten more minutes after merge for CD to deploy

Sweet! I can see alpha 42 on npm, but I am not sure how to get the playground to use the latest. Just switching between a previous version and the “latest” version menu items doesn’t change what is displayed in the playground header.

Just refresh the PG on 5.0 with ctrl+F5 and you ll get latest 42:

Yes! Hard refresh did the trick! Thanks @sebavan !

1 Like