Populate 2D texture array from image asset

Feature requests
1

Hello,
I am asking to add a way to populate a 2D texture array from an HTML image. The main purpose of this thread is to have a discussion before filling a bugreport in github.

Currently RawTextureArray2D only accepts a buffer as image source. This is also reflected by this official playground:

Here, the texture array is populated from raw image data that is created inside the application.

There are workarounds that use a canvas, draw an image asset on the canvas and then get the raw data from that canvas: https://playground.babylonjs.com/#WCH2EK#11
I also tested this approach but the performance is very bad. The raw webgl call accepts an image, so converting the image data back and forth is not optimal.

Note also that there is no straight forward way to populate a texture array from a dynamic texture if the the main use case is not just to load an image asset, but to actually draw something in 2D. This usecase was discussed here: Set content of Texture(Array) from canvas, however I feel this is two different topics.

More material I am aware of:

2

pinging @Evgeni_Popov, @sebavan

3

It seems the first PG you linked is not the right one(?)

Anyway, I think it is a good idea to be able to create a raw texture (not only a 2D array but also a regular or a 3D one) from something else than a data buffer. We could only support ImageBitmap and HTMLCanvasElement as those are the intersection between what WebGL and WebGPU support.

Let’s see if @sebavan agrees, in which case you could create an issue in the repo.

2 Likes
4

Hi,
yes, this was the wrong one. I updated the link to the playground. Thanks for the hint.

Two comments regarding HTMLCanvasElement and ImageBitmap:
Regarding canvas: The question is how does DynamicTexture fit in this picture as I had the impression that this is the solution use a canvas as a source, although it doesnt work with my usecase.
Regarding ImageBitmap: I was not aware of this class, but it seems you can easily and hopefully without too much cost convert from HTMLImageElement to ImageBitmap, but the regular Texture accepts some more data types. I do not miss anything by restricting it to canvas and ImageBitmap, these are actually the ones I currently miss, I just wanted to point out this difference.

Best regards,
Axel

5

I like the idea a lot and probably not too bad to do and I think we should support:

  • HtmlImageElement
  • HtmVideoElement
  • HtmlCanvasElement
  • OffscreenCanvas
  • ImageBitmap
  • ImageData
  • buffer view

We could do the conversion internally for convenience if not too bad ???

1 Like
6

I think you don’t need to do internal conversation for most of these at all because gl.texImage3D accepts all of these as the source parameter except OffscreenCanvas:

7

This might not be the same in WebGPU :wink: so we d need the conversion there

8

Right. I have not yet used WebGPU, so I am not familiar with the API.

I created a feature request as suggested by @Evgeni_Popov:
Populate 2D texture array from image asset · Issue #13174 · BabylonJS/Babylon.js · GitHub

1 Like
9

Good write up here

Also, u can hack the gl context like this (maybe this can help u in the short run):

1 Like
10

Hi @jeremy-coleman,
Both sources are very helpfull. AFAIK, I get from the first one that you can basically also create a texture from an ImageBitmap and Canvas, while the second one is likely usable to create a workaround. Thank you for both!

Best regards,
Axel