Texture.serialize, hardcoded image/png for all mime types

kzhsw · March 1, 2025, 7:39am

Version: 7.51.1 and git master
OS: any
Browser: any
Engine: any

Desc

In texture.serialize, if the texture contains _buffer as Uint8Array, commonly if loaded from a glb model, serializing texture will result in base64 url in image/png type.

But in gltf spec, texture can be in many types, including png, jpeg, webp, and possibily avif.

github.com/BabylonJS/Babylon.js

packages/dev/core/src/Materials/Textures/texture.ts

99402bef9


      
          

          if (!serializationObject) {

              return null;

          }

          

          if (Texture.SerializeBuffers || Texture.ForceSerializeBuffers) {

              if (typeof this._buffer === "string" && (this._buffer as string).substring(0, 5) === "data:") {

                  serializationObject.base64String = this._buffer;

                  serializationObject.name = serializationObject.name.replace("data:", "");

              } else if (this.url && this.url.startsWith("data:") && this._buffer instanceof Uint8Array) {

                  serializationObject.base64String = "data:image/png;base64," + EncodeArrayBufferToBase64(this._buffer);

              } else if (Texture.ForceSerializeBuffers || (this.url && this.url.startsWith("blob:")) || this._forceSerialize) {

                  serializationObject.base64String =

                      !this._engine || this._engine._features.supportSyncTextureRead ? GenerateBase64StringFromTexture(this) : GenerateBase64StringFromTextureAsync(this);

              }

          }

          

          serializationObject.invertY = this._invertY;

          serializationObject.samplingMode = this.samplingMode;

          serializationObject._creationFlags = this._creationFlags;

          serializationObject._useSRGBBuffer = this._useSRGBBuffer;

Texture has a prop named mime type that can be directly loaded from model, and since image formats supported by browser is limited, it can be detected from first few bytes of the buffer.

github.com/BabylonJS/Babylon.js

packages/dev/core/src/Materials/Textures/texture.ts

master


      
          private _cachedReflectionUScale = 0;

          private _cachedReflectionVScale = 0;

          private _cachedReflectionCoordinatesMode = -1;

          

          /** @internal */

          public _buffer: Nullable<string | ArrayBuffer | ArrayBufferView | HTMLImageElement | Blob | ImageBitmap> = null;

          private _deleteBuffer: boolean = false;

          protected _format: Nullable<number> = null;

          private _delayedOnLoad: Nullable<() => void> = null;

          private _delayedOnError: Nullable<() => void> = null;

          private _mimeType?: string;

          private _loaderOptions?: any;

          private _creationFlags?: number;

          /** @internal */

          public _useSRGBBuffer?: boolean;

          private _forcedExtension?: string;

          

          /** Returns the texture mime type if it was defined by a loader (undefined else) */

          public get mimeType() {

              return this._mimeType;

          }

Evgeni_Popov · March 3, 2025, 9:59am

I think it’s a known issue with the glTF serializer, cc @alexchuber and @srzerbetto.

alexchuber · March 6, 2025, 6:27pm

Thanks for the report! Yes, this is a known issue that I plan to investigate soon. I recall seeing your previous thread about this

(Sidenote, our serialize methods like the one in Texture are for our .babylon exporter, not .gltf)

kzhsw · March 10, 2025, 3:56am

Hmmm, my previous threads are about gltf exporting, it changes name and content of the texture

alexchuber · April 9, 2025, 7:11pm

Here’s a fix to address this case:

github.com/BabylonJS/Babylon.js

glTF Loader: Get `mimeType` from URI if not defined

master ← alexchuber:mimetype-from-url

opened 01:10AM - 09 Apr 25 UTC

alexchuber

+39 -23

glTF images that did not specify a `mimeType` would not set the `Texture.mimeTyp…e` field. This happens with GLBs, for example. While the field isn't required, it helps with export. The glTF exporter uses this field to try to preserve the original mime type when re-encoding the images-- if undefined, they'll default to PNG. The Babylon exporter should also be using this information when serializing its image buffers.

Texture roundtripping needs more work beyond the scope of this PR. So far, we don’t have a reliable way to preserve the original texture format-- Texture.mimeType is only a hint. While it’s true that we could read the first few bytes of the image data to get the format, there’s some extra complexity to consider. For example, if we parsed the data at load time, we’d incur extra computation for every loaded texture in preparation for something (an export) that may never happen. We also can’t parse the buffer at export time, as we don’t always have it.

labris · April 10, 2025, 4:48am

What is currently the best way to load a GLB with webp textures and export it with webp textures?

alexchuber · April 10, 2025, 7:54pm

We’d need to add EXT_textures_webp support to the glTF exporter to do it correctly, or the best way.

We already preserve imported webp images on export (except in Safari and maybe Node). However, because we don’t add the extension to the glTF, it’s technically an invalid glTF. That said, I’m still able to load the invalid glTF in Babylon and Blender. They must be forgiving about image/webp mime types used without EXT_texture_webp.

Let me add this to the exporter, as it’d be good to have while we work on KHR_texture_basisu encoding support.

labris · April 11, 2025, 12:20am

Good news!
Will it be possible in this case to load a GLB with PNG textures and export it with WEBP textures?

alexchuber · April 11, 2025, 5:55pm

Not yet-- at least, not without some manual manipulation-- but I’m certainly open to the idea. That’d make a good export optimization option

labris · April 11, 2025, 6:01pm

Here is how it done in GLTF-Transform - glTF-Transform/packages/functions/src/texture-compress.ts at main · donmccurdy/glTF-Transform · GitHub
My Sandbox implementation - https://glb.babylonpress.org/

Topic		Replies	Views
Export Babylon scene to GLB/GLTF always turns textures into .png format Questions	4	664	February 15, 2023
When Serialising a PBR Material - How to also serialize albedo texture without relying on a url Questions	2	559	July 11, 2019
BABYLON.SceneSerializer.SerializeAsync usage Questions documentation	9	591	January 12, 2022
Getting issue to generate texture using local file(base64) Questions babylon	16	1829	November 2, 2022
Issue with compressed textures of model on reloading saved serialized scene JSON Bugs material , texture , ktx2 , textures	3	275	September 5, 2023

Texture.serialize, hardcoded image/png for all mime types

Desc

Related topics