Basis/ktx cubemaps

vinhui · May 10, 2022, 1:03pm

Hi,

I’m trying to load cubemaps as efficiently as possible. Currently I’m loading 6 faces as jpegs. I’ve tried converting those faces to a single .basis file as cubemap and ktx2 but babylon doesn’t seem to want to load it. I’ve tried basisu and toktx.
Babylon gives and error with the ktx2 file that it has an incorrect header. With the basisu file, I just get a black texture without console errors. Plain 2d .basis textures do work.

Here on the forum I see some mixed messages as to if it should work or not.

bghgary · May 10, 2022, 3:08pm

This isn’t supported yet for ktx2. I’m not sure about BasisU. If you are trying to load an environment map, have you tried our custom .env format? Using An HDR Environment For PBR | Babylon.js Documentation

vinhui · May 10, 2022, 3:10pm

Our cubemaps are basic non-hdr cubemaps that we use for something other than reflections. Is the .env format still suitable for that?

bghgary · May 10, 2022, 3:21pm

@sebavan

sebavan · May 10, 2022, 6:31pm

It can work but in this case why not relying on jpg or png like here ? Skyboxes | Babylon.js Documentation

vinhui · May 11, 2022, 11:01am

Well, we’re currently relying on jpegs and it works ok. But since I’m loading cubemaps pretty often on runtime, I was hoping I could optimize it a bit more. If I were to optimize the loading more, I could potentially use 4k cubemap faces instead of 2k. Loading the 4k jpeg cubemaps takes a bit too long for my liking.

carolhmj · June 9, 2022, 4:40pm

Hello just checking in, was your question answered? @vinhui

vinhui · June 9, 2022, 5:56pm

Yes as in, I now know that it’s not supported. But it’s still something I’d like to have.

We’re loading quite a bunch of cubemaps in succession after user interactions, and we’re currently preloading low res cubemaps to ease the delay. Unfortunately there’s still quite a cpu spike.

vinhui · June 10, 2022, 3:45pm

Did some digging again today. It seems that the ktx cubemap loader requires non ktx2 cubemaps while you can use ktx2 for other textures

github.com

BabylonJS/Babylon.js/blob/master/packages/dev/core/src/Materials/Textures/Loaders/ktxTextureLoader.ts#L70

      
        
             * @param onLoad defines the callback to trigger once the texture is ready

             */

            public loadCubeData(data: ArrayBufferView | ArrayBufferView[], texture: InternalTexture, createPolynomials: boolean, onLoad: Nullable<(data?: any) => void>): void {

                if (Array.isArray(data)) {

                    return;

                }

            

                // Need to invert vScale as invertY via UNPACK_FLIP_Y_WEBGL is not supported by compressed texture

                texture._invertVScale = !texture.invertY;

                const engine = texture.getEngine() as Engine;

                const ktx = new KhronosTextureContainer(data, 6);

            

                const loadMipmap = ktx.numberOfMipmapLevels > 1 && texture.generateMipMaps;

            

                engine._unpackFlipY(true);

            

                ktx.uploadLevels(texture, texture.generateMipMaps);

            

                texture.width = ktx.pixelWidth;

                texture.height = ktx.pixelHeight;

I tried generating some ktx textures with toktx but they either become ktx2 or i’m unable to compress them properly (only compressed formats currently supported)

vinhui · June 15, 2022, 10:02am

After some more digging, it seems that ETC1S cubemap basis files work, but UASTC cubemap basis files do not.

The same basis file does work when loading it into the basis universal webgl demo.
The spawned webworker that transcodes the basis file returns a success: false in the basis.ts file.

Where are the basis_transcode.js and basis_transcode.wasm in the babylonjs repo coming from? The checksum of the wasm doesn’t match the one from the basis universal repo and the basis_transcode.js seems just different in general.

bghgary · June 15, 2022, 8:57pm

It’s probably a really old version from the Basis repo at this location: https://github.com/BinomialLLC/basis_universal/tree/master/webgl/transcoder/build. You can try a newer version by setting BasisToolsOptions to see if it will help?

vinhui · June 16, 2022, 9:58am

It wasn’t a drop in replacement, but I managed to tweak the basis.ts file and now the uastc files seem to be working

I’m not entirely familiar with TS so it might not be entirely like it should, but here’s the git patch:

diff --git a/packages/dev/core/src/Misc/basis.ts b/packages/dev/core/src/Misc/basis.ts
index 29aa53dde8..b31fdc8e74 100644
--- a/packages/dev/core/src/Misc/basis.ts
+++ b/packages/dev/core/src/Misc/basis.ts
@@ -59,6 +59,18 @@ export class BasisTranscodeConfiguration {
          * etc2 compression format
          */
         etc2?: boolean;
+        /**
+         * dxt compression format
+         */
+        dxt?: boolean;
+        /**
+         * astc compression format
+         */
+        astc?: boolean;
+        /**
+         * bc7 compression format
+         */
+        bc7?: boolean;
     };
     /**
      * If mipmap levels should be loaded for transcoded images (Default: true)
@@ -75,14 +87,27 @@ export class BasisTranscodeConfiguration {
  * Enum of basis transcoder formats
  */
 enum BASIS_FORMATS {
-    cTFETC1 = 0,
-    cTFBC1 = 1,
-    cTFBC4 = 2,
-    cTFPVRTC1_4_OPAQUE_ONLY = 3,
-    cTFBC7_M6_OPAQUE_ONLY = 4,
-    cTFETC2 = 5,
-    cTFBC3 = 6,
-    cTFBC5 = 7,
+    cTFETC1= 0,
+    cTFETC2= 1,
+    cTFBC1= 2,
+    cTFBC3= 3,
+    cTFBC4= 4,
+    cTFBC5= 5,
+    cTFBC7= 6,
+    cTFPVRTC1_4_RGB= 8,
+    cTFPVRTC1_4_RGBA= 9,
+    cTFASTC_4x4= 10,
+    cTFATC_RGB= 11,
+    cTFATC_RGBA_INTERPOLATED_ALPHA= 12,
+    cTFRGBA32= 13,
+    cTFRGB565= 14,
+    cTFBGR565= 15,
+    cTFRGBA4444= 16,
+    cTFFXT1_RGB= 17,
+    cTFPVRTC2_4_RGB= 18,
+    cTFPVRTC2_4_RGBA= 19,
+    cTFETC2_EAC_R11= 20,
+    cTFETC2_EAC_RG11= 21
 }
 
 /**
@@ -314,35 +339,46 @@ declare let Module: any;
 function workerFunc(): void {
     const _BASIS_FORMAT = {
         cTFETC1: 0,
-        cTFBC1: 1,
-        cTFBC4: 2,
-        cTFPVRTC1_4_OPAQUE_ONLY: 3,
-        cTFBC7_M6_OPAQUE_ONLY: 4,
-        cTFETC2: 5,
-        cTFBC3: 6,
-        cTFBC5: 7,
+        cTFETC2: 1,
+        cTFBC1: 2,
+        cTFBC3: 3,
+        cTFBC4: 4,
+        cTFBC5: 5,
+        cTFBC7: 6,
+        cTFPVRTC1_4_RGB: 8,
+        cTFPVRTC1_4_RGBA: 9,
+        cTFASTC_4x4: 10,
+        cTFATC_RGB: 11,
+        cTFATC_RGBA_INTERPOLATED_ALPHA: 12,
+        cTFRGBA32: 13,
+        cTFRGB565: 14,
+        cTFBGR565: 15,
+        cTFRGBA4444: 16,
+        cTFFXT1_RGB: 17,
+        cTFPVRTC2_4_RGB: 18,
+        cTFPVRTC2_4_RGBA: 19,
+        cTFETC2_EAC_R11: 20,
+        cTFETC2_EAC_RG11: 21
     };
     let transcoderModulePromise: Nullable<Promise<any>> = null;
     onmessage = (event) => {
         if (event.data.action === "init") {
             // Load the transcoder if it hasn't been yet
             if (!transcoderModulePromise) {
-                // Override wasm binary
-                Module = { wasmBinary: event.data.wasmBinary };
                 // make sure we loaded the script correctly
                 try {
                     importScripts(event.data.url);
                 } catch (e) {
                     postMessage({ action: "error", error: e });
                 }
-                transcoderModulePromise = new Promise<void>((res) => {
-                    Module.onRuntimeInitialized = () => {
-                        Module.initializeBasis();
-                        res();
-                    };
-                });
+                transcoderModulePromise = BASIS({
+                    // Override wasm binary
+                    wasmBinary: event.data.wasmBinary,
+                })
             }
-            transcoderModulePromise.then(() => {
+            transcoderModulePromise.then(m => {
+                Module = m;
+                m.initializeBasis();
                 postMessage({ action: "init" });
             });
         } else if (event.data.action === "transcode") {
@@ -413,16 +449,22 @@ function workerFunc(): void {
     function GetSupportedTranscodeFormat(config: BasisTranscodeConfiguration, fileInfo: BasisFileInfo): Nullable<number> {
         let format = null;
         if (config.supportedCompressionFormats) {
-            if (config.supportedCompressionFormats.etc1) {
-                format = _BASIS_FORMAT.cTFETC1;
-            } else if (config.supportedCompressionFormats.s3tc) {
+            if (config.supportedCompressionFormats.astc) {
+                format = _BASIS_FORMAT.cTFASTC_4x4;
+            } else if (config.supportedCompressionFormats.bc7) {
+                format = _BASIS_FORMAT.cTFBC7;
+            } else if (config.supportedCompressionFormats.dxt) {
                 format = fileInfo.hasAlpha ? _BASIS_FORMAT.cTFBC3 : _BASIS_FORMAT.cTFBC1;
             } else if (config.supportedCompressionFormats.pvrtc) {
-                // TODO uncomment this after pvrtc bug is fixed is basis transcoder
-                // See discussion here: https://github.com/mrdoob/three.js/issues/16524#issuecomment-498929924
-                // format = _BASIS_FORMAT.cTFPVRTC1_4_OPAQUE_ONLY;
+                format = fileInfo.hasAlpha ? _BASIS_FORMAT.cTFPVRTC1_4_RGBA : _BASIS_FORMAT.cTFPVRTC1_4_RGB;
             } else if (config.supportedCompressionFormats.etc2) {
                 format = _BASIS_FORMAT.cTFETC2;
+            } else if (config.supportedCompressionFormats.etc1) {
+                format = _BASIS_FORMAT.cTFETC1;
+            } else if (config.supportedCompressionFormats.s3tc) {
+                format = fileInfo.hasAlpha ? _BASIS_FORMAT.cTFBC3 : _BASIS_FORMAT.cTFBC1;
+            } else {
+                format = _BASIS_FORMAT.cTFRGB565;
             }
         }
         return format;

bghgary · June 16, 2022, 5:10pm

Do you mind submitting a PR to GitHub?

vinhui · June 17, 2022, 7:50am

Not at all, you can find it here

Escobar · August 30, 2022, 12:23am

That was finally added to the main branch?

How about the usage, I’ve tried with
skyboxMaterial.reflectionTexture = new BABYLON.CubeTexture("./textures/cubeb.basis", scene);

without luck.

Thank you in advance.

vinhui · August 30, 2022, 7:55am

Have you tried using CubeTexture.CreateFromPrefilteredData ?

Escobar · August 30, 2022, 9:23am

Thank you for the tip, unfortunately it introduces some issues:
(I cannot create a playground since I think that there are no basis cubemaps on Babylon texture resources, if you want I can create one for the tests @Deltakosh )

// Skybox
var skybox = BABYLON.Mesh.CreateBox("skyBox", 100.0, scene);
var skyboxMaterial = new BABYLON.StandardMaterial("skyBox", scene);
skyboxMaterial.backFaceCulling = false;
var cubejpg = new BABYLON.CubeTexture("./textures/cube", scene);
var cubebasis = new BABYLON.CubeTexture.CreateFromPrefilteredData ("./textures/cubeb.basis", scene);
skyboxMaterial.reflectionTexture = cubejpg;
skyboxMaterial.reflectionTexture.coordinatesMode = BABYLON.Texture.SKYBOX_MODE;
skyboxMaterial.disableLighting = true;
skybox.material = skyboxMaterial;
        
scene.onKeyboardObservable.add(function (kbInfo) {
  switch (kbInfo.type) {
    case BABYLON.KeyboardEventTypes.KEYDOWN:
      if (kbInfo.event.key == "c" || kbInfo.event.key == "C")
      {
        if(skyboxMaterial.reflectionTexture == cubejpg)
        {
          skyboxMaterial.reflectionTexture = cubebasis;
          console.log("Set cube basis");
        }
        else
        {
          skyboxMaterial.reflectionTexture = cubejpg;
          console.log("Set cube jpg");
        }
      }
      break;
    }
});

If the first texture applied is cubejpg it looks ok but switching to basis one the sky becomes black.

If I start with the basis one the initial image seems to be wrong in gamma (washed out, probably with 2.2 gamma) and switching to cubejpg works but still washed and the cube faces are not properly ordered.

Regards.

sebavan · August 30, 2022, 3:17pm

you can adapt the gammaSpace of the texture used for reflection with texture.gammaSpace = false;

Topic		Replies	Views
KTX2 As Six Sided Skybox Cubemap Textures Questions	7	796	January 28, 2022
Basis support for skyboxes? Questions	14	1686	June 10, 2020
Skybox ktx2 support Feature requests	1	636	January 31, 2022
Load cubemap as a single file Questions	7	1020	March 29, 2022
Load GLTF with .basis (not KTX2) Questions gltf	4	990	August 12, 2021

Basis/ktx cubemaps

Related Topics