How do I use @babylonjs/ktx2decoder properly?

[rassie]

A couple of weeks ago, @babylonjs/ktx2decoder package has been introduced, with the intent of hosting necessary WASM files and code locally instead of on BabylonJS’ CDN. However, there is no documentation on how to use this module.

What I’ve tried to do was this (with Vite):

import { KhronosTextureContainer2 } from '@babylonjs/core/Misc/khronosTextureContainer2';
import ktx2DecoderUrl from '@babylonjs/ktx2decoder/ktx2Decoder.js?url';

(KhronosTextureContainer2.URLConfig as Record<string, string>) = {
    jsDecoderModule: ktx2DecoderUrl
};

This seems to work in the sense that the correct file is loaded, but the following error occurs immediately: “Cannot use import statement outside a module”.

How is this package supposed to be used? Do I need to convert my project to an ESM module for this to work?

[Deltakosh]

cc @ryantrem

[Aus_Karlos]

I to am interested in how to setup the decoder locally. I thought it would work similarly to importing loaders but when a ktx2 file loaded babylonjs requests the the decoder from the cdn.

Edit:
I thought I might of been on the right track. Instead of passing a url string of the location of the local module file I instead create a new Texture container as you can pass the modules directly.

import { KhronosTextureContainer2, IKhronosTextureContainer2Options } from "@babylonjs/core/Misc/khronosTextureContainer2";
import { engine } from "../index";

const khronosContainerOptions: IKhronosTextureContainer2Options = {
	numWorkers: 4,
	binariesAndModulesContainer: {
		jsDecoderModule: require('@babylonjs/ktx2decoder/ktx2Decoder'),
		jsMSCTranscoder: require('@babylonjs/ktx2decoder/Transcoders/mscTranscoder')
	}
} 
const kContainer = new KhronosTextureContainer2(engine, khronosContainerOptions);

But I have no clue now how to tell BabylonJs to use this new container for the ktx2 loader.

[rassie]

Probably cc @RaananW too, since he’s the author of the PR (Added @babylonjs/ktx2decoder public package by RaananW · Pull Request #14811 · BabylonJS/Babylon.js · GitHub)

[roland]

Hi!

This works for me:

package.json:

{
  "name": "bjs-ktx2",
  "version": "1.0.0",
  "scripts": {
    "dev": "vite --open",
    "build": "tsc && vite build",
    "preview": "tsc && vite build && vite preview --host --open"
  },
  "typeRoots": [
    "node_modules/@types",
    "src/@types"
  ],
  "devDependencies": {
    "typescript": "^5.3.3",
    "vite": "^5.2.7"
  },
  "dependencies": {
    "@babylonjs/core": "^7.0.0",
    "@babylonjs/inspector": "^7.0.0",
    "@babylonjs/ktx2decoder": "^7.1.0",
    "vite-plugin-arraybuffer": "^0.0.6",
    "vite-plugin-wasm": "^3.3.0"
  }
}

---

src/App.ts:

import {
    Scene,
    Vector3,
    HemisphericLight,
    MeshBuilder,
    StandardMaterial,
    Color3,
    Texture,
    Engine,
    ArcRotateCamera,
} from "@babylonjs/core";
import "@babylonjs/core/Debug/debugLayer";
import "@babylonjs/inspector";

import { KhronosTextureContainer2 } from "@babylonjs/core/Misc/khronosTextureContainer2";
import { AutoReleaseWorkerPool } from "@babylonjs/core/Misc/workerPool";
import { initializeWebWorker } from "@babylonjs/core/Misc/khronosTextureContainer2Worker";
import wasmMSCTranscoder from "@babylonjs/ktx2decoder/wasm/msc_basis_transcoder.wasm?arraybuffer";
import "@babylonjs/core/Materials/Textures/Loaders/ktxTextureLoader";

export class AppOne {
    engine: Engine;
    scene: Scene;

    constructor(readonly canvas: HTMLCanvasElement) {
        this.engine = new Engine(canvas);
        window.addEventListener("resize", () => {
            this.engine.resize();
        });
        this.scene = createScene(this.engine, this.canvas);
    }

    debug(debugOn: boolean = true) {
        if (debugOn) {
            this.scene.debugLayer.show({ overlay: true });
        } else {
            this.scene.debugLayer.hide();
        }
    }

    run() {
        this.debug(true);
        this.engine.runRenderLoop(() => {
            this.scene.render();
        });
    }
}

const createScene = function (engine: Engine, canvas: HTMLCanvasElement) {
  const agentPool = new AutoReleaseWorkerPool(4, () => {
      const worker = new Worker(new URL("./worker.js", import.meta.url), {
          type: "module",
      });
      return initializeWebWorker(worker, {
        wasmMSCTranscoder
      });
  });

  const ktx2 = new KhronosTextureContainer2(engine, {
      workerPool: agentPool,
  });

    const scene = new Scene(engine);
    const camera = new ArcRotateCamera(
        "camera1",
        Math.PI / 3,
        Math.PI / 4,
        30,
        new Vector3(0, 0, 0)
    );
    camera.setTarget(Vector3.Zero());
    camera.attachControl(canvas, true);

    const light = new HemisphericLight("light", new Vector3(0, 1, 0), scene);
    light.intensity = 0.7;

    const ground = MeshBuilder.CreateGround(
        "ground",
        { width: 12, height: 12 },
        scene
    );
    const groundMaterial = new StandardMaterial("groundMaterial", scene);
    groundMaterial.diffuseColor = new Color3(0.5, 0.5, 0.5);
    ground.material = groundMaterial;
    groundMaterial.diffuseTexture = new Texture(
        "https://cdn.dedalium.com/3d/dedalium/dedalium2_x256.ktx2",
        scene
    );

    return scene;
};

---

vite.config.js:

import { defineConfig } from 'vite';
import wasmPlugin from 'vite-plugin-wasm'
import arrayBuffer from 'vite-plugin-arraybuffer';

export default defineConfig(({ command, mode }) => {
    return {
        plugins: [wasmPlugin(), arrayBuffer()],
        esbuild: {
            supported: {
                'top-level-await': true
            },
        },
        optimizeDeps: {
            esbuildOptions: {
                supported: {
                    "top-level-await": true
                },
            },
        },
    };
});

---

src/worker.js

import * as KTX2Decoder from "@babylonjs/ktx2decoder";
import { workerFunction } from "@babylonjs/core/Misc/khronosTextureContainer2Worker";
import mscTranscoderJsModule from "@babylonjs/ktx2decoder/wasm/msc_basis_transcoder";
import { MSCTranscoder as jsMSCTranscoder } from "@babylonjs/ktx2decoder/Transcoders/mscTranscoder";

globalThis.KTX2DECODER = KTX2Decoder;
globalThis.MSC_TRANSCODER = undefined;
jsMSCTranscoder.JSModule = mscTranscoderJsModule;
workerFunction(KTX2Decoder);

[bghgary]

@RaananW is out this week and will be back next week. I believe he is working on docs for this package.

[RaananW]

Documentation is coming this week! Will fully explain how to use this package to avoid using CDN resources.

[RaananW]

This is code that came from a package I previously shared, but I can’t seem to find the post. Will be a part of the documentation

[roland]

It does indeed I had to add wasmPlugin to get it working on my system.

[roland]

@RaananW

but I can’t seem to find the post.

CSP violation during KTX2 decoder load

Good day, all! We tried to remove unsafe-eval from our CSP header and start facing this issue

Error

Failed to load KTX2 texture data: Error: KTX2 container - could not transcode the data. EvalError: Refused to evaluate a string as JavaScript because ‘unsafe-eval’ is not an allowed source of script in the following Content Security Policy…

Would be really nice to have safe version of this transcoder. I saw that we can specify url, but I don’t know where I can find safe version. Place where viol…

[RaananW]

you are the best

[roland]

No, you are the master! I learnt how to do it from your post. To be honest your post came just in time, because our IT security was complaining a lot about downloading code dynamically from a foreign CDN and we were about to drop ktx compression but you are the savior @RaananW !

Thanks!

[Aus_Karlos]

I’ve tried following the example code all of you have posted and I just can’t seem to transpile it correctly.
Im using Typescript and Webpack 5.
Im unable to import a wasm file as an arraybuffer into typescript even with the the correct loaders and experimental options enabled.
Hopefully the documents will shed some light.

[RaananW]

Just a note, you can still serve the files locally and use URLs instead of array buffers. But I will try addressing issues with different bundlers when I write the docs, of course.

Have you tried this - arraybuffer-loader - npm?
(Edit - sorry, this is for webpack 4. Let me try something better )

[Aus_Karlos]

What I might do is exclude the transcode worker file from the webpack and serve is as a static resource and just set the new Worker() url to the static resource location. That might be one way as I already need to convince webpack that import mscTranscoderJsModule from "@babylonjs/ktx2decoder/wasm/msc_basis_transcoder"; does have a default export.

[roland]

Switch to vite

[RaananW]

So, i actually have a demo project using the arraybuffer-loader, which works well with webpack 5. You can set it to wasm only and then if you import from wasm you get an arraybuffer.

module: {
    rules: [
      {
        test: /\.wasm$/,
        type: "javascript/auto",
        use: "arraybuffer-loader",
      },
    ],
  },

then you can import wasm from @babylonjs/whatever/file.wasm and use it as the wasmBinary variable.

[Aus_Karlos]

Ahhh… Finally got it to work. needed to jump though some more hoops.
This is for Typescript + Webpack 5

msc_basis_transcoder.d.ts

declare module "@babylonjs/ktx2decoder/wasm/msc_basis_transcoder.wasm?arraybuffer" {
	const content: ArrayBuffer;
	export default content;
}

src/index.ts

...
import { KhronosTextureContainer2 } from "@babylonjs/core/Misc/khronosTextureContainer2";
import { AutoReleaseWorkerPool } from "@babylonjs/core/Misc/workerPool";
import { initializeWebWorker } from "@babylonjs/core/Misc/khronosTextureContainer2Worker";

import wasmMSCTranscoder from "@babylonjs/ktx2decoder/wasm/msc_basis_transcoder.wasm?arraybuffer";
import "@babylonjs/core/Materials/Textures/Loaders/ktxTextureLoader";

const agentPool = new AutoReleaseWorkerPool(4, () => {
	const worker = new Worker(new URL('./worker.js', import.meta.url), {type: 'module'});
	return initializeWebWorker(worker, { wasmMSCTranscoder: wasmMSCTranscoder });
});

const engine = new Engine(canvas, true);
const ktx2 = new KhronosTextureContainer2(engine, { workerPool: agentPool});

src/worker.js [Ignore webpack warning about MSC_TRANSCODER module has no exports]

import * as KTX2Decoder from "@babylonjs/ktx2decoder";
import { workerFunction } from "@babylonjs/core/Misc/khronosTextureContainer2Worker";
import mscTranscoderJsModule from "@babylonjs/ktx2decoder/wasm/msc_basis_transcoder";
import { MSCTranscoder as jsMSCTranscoder } from "@babylonjs/ktx2decoder/Transcoders/mscTranscoder";

globalThis.KTX2DECODER = KTX2Decoder;
globalThis.MSC_TRANSCODER = undefined;
jsMSCTranscoder.JSModule = mscTranscoderJsModule;
workerFunction(KTX2Decoder);

js.webpack.config.js || ts.webpack.config.js

modules.export = {
    experiments: {
        asyncWebAssembly: true,
        syncWebAssembly: true
    },
module: {
        rules: [
            {
                 test: /\.wasm$/,
                 type: "javascript/auto",
                 use: ["arraybuffer-loader"] // npm install arraybuffer-loader
            }
        ]
    }
}

Edit forgot tsconfig.
tsconfig.json

{
  "compilerOptions": {
    "target": "ESNext",
    "module": "ESNext",
    "resolveJsonModule": true,
    "esModuleInterop": true,
    "moduleResolution": "node",

  }
}

Play around with the tsconfig settings. Im not 100% if changing the target, module to esnext and setting resolveJsonModule, esModuleInterop to true actually did anything.

[rassie]

@RaananW sorry for nagging, but have you managed to write something up?