Real time transcoding

Feature requests
1

Any interest for real time transcoding as an upgrade to ktx2 textures? Would help reduce an extra step prepping texture assets for users already on web image formats (avif, webp etc).

Ref: GitHub - Ludicon/spark-gltf-demo: Real-Time GPU Texture Compression Demo · GitHub

4 Likes
2

question for @alexchuber for sure

3

+1 to the feature — real-time encoding from AVIF/WebP is the only practical route to GPU-compressed textures for user-uploaded content, where there’s no chance to pre-bake KTX2. That’s a real gap for UGC-heavy apps like ours.

One request on the how: spark.js is free only for non-commercial use or under $100k lifetime revenue, and its shaders are under a separate EULA, so it couldn’t be a core dependency of an Apache-2.0 engine, and commercial projects couldn’t build on it. Could this be designed as a pluggable encoder interface (image in → compressed InternalTexture out, format picked from device caps), with an open-source encoder as the reference implementation? Then anyone who has licensed Spark can plug it in, and everyone else still gets the feature.

1 Like
4

I like the idea! Yeah!

Bouncing off @increasinglyHuman , I think we’d want to create our own implementation. A few things top of mind:

  • If there’s interest in this being a separate package, then we could follow the existing KTX2 decoding conventions. Its implementation is split between @babylonjs/ktx2decoder (more agnostic logic) and @babylonjs/core (more Babylon-focused logic).
  • There’s a transcoding decision tree in the ktx2 decoder package that may be useful here…? Either to reference or reuse.
  • This could be a nice loader option
  • We’d need to clarify the tradeoff (like how there’ll be a longer load time than using .ktx2 images outright, I’d imagine?)
1 Like
5

That’s what I thought.

Ideally, I’d prefer the implementation as part of core so ktx2 becomes completely transparent. But if the code bloat is huge, a separate package would be best. Also have to consider back-compat.

erm, depends on the perf from implementation. Targeting client gpu for the transcoding shouldn’t take more than a few frames, so the load time would be far shorter vs fetching the larger ktx2 files. A simple prototype for testing/benchmarking might be best?