Note that KTX is a container, and these links are about storing BasisU in KTX which will transcode at runtime from BasisU to the appropriate runtime formats, including BC5/BC7 on Windows in some scenarios.
As far as I know, KTX2 doesn’t offer support for anything other than BasisU. After evaluating BasisU, we’ve found the client to be far too slow for our use-case, and we’d rather bake device-specific textures on our servers rather than transcode them on the client. We can gladly put them in a KTX2 container if this is a supported use case, but as far as we know, no software (including BabylonJS) supports device-specific texture formats in a KTX2 container.
Our product streams lots of these textures in and out as the user moves around as fast as they can download more. ~360ms is about 6 frames, which is quite high. Now imagine we’re doing 3 of these textures (albedo, normal, attributes) per sector, and we can have 30 sets of textures ( until they move again ). One texture by itself is not too slow, no, but it adds up quick.
Why transcode for 6ms per texture if we can bake server-side for 0ms per texture?
This is probably more like 22 frames But this is happening in parallel on a worker thread, so it shouldn’t cause frame rate issues. Also, multiple textures can be loaded in parallel. But yes, it can add up if you are loading a lot of textures.
This a trade-off. If the scenario is cross-platform, then using platform specific textures requires producing/storing multiple textures per asset. It is also a little bit more logic to juggle on the client side. KTX with BasisU will eliminate this but will have a small runtime cost. This will get better as the support for this repo is flushed out. There are pros and cons to both approaches.
But this is happening in parallel on a worker thread, so it shouldn’t cause frame rate issues
In theory, yes. In practice, we’re seeing starvation issues that imply that workers aren’t isolated, and there are priority inversion issues here too. I’ve done the profiling, I’ve stared at enough chrome://tracing logs, and I can tell you that a bunch of workers chewing through textures will absolutely cause framerate drops. Yes, some of these are browser bugs (and I’ve filed the browser bugs! I hope they get fixed!), but they’re also on browsers on devices that we also have to ship on.
This a trade-off. If the scenario is cross-platform, then using platform specific textures requires producing/storing multiple textures per asset.
Absolutely. However, based on everything we’ve seen so far (browser bugs, transcoding time), we strongly believe we’d be much better off doing the work of compressing textures on the server. Babylon already has support for device-specific textures ( Multi-Platform Compressed Textures | Babylon.js Documentation ), we’re just looking to expand this to be a more complete set of texture formats.Though honestly, even if these kinks were worked out, BasisU is not looking like it’s competitive on either quality or performance compared to native textures. Some alternate texture solution will likely be required, and pre-compressed texture formats are good to have support for, no matter what container format we want to put them in.
I’m not trying to steer you away from this solution. There are certainly cases where it makes sense to serve compressed textures per platform. My understanding is that the GIS community (e.g., Cesium) is using BasisU pretty heavily for map data and I’m wondering if they see the same type of issues. It’s also possible that Babylon’s implementation has issues.
Just want to jump in to say that part of the issue for us is also quality - particularly on normal maps. Besides BC5 there doesn’t seem to be any option that would let us have a sensible number of bits per channel when compressing normal maps.
Compressing normals with 5:6:5 compression like DXT1 gives awful results visually due to the x vector having fewer bits than the y vector. UV islands that are next to each other but rotated 90 degrees exhibit discontinuities, textures like wood grain look completely different at different orientations etc. 8:8 compression with BC5 seems to be the only sensible way to compress normals and address these issues?
IIRC, one or more of the many modes of BC7 should be able to handle normal maps fine. If you are using UASTC mode of BasisU, my understanding is that it will work well with normal maps transcoded to BC7, far better than the ETC1S mode of BasisU.
I was curious to see what Cesium’s performance was like, since my suspicion was that they were also seeing similar issues, but I was unable to find a sample that used BasisU textures. Just going to the Sandcastle and opening the Network tab, it seems that the tiles streamed in are JPEGs. https://sandcastle.cesium.com/
Do you have a sample that I can test BasisU/UASTC performance with?
Release notes saying support was added doesn’t really seem the same as “using BasisU pretty heavily for map data”. I’d love to get harder numbers on this, but I can’t find any demonstration of that… meaning that the only numbers we have are our own.
I’m saying this because the Khronos working groups I join talk about and show demos using BasisU for textures in geospatial scenarios. I don’t have the numbers unfortunately, but I can ask around to see if there are numbers to share.