Converting HDR to jpeg with gainmaps

Questions
,
1

Hello everyone,
I am looking to reduce the size of .hdr files and ran into a solution which converts hdr files into jpg/webp + gainmap files which works with Threejs (actually requires three.js renderer for conversion)

Is there a similar implementation for Babylon ?

https://monogrid.github.io/gainmap-js/

2

You can use babylon’s sandbox to convert hdr to .env and compress it using webp.

2 Likes
3

To a regular .webp ?
I can convert an .hdr file to a regular .webp file in photoshop, but its not the same as .webp + gainmap

4

No, it’s still an .env file, it just uses web compression internally.

5

image
image1075Ă—1121 133 KB

1 Like
6

How ? A .webp file cannot hold all the color information of an .hdr file

7

Babylonjs uses the a-channel of the texture for encoding to achieve high dynamic range. So the internal storage can be either png or web.
See the documentation for details

https://doc.babylonjs.com/features/featuresDeepDive/materials/using/HDREnvironment#what-is-a-env-tech-deep-dive

2 Likes
8

Incidentally, sandbox loads hdr with a resolution of 256, so the exported .env has a resolution of 256 * 256 * 6. This is typically 100kb to 200kb when using webp. If you want to export with a higher resolution you can manually load the hdr file with code and set the resolution. Then use EnvironmentTextureTools.CreateEnvTextureAsync() to export.

9

Here’s an example, using playground to export a custom resolution .env using webp compression

2 Likes
10

I just tried to export a 19mb .hdr to .env.

In Safari both png and webp ends up as 703kb, in Chrome png is 672kb and webp is 76kb

11

Maybe babylonjs is using something that safari doesn’t support (oh damn safari) causing the fallback to png.

12

Looks like it. The playground you posted is 4mb in Safari and 489kb in Chrome

13

@qq2315137135
Ok, that does improve the .hdr workflow for environments, thanks for that!
Another use that I was looking forward to was for lightmaps, but without diving too deep into the code, looks like lightmaps are not able to utilize such textures, do you have any insights ?

14

I’m currently using a relatively simple lightmapping workflow, which doesn’t take high dynamic range into account.
Currently the lightmaps in pbr materials support the same rgbd encoding as .env. All that is needed is

texture.isRGBD = true

But I don’t know if there is a good tool to export rgbd encoded .png files from belnder.

2 Likes
16

Thanks, found a related topic, experimented with .png/RGBD and faced the same issues as joie, terrible banding which I can’t seem to fix, but even if it’s fixable a .png is still pretty large and is no match for what webps/jpegs+gainmaps can offer in terms of compressed sizes and quality.

.env with webp compression (which you mentioned to be used as an environment map) gives a good compressed size and works well as a reflection texture, but it cannot be used as a lightmap.

I guess this could be something that is worth looking into for the devs

1 Like
17

Hi there, this is the author of the gainmap.js library.

Maybe I can shed light on the topic.

Gainmaps are really just a normal image + a metadata JSON + a “gainmap” (augmentation image)

You can convert an HDR to a gainmap with our tool and then export it as “Separate data” then use a pure JS decoding method in babylon, we did some tests in pure JS decoding here https://monogrid.github.io/gainmap-js/jstest.html but then we opted for a GPU assisted decoding technique (using three.js shaders in our implementation) because it is simply is an order of magnitude faster.

Decoding a gainMap is a simple pixel by pixel operation that takes into account the original image, the metadata and the augmentation image, this looked like a shader job to us so we went that way but you are free to copy/paste and adapt the code in that https://monogrid.github.io/gainmap-js/jstest.html page (inspect source and you will find the code) in order to make it work with (virtually) any engine.

Even better would be to copy/paste the GPU decoder fragment shader and make it work in babylon.js natively so it is actually usable (CPU decoding is too slow IMHO).

Nothing prevents anyone to use the “separate data” method (which also works with webp/avif/any other image format), most of our library focuses on decoding the “single JPEG” approach (called UltraHDR) where these 3 pieces of information are baked into a single file.

Hope this helps!

EDIT: added link to relevant fragment shader code

1 Like