Texture Gamma Space + Node Materials

[Valentine]

Hi folks, apologies, don’t have a PG repro for this just yet, but as this is a relatively recent issue, perhaps it will make sense in terms of what has been recently modified.

What we’re doing: Loading in GLTFs from s3 with a custom loader that creates a node material and plugs the GLTF textures - which are KTX compressed - into the node material nodes (this is why it’s a little hard to get a good PG repro easily).

Our initial issue: When we went from preview 22 to preview 23, we found that all the textures we were loading into the node material were suddenly assumed to be in gamma space - diffuse became too dark, normal textures were broken, roughness became too shiny etc. This issue persists in v31.

Our initial solve: We created a new version of our node materials that set all of the texture nodes to convert the textures back to linear space.

Our issue now: Everything works great on windows desktop, but we’re finding inconsistent results on Macs. Meshes are getting loaded in with the diffuse sometimes appearing correct, but sometimes in the wrong gamma. Only seems to affect Mac - and in both cases we’re running in Chrome.

Some examples.

Correct gamma on PC:

Incorrect gamma on Mac:

It seems to only affect the diffuse channel, so my guess is that there are some assumptions being made when the GLTF is getting unpacked about what the colour space of the textures is, which is being evaluated differently on Macs vs Windows machines for some reason - maybe it’s a decompression issue with KTX compressed textures? Inside the node material we do not have an option to treat the texture as linear or gamma space, only to apply a transform to linear space or to gamma space.

I did try checking if the Babylon Texture object, when it is being assigned to the Node Material Block has its ‘gammaSpace’ property set to true, and that appears to be set to true on both PC and Mac, though it’s worth noting that switching it to false does not seem to make any visual difference to the scene as I would expect it to - which seems incorrect.

So, still working on getting a PG-able example of this up, but since this is related to recent changes, I’m wondering if maybe you guys might have some idea what has changed that might be causing these issues?

[Evgeni_Popov]

I wonder if it’s because of our switch to sRGB buffers…

Try to add this code and see if it helps:

BABYLON.SceneLoader.OnPluginActivatedObservable.addOnce(function (loader) {
    if (loader.name === "gltf") {
        loader.useSRGBBuffers = false;
    }
});

[br-matt]

Hey @Evgeni_Popov unfortunately that did not do it.

Did manage to get a repro up though ! Due to the large amount of code with extensions and everything else, it’s a github one: GitHub - code-matt/BabylonRepro: Reproductions for BabylonJS issues

src/App.js has the flow
and
babylon/index.js the scene setup / functions

The floor pictured above are the same brightness on PC, like Valentine describing

[Evgeni_Popov]

The repro does not work:

I had a look to the node material, the settings of the textures seem wrong to me:

• Emissive Texture: the “Convert to linear space” should be checked
• ORM Texture: the “Convert to gamma space” should not be checked
• Normal Texture: the “Convert to gamma space” should not be checked

That would not explain why you have different results depending on the computer, though…

@Valentine

I did try checking if the Babylon Texture object, when it is being assigned to the Node Material Block has its ‘gammaSpace’ property

This property won’t have any effect in a node material, it is only used for std/PBR materials. In node materials, you are responsible to convert to linear/gamma space if necessary through the switches on the Texture block.

@Valentine

maybe it’s a decompression issue with KTX compressed textures

Have you tried with other files than KTX?

[Evgeni_Popov]

Another thing is that you should not add the lighting output of the PRMetallicRoughness block with the emissive color: the latter one is in linear space whereas lighting is in gamma space. You should instead add all the other outputs of PRMetallicRoughness and then add the emissive color to it. Then you should convert the result to gamma space: you can use the Convert to gamma space switch on the FragmentOutput block for that.

[br-matt]

@Evgeni_Popov sorry about that !

Hmm a few have cloned/ran it successfully, it’s on the latest commit on master branch ? If so maybe try incognito to make sure no plugins are affecting CORS. I want to use this repo for some other issues we are running into so hope can figure it out.

[Valentine]

@Evgeni_Popov

I had a look to the node material, the settings of the textures seem wrong to me:

• Emissive Texture: the “Convert to linear space” should be checked
• ORM Texture: the “Convert to gamma space” should not be checked
• Normal Texture: the “Convert to gamma space” should not be checked

I mentioned this in the original post - our original emissive, ORM and normal blocks didn’t have the conversions checked, but when we upgraded from 22 to 23, the shader got messed up. Normals were wrong, roughness was too shiny, and emissive was too dark. I couldn’t see why this happened, so the quick fix was to apply the gamma space conversions in the shader as you see above. This fix works great on desktop now, but on Mac some meshes - at least as far as the Emissive map goes - it’s behaving the way it used to before, and now the gamma space conversion is overcompensating.

[Evgeni_Popov]

It does work in an anonymous window. However, sometimes after a recompile everything is white instead of having the right materials…

Also, it’s hard to debug because I can’t open the node editor. I managed to be able to have the inspector (by adding import '@babylonjs/inspector'; in index.js) but I get an error when I try to open the node editor.

Regarding the color space of the textures, you should definitely set them correctly and work from there:

• diffuse/emissive. Those are normally authored in gamma space, so you should check “Convert to linear space” as the PBRMetallicRoughness block expects the baseColor input (as all other inputs) to be in linear space.
• ORM and Normal. Those are normally authored in linear space, so you should not check any conversion switches.

Regarding the emissive, see my previous post, as you add it to the lighting output, you should proceed as explained and not directly add them as they are not in the same color space.

[Valentine]

Again, that’s what our set-up was in v22, but it broke in v23 as the textures started all being loaded into the node material with the wrong gamma inputs, and we have had to toggle those switches to compensate. The ORM and Normal are in linear space at source, and Emissive is in gamma space. So it sounds like the default behaviour from 23 onwards is just wrong, and that it was behaving correctly prior - we just needed to fix the disparity as quickly as we could.

The main problem is that before these textures get to the Node material, they’re often (but not always) being loaded into as different gamma spaces based on hardware - so we need to make sure that the gamma space is consistent across different hardware, as we are not able to compensate.

We’re testing to see if it is specifically related to KTX compression, but there is definitely something that has changed between releases.

To be clear, this is what we are seeing with this repro. I’ve updated the repository with a slightly better mesh selection so it’s easier to see the difference:

22 on PC:

22 on Mac:

23-31 on PC:

23-31 on Mac:

These are all using the same node material, all that’s changing is the installed Babylon preview build and/or platform. When we were on 22, we did have ‘Convert to gamma space’ checked on the Emissive material for example, and so it looked like it does now on 23+, but when we switched to 23, our emissive became too dark and that is why we had to disable the ‘to gamma space’ transform. Similar happened to the linear textures.

So:

1. Gamma space of textures has changed between 22 and 23
2. From 23 onwards gamma space works as previously on some meshes/textures on Mac

Regarding adding up PBRMetallicRoughness and Emissive, that’s a great note that I will fix on our end, but the pressing issue is the gamma space inconsistency.

[Evgeni_Popov]

It is hard for me to debug something because the inspector is not working, nor Spector as I must start in an anonymous window…

I really think it is the fact that we added support for sRGB buffers in the glTF loader, as it is new in alpha-23.

SceneLoader.OnPluginActivatedObservable.addOnce(function (loader) {
    if (loader.name === "gltf") {
        loader.useSRGBBuffers = false;
    }
});

should disable this support, but to be sure you should check with Spector that the textures are not sRGB textures.

Not doing loader.useSRGBBuffers = false should/will definitely make a difference with the default setting (loader.useSRGBBuffers = true) because with the latter, the textures flagged with nonColorData=false (all by default) will be loaded in sRGB buffers, which means no gamma to linear space conversion should take place because the sampling from these buffers already returns linear values.

What would help is a link to a working sample with loader.useSRGBBuffers = false; so that I can start the inspector / Spector and see what’s going on.

[Evgeni_Popov]

When we were on 22, we did have ‘Convert to gamma space’ checked on the Emissive material for example, and so it looked like it does now on 23+, but when we switched to 23, our emissive became too dark

To me it means you didn’t do:

SceneLoader.OnPluginActivatedObservable.addOnce(function (loader) {
    if (loader.name === "gltf") {
        loader.useSRGBBuffers = false;
    }
});

before loading the glTF files.

[br-matt]

@Evgeni_Popov We have that loader.useSRGBBuffers = false; in the repro already from when you suggested the other day, did not make a difference though sadly I pushed an update to add a toggle inspector button.

I don’t understand why you are having to use anonymous tab since no one else getting those CORS errors. Can you give it another shot in regular ?

[br-matt]

Oof, changing from addOnce to add helps! Sorry I did not see that before

[Evgeni_Popov]

That’s a good news

[Valentine]

I have just recalled, this is set up as it is due to the tonemapping set-up in our main application - as discussed in some previous threads, I believe as it stands the output of the shader is linear and gets converted to gamma later in the pipe, to ensure correct blending with other elements like glass.

In the main application proper, rather than the repro repo, there is no visual difference between wiring up lighting or specularInd to add to the emissive