NME: Make texture level work for normal maps as it does in other materials

Currently the TextureBlock always multiplies the level directly into the color value.
For normal maps this leads to behaviour that differs from the default materials. This makes changing textures or levels on normal maps from code very problematic.
As discussed here this is not a bug but the intended behaviour of the TextureBlock: Strange behaviours in Node Material normals - #12 by Evgeni_Popov

I’d love to have the possibility to use normal maps the way that I can use them as for example in PBRMaterial where the level changes the normal intensity.
It would be invaluable to have this solved for being able to make alternative, flexible materials based on NME that are usable from code.

I’m not entirely sure how to pull that off without breaking the compatibility to the current implementation though.

As far as I can see the related code for the multiplication should be in Babylon.js/textureBlock.ts at 6c22914668aaa747fb179114c153ac5b0efede50 · BabylonJS/Babylon.js · GitHub around line 337.

After thinking about it a while I came up with a few proposals. Not sure which is best and fits your vision best.

  1. Give the TextureBlock a property “isNormalMap”

    • have a property “isNormalMap” that influences the calculation of the level
    • if true calculate the correct intensity of the normal based on the level instead of multiplying it.
    • convert it back to color (val * 0.5 + 0.5) to keep compatibility with the color input of the PerturbNormalBlock
  2. Have a separate NormalTextureBlock

    • have a separate NormalTextureBlock that does not multiply the level with the color
    • have an output on the block for the level
    • this gives the ability to route the color to the input of a PerturbNormalBlock and the level to the strength input.
    • also gives possibility to work with the values between the NormalTextureBlock and the PerturbNormalBlock.
  3. Have a NormalTextureBlock that outputs vector

    • have a separate NormalTextureBlock that converts the pixel value to vector
    • make it calculate the correct normal based on the level
    • make the PerturbNormalBlock accept a normal vector instead of color
    • possibly makes the PerturbNormalBlock strength input obsolete

The second proposal is the only one that would not interfere with the current implementation although I think the third one is the cleanest one.
Proposal one seems to be more overhead in calculations than necessary.
None of them is a perfect fitting solution but rather just ideas of how to make it possibly work.

I would very much appreciate it if you could look into this.

This playground shows the node material and the PBRMaterial side by side. PerturbNormal strength is exposed in the inspector for convenience.
https://playground.babylonjs.com/#NCD99X#2

I like option 1 (easiest to implement) but let me add @Evgeni_Popov and @PatrickRyan to get more opinions

1/ and 3/ can’t work because depending on the case (standard material / pbr material), the normal from the normal map is not modified by the level/strength, it is the normal of the geometry that is modified when constructing the cotangent frame inside the PerturbNormal block.

I would go for a simplified 2/:

  • don’t create a new texture block
  • add a new “level” output to the existing texture block
  • add a switch on the texture block to disable multiplying the texture fetch by the level

For your use case, you would enable this switch and would connect the “level” output to the “strength” input of the PerturbNormal block.

2 Likes

Sounds good to me! That would solve the problem.