Odd UV coordinates edge cases

[ChristofferA]

Summary
We get unexpected colors from texture mapped materials missing UVs or having them all set to zero.

Question
Does the “standard” allow UV coordinates that are all zero or all identical? If so, is the edge cases demonstrated below to be considered a bug?

Background
I was debugging a model that was rendered differently in Three.js compared to Babylon.js and finally tracked it down to the UV coordinates all being zero in combination with the existence of an normal-map.

As mentioned in earlier posts, we get the data from an external API and thus have to make the best of what we get, so “fix the model” isn’t really an option here.

I understand that having identical UV coordinates (for example all being zero) is quite an edge case, but is it breaking any specs?

Say that all UV coordinates are zero, I would expect that the texture samplers would take the value of that exact position in the maps and in practice end up with a constant values over the render faces, be that albedo or normals.

Playground
I created a test case playground to test this theory:

All squares uses the same albedo texture. All but the front left one has the same normal-map set.

1. Front left: Normal UVs from [0 - 1]
2. Front center: Normal UVs from [0 - 1]
3. Front right: UVs set in range [0 - 0.001]
4. Back left: UVs all set to 0
5. Back center: UVs all set to 0.5
6. Back right: UVs all set to 0.001

1 & 2 is exactly as expected, just normal uv-mapping.

I expected 3 to basically be the green color from the bottom left pixel of the albedo texture, but instead it starts with gray. The smaller the range, the more gray it gets. Hmmm, perhaps it is from the center of the albedo map? Haven’t checked that.

I expected 4 to be the green color from (0,0) in the albedo texture and use the neutral normal from the normal map at (0, 0). It renders fully black, I expect this is due to the computed normal is broken.

I expected 5 to use the color from the middle (0.5, 0.5) of the albedo texture and use a neutral normal from the normal map at (0.5, 0.5). Instead we get a fascinating repeating texture, identically strangely mapped in albedo and normal map.

Case 6 is the weirdest. Here you get a static gray color in the same way as case 3 above. The bumps though, they go all crazy and the pattern depends on the camera position.

[Evgeni_Popov]

In such cases I think it’s a good idea to do an unlit render first:

Your pattern for #5 is because you made a mistake in your uvs, you used 0,5 instead of 0.5. After correction:

Next make sure the filtering does not get into your way by setting it to NEAREST_NEAREST:

At this point, I think you get what you expect from an unlit rendering (https://playground.babylonjs.com/#2TCZZ9#5).

Now back to lit (https://playground.babylonjs.com/#2TCZZ9#6):

The problems you see with the normal map is because a uv gradient is used to compute the bump mapping. When using the same values for all uvs you are getting a u/v gradient of 0, which leads to an undefined cotangent frame because tangent = bitangent = vec3(0). The banding you see in 5 and 6 maybe due to floating point precision issues where the gradient is not exactly equals to 0.

[ChristofferA]

Thanks so much @Evgeni_Popov for the detailed reply.

@Evgeni_Popov

Next make sure the filtering does not get into your way by setting it to NEAREST_NEAREST:

Didn’t think about the filtering, yeah the texture sample at the extreme (0, 0) of course gets affected by the pixels at (1, 0), (0, 1) and (1, 1) since the texture by default wraps. Changing the wrap to CLAMP_ADDRESSMODE does the same as setting the filtering mode.

@Evgeni_Popov

Your pattern for #5 is because you made a mistake in your uvs, you used 0,5 instead of 0.5.

Region formatting strikes again, decimal separator is “,” in Swedish and I guess I was a bit to fast while testing. Nice catch.

So, yeah, albedo sampling thus works as expected.

I’m still a bit unsure around the bump mapping though. I can’t locate any definition in any standard if it allowed to have constant UV mapping coordinates or not. I can see how it in the case of old style bump mapping (based on height maps) would be sort of undefined, but with normal mapping it feels very well defined which normal should be used; “Simply” sample the normal map.

I can understand that the calculations could go haywire in what I would guess is a ´0 / dx´ type of situation, where the result is extremely well defined for all dx values except 0, but at the same time ‘lim(0 / dx), x → 0’ is well defined as 0.

So if the “uv-gradient” is well defined for infinitely small uv-steps, then it shouldn’t really fail for the case where the step size is zero.

Note that I of course can detect this case when loading the geometry and do a fallback to a modified cloned material. I will have to in any case since we are affected by this today, but if constant UV-values is allowed in the standard I think it would be best if the shaders support it as well, especially when it looks like Three.js does handle it.

[Evgeni_Popov]

If you provide your own tangents with the mesh geometry you won’t get the artifacts because the tangent does not need to be recomputed:

https://playground.babylonjs.com/#2TCZZ9#7

For normal mapping, Threejs is using the implementation from The Hacks of Life: Per-Pixel Tangent Space Normal Mapping and Babylonjs from Followup: Normal Mapping Without Precomputed Tangents | The Tenth Planet. Those implementatins are similar and both are using the u/v derivatives to build the tangent / bitangent: if those derivatives are 0 both vectors ends up being vec3(0.).

Maybe you were using bump mapping (and not normal mapping) in Threejs, else I don’t see how it could work in those edge cases (or you were passing pre-built tangents).

[ChristofferA]

@Evgeni_Popov

If you provide your own tangents with the mesh geometry you won’t get the artifacts because the tangent does not need to be recomputed:

Interesting, I will have to look into that, would be nice if our workaround doesn’t need to touch the materials. Thanks!

@Evgeni_Popov

Maybe you were using bump mapping (and not normal mapping) in Threejs, else I don’t see how it could work in those edge cases (or you were passing pre-built tangents).

Actually, you are right there. In this particular case the input from the external source is a height map, that we then need to manually derive into a normal map to supply to Babylon.

I would however like to come to a conclusion if constant UV coordinates are valid according to the specs or not to get some kind of closure on the subject. =)

[Evgeni_Popov]

Of course constant uvs are valid, you can throw any value you want in uv coordinates (except maybe +/- Infinity or NaN values)!

As explained above, the problem here is not the uv values themselves but how they are used in the normal mapping code.

You have another workaround which is to implement your own bump mapping on top of the CustomMaterial / PBRCustomMaterial or with a node material, if normal mapping is not what you need.

[ChristofferA]

@Evgeni_Popov

Of course constant uvs are valid, you can throw any value you want in uv coordinates (except maybe +/- Infinity or NaN values)!

OK then, that means I need to support it in our code. I will add some workarounds based on your help.

Do you think this will ever be supported in Babylon.js or is it to expensive to workaround the edge case?

[Evgeni_Popov]

The normal mapping code will stay this way, so I guess you need to cope with this in a way or another.

[ChristofferA]

Wow, this took a while to debug. About 1/4 of my computed tangents didn’t show up and the rest of them was RGB noise according to the excellent attribute debugger in the Inspector.

I think I finally tracked it down to a bug in the documentation @Evgeni_Popov:
https://doc.babylonjs.com/api/classes/babylon.vertexdata

Normals
An array of the x, y, z normal vector of each vertex […, x, y, z, …]

Tangents
An array of the x, y, z tangent vector of each vertex […, x, y, z, …]

Notice how both of them are documented to have three components; <x, y, z>.

However, after looking at your PG again I noticed that you did not provide <x, y, z> but <x, y, z, 1> for each tangent. So I searched the Babylon source and found this:

     /**
     * Deduces the stride given a kind.
     * @param kind The kind string to deduce
     * @returns The deduced stride
     */
    public static DeduceStride(kind: string): number {
        switch (kind) {
            case VertexBuffer.UVKind:
            // ...
                return 2;
            // ...
            case VertexBuffer.NormalKind:
                return 3;
            // ...
            case VertexBuffer.TangentKind:
                return 4;
        }
    }

Changing my tangent array to have four components and adding a 1 to each vector seems to solve it for me. Buy why is the 1 needed and what use does it have in this case?

Btw, looking more at the Babylon source by searching for tangentKind seems to indicate that the tangents are sometimes treated as having 3 values rather than 4. But perhaps that is just for serialization and other things that only needs the first 3 values?

[Evgeni_Popov]

The additional 1 serves as a multiplier (it could also be -1) for the binormal (I think). Frankly, I don’t know when the value should be +1 and when it should be -1, I guess the DCC tool will put the right values at export time…

[ChristofferA]

OK, so I will just keep it at 1 then, thanks.

Perhaps someone at your end can update the documentation so it states it needs 4 components, not 3?