Node PBR material renders black if mesh doesn't have normals

Node material normal (and tangent) inputs return zero vector if mesh doesn’t have those attributes. This for example causes PBRMetallicRoughnessBlock to render black, but I guess it also applies in general since most custom shading node trees would also rely on normal.

Example playground code here: Babylon.js Playground

A good solution would probably be to populate fragment shader “normal” and “tangent” type InputBlock with a derivative calculated normal and tangent when the attribute is missing.

Babylon.js seems to already have shader code for normal calculation: Babylon.js/default.fragment.fx at 2b72313def0986eb49d30dbd36a9d80e6505c33c · BabylonJS/Babylon.js · GitHub

And also for tangents: Babylon.js/bumpFragmentMainFunctions.fx at 2b72313def0986eb49d30dbd36a9d80e6505c33c · BabylonJS/Babylon.js · GitHub

If InputBlock of normal / tangent type would auto generate derivative based values for fragment shader when mesh attributes are missing, that would make NodeMaterial better usable for all kinds of meshes and also avoid confusion in those cases when the attributes are missing.

Insightful thoughts @MiikaH. I’m curious, does the derivative node in the NME allow you to work around this potentially?

HTH

Yes, using the derivative node you can indeed do a workaround.

However, because it isn’t possible to use preprocessor conditions (Eg. #ifndef NORMAL) with nodes alone, it would get executed even for meshes that include normals. And then you would also have to use length of the normal attribute in combintation with Lerp node or similar to determine whether the attribute or derivative-calculated normal is used. So… Possible, but not practical. :slightly_smiling_face:

For my personal workaround I already created a custom EnsureNormalBlock node block, that replaces the input normal with derivative based one if attribute isn’t available. But having Babylon.js take care of the calculation automatically when needed would no doubt be the least confusing solution for all users in general.

Let me cc @Evgeni_Popov who worked on these nodes

So far my stance is that it is by design but I also agree that we should provide an alternative somehow

2 Likes

Yes it is by design but not related to the PBR block, if you use the normal / tangent mesh attribute anywhere in your node material it is assumed it is not empty.

I’m not sure if it would be easy to generate the normals/tangents in the node material when not provided, it is by far easier to simply do it in js code when you load a mesh: check if there are some normal/tangent data and if not generate them. Also, the shader code will be faster as the data are already available and won’t have to be computed.

Even if normals were generated in the shader, you could only achieve flat shading because the normals computed using the derivatives are the same accross the triangle (because it is a flat plane).

That is a really good point. We do not want to regenerate them on every frame

So yes definitely something to do once and for all:
Mesh | Babylon.js Documentation (babylonjs.com)

To test presence of normals:

mesh.getVerticesData(BABYLON.VertexBuffer.NormalKind);

Ok, that makes sense. I can manage with my workaround custom node block for my use case. :slightly_smiling_face:

Though its worth noting that in some cases calculating the normals in javascript may be unnecessarily slow if the mesh has millions of polygons. And you also end up unnecessarily doubling memory used for the mesh data.

Another complication are for example morph targets. In order for those to be shaded correctly you would need to calculate normals also for the morph target attributes. I don’t think mesh.createNormals() does that at the moment.

I think that is why the current PBRMaterial uses derivative normals as a fallback. It is practically no cost operation in the shader and ensures that the shader works correctly regardless of whether the mesh itself has normals or not.

Even if normals were generated in the shader, you could only achieve flat shading because the normals computed using the derivatives are the same accross the triangle (because it is a flat plane).

Yeah, flat shading would be what is expected. Otherwise you would be making assumptions about the smoothing conditions. For example GLTF spec specifies that primitives without normals are to be flat shaded.

@Evgeni_Popov: can we add a way in the PBR nodes to use the fallback if no normals?

Doing it for PBR nodes only wouldn’t likely make sense. Then you could end up with weird inconsistencies where PBR block works but others that also use normal input don’t.

So it would either have to be:

  • Automatic fallback applied in the InputBlock
  • A separate “fallback attribute” node that user has to specifically add in between the node inputs to ensure a normal
  • Or assume that NodeMaterial only works with meshes that have precalculated normals

It is not related to the PBR node but to the normal input attribute. The PBR block does not compute the normal itself, it is taking it as an input:

image

So, as @MiikaH is saying, we would either need to update the Input block to generate code for the normal (if the normal attribute does not exist) or use a new additional node to ensure the normal does exist. However, we may encounter some problems because generating the normal with derivatives can only be done in the fragment shader, and if the normal must be used in the vertex shader it can’t work…

This is the custom “EnsureNormal” block that I added as a workaround for my use case:

Something like that could work if automatic generation gets too complicated.

It takes normal (the mesh attribute) and position (to use in the derivative calculation). and based on #ifdef NORMAL preprocessor condition it replaces the normal with a generated one. It also has a boolean “force flat normals” property that can be used to always overwrite the mesh normal with a flat normal if preferred. (Which in my use case is option I needed anyway)

Anyway, even that gets somewhat complicated… Because it is a fragment block it also means that the following worldNormal block has to be a fragment only block. Though I suppose that is unavoidable when dealing with fragment generated value. :slightly_smiling_face:

And if I recall correctly, it also turned out that ReflectionBlock was throwing an error if PBR worldNormal input was a fragment only connection, but setting reflection.forceIrradianceInFragment to true fixed that. This is probably more of a bug in ReflectionBlock itself. It should auto detect fragment only input and set that to true automatically.

Yeah I understand all of that and that’s why my stance here is to stick with the current plan: NME does not support meshes with no normal or tangents (If you use them)

@MiikaH: Would you want to do a PR of your node? Maybe we can integrate it if it makes sense (even though I’m not yet convinced ;))

My current node only implements parts I needed to make it work, so as-is it isn’t intented for public integration.

However, if a someone more familiar with babylon nodes wants to add a proper “ensure normal” node or something like it I can then of course send my source code to them as reference. It is very simple node after all. Just couple of lines of glsl.

It would be nice to have official way to deal with meshes without normal attributes without having to rely on custom node blocks. Not just for me but also for the next guy who faces similar issues. :slightly_smiling_face:

2 Likes