Shader Lightmap Final Color Contributions Seem Wrong

MackeyK24 · May 9, 2020, 8:23pm

Hey guys… I noticed something a little weird in the lightmap texture final color contributions (especially compared to unity).

In unity… The other lights in the scene DONT affect geomtery that has a baked lightmap… But in BabylonJS… Even ambient light (which is basically an additional hemispheric light in the scene to fake ambient lighting) is being applied to to the baked lightmap color…

Seems to me, that in the Shaders, the lightmap section should be AFTER the lighting section but BEFORE the fog section.

What do you think about that ???

@Deltakosh and @sebavan ???

Evgeni_Popov · May 9, 2020, 9:47pm

Lightmaps can be seen as an additional way to account for indirect lighting, but it does not preclude other types of indirect lighting. Ambient lighting is another indirect lighting (even if a crude one), that’s why it adds to the lightmap.

However, you can prevent the direct lighting to add to the lightmap lighting by setting light.lightmapMode = Light.LIGHTMAP_SHADOWSONLY. With this setting, only shadows (if there’s is a shadow generator declared for this light) will apply, you won’t get any specular/diffuse contribution. You can also use Light.LIGHTMAP_SPECULAR to get only the specular part of the direct lighting and not the diffuse contribution.

MackeyK24 · May 9, 2020, 10:23pm

I was using Light.LIGHTMAP_SPECULAR and Light.LIGHTMAP_SHADOW_ONLY on the hemispheric ambient light… Which worked great for the meshes with a lightmap… But then the other meshes in the scene DONT get the full ambient lighting (And they should)

…

I think setting an unlit mode on the material if lightmapTexture != null …

But i noticed the pbr shader fragment for #UNLIT is missing the #LIGHTMAP section that has been moved to pbrBlockDirectLight.fx…

But the pbtFinalColorComposition.fx does have support for the lightmap…

// _____________________________ LightMappping _____________________________________
#ifdef LIGHTMAP
    #ifndef LIGHTMAPEXCLUDED
        #ifdef USELIGHTMAPASSHADOWMAP
            finalColor.rgb *= lightmapColor.rgb;
        #else
            finalColor.rgb += lightmapColor.rgb;
        #endif
    #endif
#endif

So my question is… shouldn’t pbrMaterial.unlit = true … still at least support lightmaps ???

If so we can just add to the #define UNLIT section’s if statement:


#ifdef UNLIT
  vec3 diffuseBase = vec3(1., 1., 1.);

  #ifdef LIGHTMAP
    vec4 lightmapColor = texture2D(lightmapSampler, vLightmapUV + uvOffset);

    #ifdef RGBDLIGHTMAP
        lightmapColor.rgb = fromRGBD(lightmapColor);
    #endif

    #ifdef GAMMALIGHTMAP
        lightmapColor.rgb = toLinearSpace(lightmapColor.rgb);
    #endif
    lightmapColor.rgb *= vLightmapInfos.y;
  #endif
#else

...

#endif

in the pbr.fragment.fx shader

Evgeni_Popov · May 9, 2020, 10:40pm

I don’t think so. unlit means no lighting at all, and lightmap is a kind of light source.

I will let @Deltakosh / @sebavan answer regarding the fact that we may allow to remove the ambient lighting when using lightmaps.

Regarding the block in the final color compositing:

// _____________________________ LightMappping _____________________________________
#ifdef LIGHTMAP
    #ifndef LIGHTMAPEXCLUDED
        #ifdef USELIGHTMAPASSHADOWMAP
            finalColor.rgb *= lightmapColor.rgb;
        #else
            finalColor.rgb += lightmapColor.rgb;
        #endif
    #endif
#endif

I think it’s a mistake and it should be:

// _____________________________ LightMappping _____________________________________
#if defined(LIGHTMAP) and !defined(UNLIT)
    #ifndef LIGHTMAPEXCLUDED
        #ifdef USELIGHTMAPASSHADOWMAP
            finalColor.rgb *= lightmapColor.rgb;
        #else
            finalColor.rgb += lightmapColor.rgb;
        #endif
    #endif
#endif

Else, in unlit mode with a lightmap enabled it will crash, because the lookup in the lightmap texture (variable lightmapColor) is done only when unlit=false (in the pbrBlockDirectLighting.fx file).

Evgeni_Popov · May 9, 2020, 11:11pm

My mistake, your ambient is the HemisphericLight whereas I’m speaking of the ambient term in the light calculation…

What you want is that the contribution of the HemisphericLight is 0 when there’s a lightmap (more precisely, take only the value read from the lightmap), something that is different from just allowing to not take into account the ambient term in the light calculation: the diffuse term would still contribute, as well as the indirect lighting terms…

Let’s see what the core team think about this!

[…]

However, I’m surprised about this:

It should work, see this PG: https://playground.babylonjs.com/#J8B2ID#1

The ground has a material with a lightmap.

The hemispheric light is set with lightmapMode = Light.LIGHTMAP_SHADOWSONLY so the ground is only “lit” by the lightmap.

The sphere has a material without lightmap and is correctly lit by the light (else it would be all black).

Change the light intensity: you will see it does not affect the ground but it does affect the sphere.

sebavan · May 9, 2020, 11:20pm

yup would be cool to have a PR for it

Why not using the include and exclude for lights than need the hemi only ?

Evgeni_Popov · May 9, 2020, 11:24pm

In fact, as said in the latest part of my comment, it seems it should already work as expected with the existing state of things.

Let’s see @MackeyK24 comment about this.

MackeyK24 · May 10, 2020, 1:54am

I get what your saying about BABYLON.Light.LIGHTMAP_SPECULAR

I guess what i am saying…i need the ambient (hemispheric) light to be SPECULAR on items with a lightmap and DEFAULT on items without a lightmap…

So i would have to use TWO ambient lights… One with Specular lightmap mode and other with Default lightmap mode…

I would also have to include/exclude meshes to each ambient light depending on if it is lightmapped

MackeyK24 · May 10, 2020, 3:55am

excludeWithLayerMask is a solution

DefaultAmbientLight = new BABYLON.HemisphericLight("Ambient Lighting", new BABYLON.Vector3(0, 1, 0), this._loader.babylonScene)
DefaultAmbientLight.falloffType = BABYLON.Light.FALLOFF_STANDARD;
DefaultAmbientLight.lightmapMode = BABYLON.Light.LIGHTMAP_DEFAULT;
DefaultAmbientLight.excludeWithLayerMask = BABYLON.SceneManager.LightmapLayerMask;

Then the parser will check each mesh entity is lightmapped

if (mesh.hasLightmap) {
  mesh.layerMask |= BABYLON.SceneManager.LightmapLayerMask
}

Evgeni_Popov · May 10, 2020, 11:07am

Glad you found the setting you needed, but:

is what you get with lightmapMode = BABYLON.Light.LIGHTMAP_SPECULAR (ground has no lightmap, sphere has lightmap - the lightmap is the texture you can see on the sphere):

lightmapMode is DEFAULT:

lightmapMode is SHADOWSONLY:

lightmapMode is SPECULAR:

MackeyK24 · May 10, 2020, 11:24am

Yo @Evgeni_Popov … Here is what i am going for… Unity —> BabylonJS … Full Lighting Exported…

Example scene with the cube is the the ONLY real time lit object. The directional light is blue and the ambient light is green

Everything else is baked lighting

All the HDR (RGBD) Skybox textures
All the HDR (RGBD) Lightmap textures
All the Baked Lighting (Including Ambient Occlusion)
All the Baked Shadows
Everything

Note: the only difference between my fully baked Unity Scene and my Babylon Toolkit Scene is the final runtime shadow color contributions… In unity , they are still picking up the scene Ambient Light Color for the cube runtime shadow… But i can live with not having that shadow so light green… My Babylon Scene Looks Better Anyway

And all the lightmaps are packed into a single 4K lightmap texture atlas (Exported as a jpeg just for smaller image to upload) with all the uv2 tex coords updated for the proper scale and offset within the texture atlas… Beautiful

Thanks again @Evgeni_Popov

Deltakosh · May 11, 2020, 3:33pm

This looks REALLY good man! You rock

MackeyK24 · May 11, 2020, 8:00pm

Thanks @Deltakosh

twhittingham · June 4, 2020, 7:47am

Hi MackeyK24
This looks great, I am wondering if you can help me I just posted a unity question.
My lightmaps that i bake in unity, and export my scene with the babylon exporter. The lightmap looks different, the colours are more green and doesnt look correct, but the lightmap texture looks fine. Just wondered if you have any suggestions unity or babylon side that we are missing.
Regards
Terry

MackeyK24 · June 4, 2020, 12:07pm

Can you post details and screen shots so i can see what you are talking about ?

FYI: The current version of the exporter doesnt do lightmaps that great… I think that version has to do with the actual light and its lightmapMode… Depends on Realtime, Mixed Or fully baked light.

So if you baked with mixed light options… you need to set your direction light in th babylon scene to either LIGHTMAPMODE SPECULAR OR SHADOWONLY… Try that

Now i fixed all this stuff in the new GLTF version of the toolkit… But you will have to wait a bit until i get the new Babylon Toolkit Pro Edition Ready

Topic		Replies	Views
Unity Babylon Exporter Lightmap problem Questions unity-exporter	19	2177	November 20, 2020
50% gray for bake source of tinted unlit shading Content creation (3dsMax, Maya, Blender, glTF)	14	760	February 17, 2023
About lightmap, there is something strange Questions	2	215	December 21, 2022
Weird lightmap texture error Bugs	3	438	January 27, 2020
Issues lighting a blender exported model with baked texture Questions	3	716	December 14, 2020

Shader Lightmap Final Color Contributions Seem Wrong

Related Topics