Create Spherical Polynomial Lighting

[MackeyK24]

Yo @Deltakosh or @sebavan … i am having trouble creating spherical polynomial diffuse ibl lighting from harmonics array i serialize from Unity:

"sh": [
                0.180283263,
                -0.00568266725,
                -0.0125605986,
                0.00725202262,
                0.00569990976,
                -0.009872374,
                0.01033011,
                -0.0104362741,
                0.0250354819,
                0.225713328,
                0.0413432159,
                -0.0197532848,
                0.011403271,
                0.00911192,
                -0.0157832224,
                0.0143798534,
                -0.01522373,
                0.0344359651,
                0.306922019,
                0.127484113,
                -0.0340535268,
                0.0196564719,
                0.0163650271,
                -0.02834928,
                0.017715618,
                -0.02208532,
                0.0404653475
              ],

I try to create the diffuse lighting like this… Are you supposed to be manually create IBL Diffuse Lighting ?

                // Extract RGB channels from the Unity SH array (9 coefficients per channel)
                const rCoeffs = shCoeffs.slice(0, 9);   // Red channel
                const gCoeffs = shCoeffs.slice(9, 18);  // Green channel
                const bCoeffs = shCoeffs.slice(18, 27); // Blue channel
                
                // Create spherical harmonics from Unity SH data
                const sh = new BABYLON.SphericalHarmonics();
                
                // Set the SH coefficients directly on the harmonics structure
                // Unity order: L00, L1-1, L10, L11, L2-2, L2-1, L20, L21, L22
                sh.l00 = new BABYLON.Vector3(rCoeffs[0], gCoeffs[0], bCoeffs[0]);
                sh.l1_1 = new BABYLON.Vector3(rCoeffs[1], gCoeffs[1], bCoeffs[1]);
                sh.l10 = new BABYLON.Vector3(rCoeffs[2], gCoeffs[2], bCoeffs[2]);
                sh.l11 = new BABYLON.Vector3(rCoeffs[3], gCoeffs[3], bCoeffs[3]);
                sh.l2_2 = new BABYLON.Vector3(rCoeffs[4], gCoeffs[4], bCoeffs[4]);
                sh.l2_1 = new BABYLON.Vector3(rCoeffs[5], gCoeffs[5], bCoeffs[5]);
                sh.l20 = new BABYLON.Vector3(rCoeffs[6], gCoeffs[6], bCoeffs[6]);
                sh.l21 = new BABYLON.Vector3(rCoeffs[7], gCoeffs[7], bCoeffs[7]);
                sh.l22 = new BABYLON.Vector3(rCoeffs[8], gCoeffs[8], bCoeffs[8]);
                
                // Unity SH coefficients need to be scaled - they're not pre-scaled
                // Let Babylon.js apply its own scaling during FromHarmonics conversion
                sh.preScaled = false;
                
                // Scale by IBL intensity if needed
                if (iblIntensity !== 1.0) {
                    console.log("Scaling SH coefficients by IBL intensity:", iblIntensity);
                    sh.scaleInPlace(iblIntensity);
                }
                
                // Convert to spherical polynomial using Babylon's conversion
                const sp = BABYLON.SphericalPolynomial.FromHarmonics(sh);
                
                // Apply to the cube texture using public API
                babylonScene.environmentTexture.sphericalPolynomial = sp;
                
                // Force ALL materials to pick up the new environment diffuse IBL path
                babylonScene.markAllMaterialsAsDirty(BABYLON.Material.AllDirtyFlag);  

I get no ambient light at all, amd i creating it right ?

[sebavan]

Can you share a repro in the playground so we have a look ?

[MackeyK24]

Yo @sebavan
Here is playground, I set the create prefiltered call to not create the polynomials, so i can supply them code:

https://playground.babylonjs.com/#C1SETK#2

[sebavan]

Thanks for the repro, it seems to be a timing issue where the poly you set are erased later on.

I ll fix it tomorrow as we should also not try to generate them in the first place.

I want a nice way to setting this up.

[sebavan]

This should fix the setter issue: Fix IBL Manual setup by sebavan · Pull Request #17563 · BabylonJS/Babylon.js · GitHub

I ll let you address the rest which is app specific

[MackeyK24]

Yo @sebavan
Not sure i see the difference, Do I still set createPolynomials to false in the CubeTexture constructor. Because i am going to manually set the envTex.sphericalPolynomials later ?

Seems like that local createPolynomials need to be true, so they get created on construction and then later overwriteten with direct call to set the envText.sphericalPolynomials… or am I readng that wrong ?

[sebavan]

I am simply using your playgound which goes from

to

https://playground.babylonjs.com/?snapshot=refs/pull/17563/merge#C1SETK#2

[MackeyK24]

Hmmm… i would expect it to be more blueish overall, something like this:

It must be something with the packing of the RGB order. Maybe it doesnt like my coeffecient packing from unity.

        public static float[] ExportAmbientProbeSH27()
        {
            SphericalHarmonicsL2 sh = RenderSettings.ambientProbe;

            var arr = new float[27];
            int i = 0;

            // Unity coeff index order 0..8 is L00, L1-1, L10, L11, L2-2, L2-1, L20, L21, L22
            // (Unity exposes coeffs via sh[rgb, coeff]) :contentReference[oaicite:6]{index=6}
            for (int rgb = 0; rgb < 3; rgb++)
                for (int c = 0; c < 9; c++)
                    arr[i++] = sh[rgb, c];

            return arr;
        }

That is how i us the ambient probe in unity to grab the true ambient lighting from the scene, along with global environment texture .dds for specular reflections. giving me much richer diffuse ibl for the ambient lighting and of course the same global reflection texture from the unity scene.

Anyways, thanks for fixing the direct assignment of the polynomials. I will fart around with packing the ambient probe coeffecients

[MackeyK24]

Yo @sebavan … So it looks like the root issue is that the polynomials are already prescaled.

The key is your fix, plus the color space issue is the Unity SH are already preScaled:

harmonics.preScaled = true;

https://playground.babylonjs.com/?snapshot=refs/pull/17563/merge#C1SETK#5

This is the Unity Default Skybox reflection texture and now using PREBAKED Unity Spherical Harmonics instead of at runtime scanning the cubmap faces to calculate the polynomials…

I think its working

[MackeyK24]

You know what would be really kool. You can have per-material spherical polynomials. That is great. I am now working on export of the Unity Light Probes, which is basically the light probe network positions and SH at that position:

Trip this code to basically export all the Unity Light Probe Info.

        /// <summary>
        /// Exports baked Light Probe network:
        /// - positions: float[probeCount * 3]  (x,y,z per probe)
        /// - sh:        float[probeCount * 27] (RGB blocks, 9 coeffs each)
        /// Coeff order per channel is Unity's SphericalHarmonicsL2 index 0..8:
        /// L00, L1-1, L10, L11, L2-2, L2-1, L20, L21, L22
        /// </summary>
        public static Dictionary<string, object> ExportLightProbeNetwork()
        {
            var result = new Dictionary<string, object>();

            var lightProbes = LightmapSettings.lightProbes;
            if (lightProbes == null)
            {
                result["positions"] = Array.Empty<float>();
                result["sh"] = Array.Empty<float>();
                result["count"] = 0;
                result["note"] = "No LightProbes found (LightmapSettings.lightProbes == null).";
                return result;
            }

            Vector3[] positions = lightProbes.positions;
            SphericalHarmonicsL2[] baked = lightProbes.bakedProbes;

            if (positions == null || baked == null || positions.Length == 0 || baked.Length == 0)
            {
                result["positions"] = Array.Empty<float>();
                result["sh"] = Array.Empty<float>();
                result["count"] = 0;
                result["note"] = "LightProbes present, but positions/bakedProbes are empty. Did you bake lighting?";
                return result;
            }

            int count = Mathf.Min(positions.Length, baked.Length);

            // positions: [x,y,z,x,y,z,...]
            var posOut = new float[count * 3];
            for (int i = 0; i < count; i++)
            {
                Vector3 p = positions[i];
                posOut[i * 3 + 0] = p.x;
                posOut[i * 3 + 1] = p.y;
                posOut[i * 3 + 2] = p.z;
            }

            // sh: [R0..R8, G0..G8, B0..B8] per probe, concatenated
            var shOut = new float[count * 27];
            int w = 0;
            for (int i = 0; i < count; i++)
            {
                SphericalHarmonicsL2 sh = baked[i];

                for (int rgb = 0; rgb < 3; rgb++)
                {
                    for (int c = 0; c < 9; c++)
                    {
                        shOut[w++] = sh[rgb, c];
                    }
                }
            }

            result["count"] = count;
            result["positions"] = posOut;
            result["sh"] = shOut;

            // Helpful metadata for your loader:
            result["shStride"] = 27;
            result["posStride"] = 3;
            result["space"] = "Unity world space (x,y,z)";

            return result;
        }

I can TRY can make a light probe network in babylon for dynamic moving objects, by swapping out the object material clone, reflection texture spherical polynomials…At least in theory.

Should be a kool experiment to see if I can export unity light probe metadat and create a dynamic light probe system in Babylon Toolkit generated scenes

[sebavan]

You can do that by assigning a reflectionTexture to the material. It should allow each materials to have a different setup. We only use scene.environmentTexture when there are no reflectionTexture being set.

[MackeyK24]

Hey @sebavan … The broken spherical polynomials are back with WEBGPU.

In webgpu mode , the spherical polynomials create from SH doe snot work… its all black on WebGPU:

https://playground.babylonjs.com/?webgpu#RNE00Y

[Evgeni_Popov]

The same fix @sebavan did for WebGL2 has been ported to this PR for WebGPU:

FrameGraph: Support multi-RTT in clear texture and object renderer tasks

The problem will be fixed when the PR is merged.

[MackeyK24]

@Evgeni_Popov

FrameGraph: Support multi-RTT in clear texture and object renderer tasks by Popov72 · Pull Request #17582 · BabylonJS/Babylon.js · GitHub

Yo @sebavan or @Evgeni_Popov …
Any idea when they gonna make a new release with the merge for the proper polynomials create or not to create ?

[sebavan]

It should be there on the 1st, and if not I ll publish one Friday. Does it fit your schedule ?

[MackeyK24]

Thank you very much

[MackeyK24]

Yo @sebavan / @Evgeni_Popov i dont know if the ability TO CREATE or NOT CREATE spherical polynomials… which i beleive is the IBL diffuse lighting. i fixed in 8.44.0

Its is hard to replicate … it seems to work on playground. But NOT in my app it is not.

This is a snapshot of my code to load cube map… its remarkard out. just to show its this one line of code to create a BABYLON.CubeTexture and passing createPolynomials: false:

And what you get.. as expected no ibl lighting (The buildings are lightmapped)

But when i enable that one line:

AND FORCE createPolynomials = false…

It still has all kind of AMBIENT lighting

And is it has generated spherical polynomials… I would expect those to be ZERO or BLACK

…

I cant explain it… Is there ANYTHING else that causes the IBL ambient light to be create besides the createPolynomials flag, which does not appear to be working as i would expect it to…

[MackeyK24]

Its interfering with the real Unity Spherical Harmonics i wanna use that I export from Unity.

I want to use Unity’s baked/authoritative ambient probe lighting (e.g., it includes Unity’s lighting bake, probe occlusion decisions, exposure chain, etc.) as my main global environment lighting.

I dont see it reflected in my scene, i see the orginal spherical polynomials lighting, that should not even be there, because it was created with createPolynomials = false.

Something is still causing all the default ibl diffuse lighting an NOT allowing me to overwrite. For example: scene.environmentTexture.sphericalPolynomials = new BABYLON.SphericalPolynomials() would breate new all BLACK polynomials, BUT IT DOES NOT CHANGE that original ibl when the cube was created.

What the heck could be influencing that obl lighting like that for me ???

[MackeyK24]

Hmmm… Yo @sebavan and @Evgeni_Popov .. its is something t to do with loading a prefiltered .env vs .dds.

With the prefiltered .env it is always creating the sphericalPolynomials and you cant even override them with new spherical polynomials…

But when you use a prefiltered .dds… IT DOES respect the createPolynomials true or false and you can overide later with new or black spherical polynomials.

This is what i expect when createPolynomials = false… it shows the specular reflections but NO IBL Diffuse… Perfect so far…

So something with the prefiltered .env. on my system using 8.44.0 kinda LOCKS in sphericals polynomials on creation and does not let you change it

[sebavan]

.env contains them it does not need to create them so you could place them to 0 in the file manually ?

Or you for them back to to 0 onload ?