Size of a splat in Gaussian Splattings

Questions
1

Hello,
According to this discussion Location and size of Gaussian Splatings - #3 by Cedric
" if you want a splat radius of 0.5," set its scale to “square root (sqrt(0.5) = 0.70710678118) then divide by 2.”

I am not sure if this holds. In the playground below, I create a splat with the following scale

   const radius = 0.5;
   const scl = new BABYLON.Vector3(Math.sqrt(radius) / 2, Math.sqrt(radius) / 2, Math.sqrt(radius) / 2);
....
   // set splat's scale
   fBuffer[3 + 0] = scl.x; fBuffer[3 + 1] = scl.y; fBuffer[3 + 2] = scl.z;

then I created a sphere of radius 0.5

var sphere = BABYLON.MeshBuilder.CreateSphere(“sphere”, {diameter: 2 * radius, segments: 32}, scene);

and as you can see the sphere is smaller than the splat

Please let me know how I should adjust the radius of the sphere and / or the scale of the splat to see the match?

Thank you,
David

2

let me add @Cedric who should be able to help soon

1 Like
3

@sebavan @Cedric Any updates on this topic?
Thank you

4

@Cedric will be back from summer vacation next week

1 Like
5

sounds good. Thank you

7

I see my mistake! Billboard size is multiplied by 4 and not 2:

So, scl should be computed this way:

const scl = new BABYLON.Vector3(Math.sqrt(radius) / 4, Math.sqrt(radius) / 4, Math.sqrt(radius) / 4);

This said, the single triangle I’m using for the billboard is a bit aggressive and a small portion is not rendered properly. More specifically with more opaque splat. I’m doing a PR to fix that:

3 Likes
8

@Cedric Thank you!!!

In case if someone needs it, here is the updated playground that shows the complete match

1 Like
9

Hi @Cedric
It looks like the math is not working out for splats with scale different from 0.5. In the playground below, I set radius to 5 (see line 40). I continue to compute the spat’s scale as
const scl = new BABYLON.Vector3(Math.sqrt(radius) / 4, Math.sqrt(radius) / 4, Math.sqrt(radius) / 4)
but you can see the size of the resulting splat is much smaller than expected (click to toggle the view)

It seems I am getting more reasonable results if I compute scl as following
const scl = new BABYLON.Vector3(radius / 4, radius / 4, radius / 4);
(no square roots)

Please let me know how I can correctly compute a splat’s scale, based on a target radius

Thank you

10

Hi @DavidP I’ll be back to work in 10days until then, here are a few things you can check:

you can change this value to be (1,1,1) to get the splat without opacity and make comparison easier Babylon.js/packages/dev/core/src/Shaders/ShadersInclude/gaussianSplattingFragmentDeclaration.fx at 447517cdb1e15b7141e5a49a5be070c7ee4755d3 · BabylonJS/Babylon.js · GitHub

The missing part might be in these lines : Babylon.js/packages/dev/core/src/Shaders/ShadersInclude/gaussianSplatting.fx at 447517cdb1e15b7141e5a49a5be070c7ee4755d3 · BabylonJS/Babylon.js · GitHub

It depends on variance wich is computed and send to the shader here : Babylon.js/packages/dev/core/src/Meshes/GaussianSplatting/gaussianSplattingMesh.ts at 447517cdb1e15b7141e5a49a5be070c7ee4755d3 · BabylonJS/Babylon.js · GitHub

See m[0]*m[0] hence the sqrt call. I might have missed a computation in the shader that makes the size not correct.

1 Like
11

It seems size if no a linear function and using Claude, I get :

length(majorAxis) = min(sqrt(2.0 * ((c00 + c11) / 2.0 + sqrt(((c00 - c11) / 2.0)² + c01²) + 0.0001)), 1024.0)

[c00  c01] = T^T * [covA.x  covA.y  covA.z] * T + [kernelSize    0      ]
[c01  c11]         [covA.y  covB.x  covB.y]       [   0      kernelSize]
                   [covA.z  covB.y  covB.z]

You can check that by making a bunch of splat with incremental size value and check visually if the progression is linear or not.

1 Like