New feature: Niantic .SPZ Gaussian Splatting and Spherical Harmonics

Announcements
1

Niantic .SPZ Gaussian Splatting file import and Spherical Harmonics support has been added to Babylon.

More background on SPZ: GitHub - nianticlabs/spz: File format for 3D Gaussian splats. About 10x smaller than the PLY equivalent with virtually no perceptible loss in visual quality. Offered as open source by Niantic Labs. More details at https://scaniverse.com/spz #3dgaussiansplats #gaussiansplatting

You can test it with the snapshot and it will be available later today with the Playground

image
image1261×1088 111 KB

Test PG:

SH from .PLY are not supported yet. The file size is ridiculous in that configuration so it’s way better to use .SPZ.

PR is here : SPZ splat file loader by CedricGuillemet · Pull Request #15849 · BabylonJS/Babylon.js · GitHub

19 Likes
2

Very cool! Thanks!! :sunglasses:

1 Like
3

Hi.
We were looking forward to that!
Amazing.
We have created a converter for spz format and wanted to check if everything is correct.
Atm we see some vey obvious quality loss in our tests, but we are not sure if that is the compression or we do something wrong.
Do you have the original .ply dataset of the gekko scene above to compare (at least visually) the .ply (or .splat) and the .spz scene.
When zooming close to the surface of the sculpture we can see kind of the same issues appear here.
But we want to investigate deeper.
Thanks and best.
Werner

4

Hi @yamaciller
I’ve fixed not later than yesterday an issue with splat orientation and 1 with better color conversion

I could not see any difference between ply and spz.
Also, for spz, take care of the coordinate system : Babylon.js docs

1 Like
5

Amazing!
Yes I think, that did the deed.
We are comparing, but it looks promising.
Btw. The .ply view in Babylon for GS doesnt carry through the SHs or?
Best.
Werner

6

No, for legacy reason. When .ply started to be used, filesize was an issue and SH support was dropped.
Then came .spz and SH was less an issue but its parsing was added for .ply.

1 Like
7

Is spherical harmonic support only available with webGPU? @Cedric

8

This works in webgl on my android phone, but on my MacBook only webGl has no spherical harmonics

9

There was an issue with Mac + Chrome + SH that’s been fixed with bind eye position GS by CedricGuillemet · Pull Request #16466 · BabylonJS/Babylon.js · GitHub

10

Thank you! I see some other fixes too related to SH, excited to try it out. Is there also now support for compressed ply?

11

Is there a build out for this yet?

12

Compressed PLY has been supported for some time now so it’s been available for many npm releases.

13

Okay I will test today. I could have sworn it was broken for me and looked like a bunch of random splats in random colors all over-- could be a bug or maybe my version was outdated, I will keep you posted.

Do you know if there was an npm release for the other spherical harmonics fixes mentioned above?

The changes seem to be in the preview CDN, so I hosted that version myself and reference it for StorySplat exports which work great but my editor runs off the npm package and that doesn’t seem to have the changes necessary for the SH to look right – I could just be missing something here on how to get the latest in my node moduels

Thank you!

14

Here is the same file loaded as a compressed spz in the latest playground – https://playground.babylonjs.com/#KUHXDT#13

Am I missing something?

16

Please note that @Cedric will be oof for a bit so bear with us in the meantime

17

Of course, thank you for the heads up and for all of your help/work as always!

1 Like