At Hyperinteractive AI, we’re excited to announce that we are open-sourcing HYP, a new optimized file format for Gaussian Splatting, designed specifically for web applications and eCommerce product representation!
HYP is built to work seamlessly with BabylonJS and provides a lightweight data format for rendering Gaussian Splatting in the browser. HYP is optimized for discrete objects rather than large environments, making it ideal for eCommerce and product showcases.
The key features:
over 2x compression compared to SPLAT files and even higher compression rate for PLY files.
flexibility to specify a desired level of compression for color space.
Easy-to-use API that converts SPLAT, PLY, and BabylonJS Gaussian Splatting mesh objects to and from HYP
Future Expansion:
Currently supports Spherical Harmonics (SH) of order 0
Future plans to support higher-order SH for better shading accuracy
Here are some examples where we’ve used our format
To start, make sure to drag your mouse to spin
Try It Out!
Live Demo on Hugging Face:
Open-Source Code on GitHub:
Please let us know your thoughts, feedback, and any suggestions for future improvements.
Looking forward to hearing what you think!
The purposes of spz and hyp file formats are different since spz is a generic format for 3DGS while hyp is specifically designed for objects. Nevertheless , here is the breakdown
Positions:
spz uses 4 bytes per x,y,z position components
hyp uses 2 bytes per x,y,z position components. How is it possible? Using two bytes, we can represent 65536 distinctive numbers, which translates to 0-65535 range of integer numbers. This range is generally enough to represent an object of size of a person. To take the full advantage of the range, we need to scale it by the bounding box of our Gaussian Splattings object. Thus, mapping a position from hyp representation to conventional one becomes p.x = min.x + (hyp.x / 65535) * (max.x - min.x); p.y = min.y + (hyp.y / 65535) * (max.y - min.y); p.z = min.z + (hyp.z / 65535) * (max.z - min.z); where hyp is hyp 3DGS representation of position, min and max are bounding box coordinates p is the resulting position of 3DGS
Scales and Rotations (Quaternions)
The same - both hyp and spz use one 1 byte per channel
Colors
SPZ uses 4 bytes per color (or 1 byte per channel)
HYP uses about 2.5 bytes per color.
Here’s a full explanation for this unusual number:
HYP offers variable levels of compression for colors by utilizing color lookup tables. This approach takes advantage of the fact that many similar colors exist within any Gaussian Splattings object. For example, a Gaussian Splatting representation of a person wearing a blue dress would have a large number of splats of similar colors within a skin color range and a blue range.
Instead of storing full 4-byte RGBA data for each splat, HYP stores a 2-byte index referring to the color lookup table. (The maximum number of entries in the table is 65,535, as each index is stored in 2 bytes.) Thus, for a Gaussian Splatting object containing N splats, HYP requires:
N x 2 bytes for color indices
Table size, which varies based on the desired precision
In practice, we found that the best balance between quality and memory efficiency is achieved when each unique RGBA color in the table is used by an average of 8 splats. Under this condition:
The table contains N/8 entries, each requiring 4 bytes (RGBA)
The total table size is (N/8) × 4 bytes
=> The total storage needed is N × 2 + (N/8) × 4 = 2.5 × N bytes, or 2.5 bytes per splat
Spherical Harmonics
Currently hyp supports only order 0
Here are the source
SPZ:
HYP:
Let me know if you have any questions / suggestions. Any input welcome
@yamaciller Thank you! Yes, I agree 100% with your observation on spinnable.
However, currently we try to limit how much stuff we have loaded at any given time. Plus sometime we are in situation when GS’s on the front and back intersect. Therefore, we show one at the time.
Regarding SH, the examples that we provided would definitely from SH because surface are shiny. However, we are working on Gaussing Splattings of women’s dresses and we see that SH makes little difference on most fabrics. On the other hand, if a dress has, for example, a shiny leather portion, we would want to have SH greater than 0 there. That leads us to our next optimization - instead of applying a uniform SH order across the entire object, we aim to assign varying SH orders when converting PLY files. (This version is still under development though)
This is very cool! GS is a cool domain to experiment with compression/quantization.
I love your indexed color mecanism.
Did you try to sort splats by color or spatial position then add a delta packing?
I think there is a limit where packing datas too tighly makes a zip compression totally useless.
Another thing to try is to use LZ4 compression. It’s among the best and I believe decompression is very fast.
Sometimes there is not a huge visual difference at the first glance, only direct comparison would reveal, the richness of SHs.
Probably most interesting for photorealistic scenes/objects.
Partial SHs sounds like an intelligent optimization.
Did you try to sort splats by color or spatial position then add a delta packing?
Not yet, but that and zip’ing are definitely on the list of things to try
Our main concern is unpacking part might take long time on legacy smart phones, and thus we would not get benefits from downloading files of smaller size.
BTW We found that the HYP format performs really great when using blobs to download 3DGS data and converting them into Babylon Gaussian Splatting objects on demand. In our case, we download blobs for all dresses, but only create GS objects for those that the user selects
I wonder how it compares to .spz which also has nice compression properties?
