Gaussian Splatting With Enable Edges Rendering

Hey guys, I was working with gaussian splatting and I noticed that when edges rendering is enabled the line system seems to ignore z offsets against the splat mesh.

P.S Splats are amazing :slight_smile:

Indeed, depth is not set in the zbuffer for GS. Mainly for performance/bandwidth reason because splats are sorted back-to-front and depth is not needed.
I can add an option to enable depth writing but this might lead to some rendering glitches.
Most of the splat quad is transparent but the splat quad will write to depth. It means lines will be discarded where no GS are visible.

Hey @Cedric thanks for your explanation! I think I get it, I suppose you mean that even though splat ellipse is transparent it would hide the edge completely?

I was using simple line system without enabled edge rendering and lines were hidden behind the splat correctly, this playground seems to work fine, could you explain how those two situations are different?

Line system with edge rendering disabled:


Line system with edge rendering enabled:

I just thought this could be normalised without causing unintended consequences :slight_smile: An optional depth writing probably would be nice to experiment with, I wonder how bad that glitching would get of course.

Here’s the actual screenshot of problem I’m facing… 3D model edges are on top of the splat, while the mesh itself is hidden nicely behind the splat. I use enabled edges rendering to be able to control width and colour of the drawn lines.

I understand that this is a new feature and if this would be detrimental for performance I guess it makes sense to leave that depth buffer intact. I’m also just experimenting a bit at this point.

Off topic. Is there any way to enable shadows for these gaussian meshes? Thanks a lot for your time.

No shadows yet. It’s more or less on our roadmap.
Work around is to change rendergroup id
like this:

I’ve checked rendering with Spector. Line and GS are using blend so there are sorted by depth.
Why is linesystem using blend? I guess edge rendering is assumed to happen at the very last in frame rendering. @sebavan

1 Like

Wow, that’s amazing! Workaround works :smiley: Me and the “Planty” thanks you.

And good news about shadows, that will be awesome! It’s super cool to combine those splats in scenes.

2 Likes

Hey guys, while renderingGroupId did solve the issue at hand, it works for specific situations where splat is always in front of other geometry. It would be great if @sebavan could elaborate a bit on the edge rendering. I went through this explanation here Babylon.js docs and I guess I’m faced with similar situation. Regular meshes can have materials with ‘forceDepthWrite = true’, the splat’s don’t have materials assigned and thus do not have this option. Maybe there’s something hidden that would do the similar thing for splats? :wink:

the GS have a material but it’s set during the first rendered frame.

1 Like

Hi @Cedric thanks for pointing that out! Now I see what you meant with ‘glitches’ in your first response :slight_smile: I’m sharing the result of this experiment here. It does work, splat ellipses hide the edge rendered, then line system remains visible and gets hidden by the splat.

I think it’s really not the issue of the splat, it’s that rendering edges with specified width is difficult. It would be awesome if standard LineSystem would support width, as regular line system seems to work fine. Enabling edge rendering really causes all these issues with depth. I also played with an idea of using one ‘invisible’ splat for hiding the rendered edges through forced depth writing, and the other visible one to remain on screen without forced depth writing to avoid glitching, but that did not work out :smiley: Anyway, I had to give this one more try. Thanks a lot for your time.

1 Like

Ok, so I decided to move from enabled edge rendering to instanced greased lines. I might need to do some work, but greased lines provide consistency due to mesh based nature of rendering.