Gaussian Splats - Emerging Potential Improvements

[lxw404]

I'm not totally sure the exact implementation currently used for rendering gaussian splats in Resonite, so this is more of a curiosity and list of potential emerging improvements for Gaussian Splats in general given that they are such a heavy rendering primitive, any sort of improvements offer extra rendering budget.

Here are some promising new developments:

• Speedy Splat - Two key improvements to rendering:
  • Reducing the total number of splats used
  • Better estimation of the tile extent of Gaussians to cull unnecessary regions
• Sort Free Gaussian Splats
  • An attempt to use Order Independent Transparency techniques to prevent the "popping" artifacts present in current rendering methods as well to reduce the total amount of computation done sorting

[Frooxius]

The implementation was actually mentioned in changelog, it's a port of this: https://github.com/aras-p/UnityGaussianSplatting

We use Bitonic sort instead of Radix sort, since the version of Unity we use doesn't support wave intrinsics which are necessary for that.

We might be limited on this until we switch graphics engine, to have access to newer methods. However we can also open this up to community contributions.

[Frooxius]

Also the Speedy Splat is interesting, but since it requires change in training method, this will probably take a while for it to "trickle down" to something we can use - it'll require the 3D reconstruction software to adopt it first.

[lxw404]

Speedy Splat's data reduction does come at the training step unfortunately, however it is based on PUP 3D-GS which allows for significant reduction without modifying the training method.

Also, at least the tile extent estimation does not seem to require any changes to the training method, and it is responsible for a significant portion of the performance increase in that method as well.