Parameter simplifications

[oliver-batchelor]

Hi there,

Thanks for your great piece of work - I'm keen to try simplifying a few things and interested in your opinion:

First one is relatively simple:
There's lots of code like this where there's a parameter:
in_camera_grad_color_buffer: ti.types.ndarray(ti.f32, ndim=2), # (M, 3)

Then some code later:

      point_grad_color = ti.math.vec3(
           in_camera_grad_color_buffer[idx, 0],
           in_camera_grad_color_buffer[idx, 1],
           in_camera_grad_color_buffer[idx, 2],
       )

But did you realise you can declare the parameter like this?
Directly creating the vec3 instead:

in_camera_grad_color_buffer: ti.types.ndarray(ti.math.vec3, ndim=1), # (M, 3)

Second one is potentially packing parameters into vectors a little like how you did the Gaussian3D struct, so instead of having a bunch of input parameters:

    point_uv: ti.types.ndarray(ti.f32, ndim=2),  # (M, 2)
    point_in_camera: ti.types.ndarray(ti.f32, ndim=2),  # (M, 3)
    point_uv_conic: ti.types.ndarray(ti.f32, ndim=2),  # (M, 3)
    point_alpha_after_activation: ti.types.ndarray(ti.f32, ndim=1),  # (M)
    point_color: ti.types.ndarray(ti.f32, ndim=2),  # (M, 3)

They could be packed into a ti.types.ndarray(vec12, ndim=1) and unpacked into a Gaussian2D struct, a few helper abstractions can simplify it to avoid creating even more boilerplate...

Thanks!
Oliver

[oliver-batchelor]

FWIW: Along this line I've been been making a derivative taichi_3d_gaussian_splatting (just the rasterizer for now), key idea is to split everything into parts, so that it's easy to say, replace spherical harmonics with something else. I don't think it loses much efficiency in doing so but gains a lot of flexibility.

https://github.com/uc-vision/taichi_gaussian_rasterizer