Artifacts over time

[SoylentGraham]

I'm trying to get a better handle on the code that's refining splats; (Running on MPS, with libtorch 2.7 and 2.1)

After 1250 steps, things aren't looking bad from a validation camera

Shortly after this, I'm getting more and more purple splats...

and then at 3000 - I've now got some brightly coloured splats (In my renderer, these are lumious, but not as solid as in the render here - will try and dig)

Can you tell me if the purple (and solid) splats are likely down to the forward/rendering producing odd results and the model running off that, or if it's likely to be the other way around?

Similarly, I'm comparing the output with 1000 runs in Brush
Their splats are far smalle, less elongated and more sensible hues - are these likely to come from new splats added in, or growths that have gotten a bit out of hand? (Can you point me to this area of the shader?)

[SoylentGraham]

Given the amount of grey - purple and green can easily come from a tiny variance of hue in the grey and being exaggerated.... there may be an easy win here... if I can figure out where in the code it's coming up with new/changing splat colour/sh

[SoylentGraham]

https://github.com/pierotofy/OpenSplat/blob/main/model.cpp#L185

rgb = torch::clamp_min(rgbs + 0.5f, 0.0f);

This could be the source of over-bright splats... no clamp to 1.0 (Though I need to read more of the renderer to see if that's really a problem)

edit: Oh, there's a max-clamp to 1.0 at the bottom of the func :)

[SoylentGraham]

hmmm, just realised the initial step renders are purple; maybe there's something wrong with the bg colour

edit:

backgroundColor = torch::tensor({0.6130f, 0.0101f, 0.3984f}, device).requires_grad_(); // Nerf Studio default

I guess that means the purple results above may be holes, rather than splats...

[SoylentGraham]

More interesting artifacts - this time when my code is reporting 3 NaNs in scale data.

I thought perhaps it may be a tile-render issue, but maybe a badly rendered point makes sense

[SoylentGraham]

Just another example at 6550 steps

(Pretty though! :)

[SoylentGraham]

This seems to happen when I have malloc scribble enabled... ASAN isn't picking anything up, but maybe there's some tensor corruption happening, and its all in libtorch's allocation/heap (which is why libs shouldn't do their own memory management! :P)

edit: malloc scribble is causing some massive gaussians of just green or pink (and for some reason not being culled) - but after 2000 steps or so, I'm getting these artificats still