How do you deal with the distractor points included in the initial Gaussian point cloud?

[houchenfeng]

Could you explain how you deal with distractor points included in the initial Gaussian point cloud?

I'm trying to reproduce your code. When processing real-world captured scenes, I found that when using the SfM point cloud obtained from COLMAP sparse reconstruction, if there are similar distractors in adjacent views, some points from these distractors may pass the SfM consistency check. As a result, the initialization point cloud for static Gaussians will contain points originating from distractors. This leads to artifacts in the static scene, as seen in the "spot" scene from the NeRFOnTheGo dataset.

Simple downsampling or random initialization of the point cloud doesn't resolve this issue.

How did you address it? This seems to significantly impact your method's final results.

[johanna0626]

Hi! Thanks for your interest. This is indeed our limitation, which we have stated in our paper. This issue is because our method learns to distinguish distractor and static objects via photometric loss. If there are same distractors shared by many frames it may cause confusion to the model. This case is apparent in "Patio" scene, but in "Spot" scene, our method should perform well compared to other baselines.
I reran the experiment. Although the initialization involves some randomness, the results should be reproducible. I have uploaded the script for preprocessing the on-the-go dataset. You might use this preprocessing file and check the result again.
You could also double check whether the input image downsampling rate is consistent with that used in the other baselines.