Scene reconstruction and mask reconstruction of objects in the scene？ #86
Description
Hello, thank you very much for your open-source contribution. Your work is excellent. However, I encountered some confusion while learning and using 3DGS, and I would appreciate your insights.
In our tests on a self-collected dataset, we observed the following:
When we input 100 scene images, 3DGS can successfully reconstruct the entire scene, including the target object, with good quality.
However, when we first use SAM masks to extract the target object from these 100 images and then perform 3DGS reconstruction using only these object images, the process fails. Specifically, the convert step in COLMAP reports failure, with only 2 images successfully matched.
Even when we use the 100 masked images with COLMAP (sparse reconstruction) or MASt3R and then apply your method, the reconstruction quality is still worse than directly reconstructing the entire scene with the same number of images. The resulting model of the object is incomplete (e.g., the visual hull has missing regions).
Why might this happen? Scenes can be reconstructed well, and the target object surfaces also show many Gaussian points in the scene-level reconstruction. But under the same number of views, extracting only the object masks seems to fail, and even with your method, the reconstruction cannot reach high quality. Could you provide some suggestions to help us improve mask extraction of objects and achieve accurate colmap reconstruction?