Grey colored voxels in 3D reconstruction

[saichand44]

Hello, I am trying to perform 3D reconstruction using RGB images, depth images, camera intrinsics and extrinsics. I want to perform the 3D reconstruction using limited number of frames (~ 25) for my use case and started experimenting by extracting the pointcloud (verticels, color) for one frame only.

I observed that the TSDF Layer gives pretty good results for voxel positions even with one frame, but the Color layer is grey at some voxels (particularly around the boundary of the objects) as shown in the picture.

My understanding so far:

1. TSDF and Color layer are treated independently in the backend and hence accumulate different weights during the integration process.
2. When using only one frame, TSDF Layer has enough weights in the voxels during the mesh integration (defaulted to grey color (127, 127, 127)) but the Color Layer does not have enough weights in the voxels at the highlighted regions to assign rgb value.

Questions:

1. Are there any parameters I can tune to handle different minimum weights for different layers during integration, so that rgb values are assigned to all voxels visible from rgb image?
2. Does ViewCalculatorParams play a role in the grey voxels by limiting the camera FOV for integration?

Please let me know any leads to handle this issue.

[deivse]

Hi, have you changed MapperParams().projective_integrator_params.projective_integrator_max_integration_distance_m by any chance? I think there's a bug which doesn't register color updates beyond the default max integration distance, even if the parameter is updated after constructing the color layer.

Even after fixing this, there are still gray voxels, but to a much lesser extent.