IoU Calculation Bug: IoU > 1 in Some Cases

[npurson]

When running calculate_iou with two depth maps, the computed IoU occasionally exceeds 1, which is mathematically incorrect. I suspect the issue originates from datasets_preprocess/scannetpp_preprocess.py#L200:

union = torch.sum(valid_mask) + torch.sum(depth2 > 0) - intersection

Here, torch.sum(depth2 > 0) counts all non-zero depth points in depth2, rather than only those corresponding to the projected depth1 pixels. As a result, this can underestimate union, leading to an IoU value greater than 1.

To correct this, I believe the calculation should be adjusted as follows:

valid_depth2_mask = depth2[pixels_img2[:, 1], pixels_img2[:, 0]] > 0
union = torch.sum(valid_mask) + torch.sum(valid_depth2_mask) - intersection

[Jian-Zhang-3DV]

You're right—IoU can exceed 1 in the current calculate_iou implementation, which is invalid (IoU must be in [0, 1]). The issue stems from union = torch.sum(valid_mask) + torch.sum(depth2 > 0) - intersection, which underestimates the union when multiple depth1 points project to the same depth2 pixel, inflating IoU (e.g., IoU = 3 when 3 points overlap a single pixel).

The fix requires a pixel-wise approach using binary masks:

• overlap_mask_2d: Pixels where projected points match depth2 within the threshold.
• projected_mask_2d: Pixels with any projected points.
• Intersection = torch.sum(overlap_mask_2d).
• Union = torch.sum(projected_mask_2d | (depth2 > 0)).

Here’s a corrected snippet:

overlap_mask_2d = torch.zeros((h, w), dtype=torch.bool, device='cuda')
projected_mask_2d = torch.zeros((h, w), dtype=torch.bool, device='cuda')
for i in range(pixels_img2.shape[0]):
    px, py = pixels_img2[i, 0], pixels_img2[i, 1]
    projected_mask_2d[py, px] = True
    if depth_diff[i] < depth_threshold:
        overlap_mask_2d[py, px] = True
intersection = torch.sum(overlap_mask_2d)
union = torch.sum(projected_mask_2d | (depth2 > 0))
iou = intersection.float() / union.float() if union > 0 else 0.0