Overlap between the MegaDepth data used for training and the MegaDepth1500 data used for evaluation

[kengomitsui]

First of all, thank you for the excellent repository.
I have one question: does XFeat exclude the MegaDepth1500 data during training?

The pairs in assets/megadepth_1500.json that are read at evaluation include items that are not filtered out by the following operation in megadepth.py:

self.pair_infos = [pair_info for pair_info in self.pair_infos
                   if pair_info[1] > min_overlap_score and pair_info[1] < max_overlap_score]

—in other words, those with min_overlap_score = 0.3 or higher.

For example, the pair listed in assets/megadepth_1500.json

"pair_names": [
  "Undistorted_SfM/0015/images/205993922_4ad25ed386_o.jpg",
  "Undistorted_SfM/0015/images/2912685234_7c4c21ee78_o.jpg"
],

corresponds to the indices [356, 898] indicated by npz['pair_infos'][3251] in scene_info_0.1_0.7/0015_0.3_0.5.npz.
The overlap value is 0.3182, so it appears this pair would be included in the default training.

[zh7117]

yes, you are right. The test pairs are in cluded in the training.
In loftr, it has following configuration:
"""
cfg.DATASET.TRAIN_LIST_PATH = f"{TRAIN_BASE_PATH}/trainvaltest_list/train_list.txt"
"""
this file records npzs for training, and it does not contain test scene(0015,0022).

While this code seems load all the the scene for training.

[kengomitsui]

Thanks.
This suggests that the evaluation using MegaDepth1500 in the original xfeat might have been overestimated.