Question about repeat subtasks and metrics

[jzhzhang]

Thank you for sharing this benchmark. I would like to evaluate my method on this benchmark. However, I notice that the episode target contains repeated targets.

Is this a bug, or is it intended to function this way? Additionally, I noticed that the scene_id contains hm3d/val//***. Is the double slash // correctly formatted?

Furthermore, I observed that self.config.habitat_baselines.eval.video_option is not defined in the YAML file. Could you provide some guidance on how to render videos for this dataset?

I would greatly appreciate it if you could help me. And thanks again for sharing this dataset.

[jzhzhang]

I also notice that the GoatSuccess uses istance_to_target["prev_distance_to_target"] < self._config.success_distance to decide whether the episode is successful or not, while the GoatSPL uses "distance_to_target" to compute spl (L386 in nav.py). Could you please kindly elaborate more on why consider different types of DTG are for SR and spl?

[NormanMises]

and what do these 7 and 14 mean?

[jzhzhang]

I do not find any specific details about this.

[NormanMises]

and what do these 7 and 14 mean?这些 7 和 14 代表什么意思？

it means the index in image_goals

[SakuraLiHe]

Though these two tasks try to find the same object, the images they use to query are different. They are not repeat targets.