Customize an episode for evaluation in Habitat OVMM

[joanne40226]

Hi, thank you so much for the great work!

I am now working on evaluation in habitat environment.
I found that by
python projects/habitat_ovmm/eval_baselines_agent.py --env_config projects/habitat_ovmm/configs/env/hssd_demo.yaml
it will be evaluated under the ovmm dataset by hssd_demo.yaml
EVAL_VECTORIZED: simulator_gpu_ids: [1, 2, 3, 4, 5, 6, 7] # IDs of GPUs to use for vectorized environments split: val # eval split num_episodes_per_env: null # number of eval episodes per environment record_videos: 0 # 1: record videos from printed images, 0: don't record_planner_videos: 0 # 1: record planner videos (if record videos), 0: don't metrics_save_freq: 5 # save metrics after every n episodes
however, i hope i can evaluate an episode where I can set the "object_category", "start_recep_category" and "goal_recep_category" by myself.
I am not sure how can i achieve this. I wonder if there are any hints or examples?

Thank you.

[WhyxyxIsNotAnAvailableName]

however, i hope i can evaluate an episode where I can set the "object_category", "start_recep_category" and "goal_recep_category" by myself. I am not sure how can i achieve this. I wonder if there are any hints or examples?

Thank you.

Hello @joanne40226 , recently I've also encountered this issue. I also want to create my own episodes using my scene files and object files for training and evaluation. Have you solved this problem or found any hints or examples?

Thank you very much for your help.

[yvsriram]

You can change it here:

home-robot/src/home_robot_sim/home_robot_sim/env/habitat_ovmm_env/habitat_ovmm_env.py

Line 226 in ede6a67

def _preprocess_goal(

You can look up self._obj_id_to_name_mapping and self._rec_id_to_name_mapping here for mapping ids to names.