Error when pushing to challenge branch

[nushakrishnan]

Hello,

Thanks for the great work! I seem to be facing an issue when I push my code to the challenge branch, with regard to the challenge config yaml. I was able to create the Random Number Generator Challenge to test the working of the code. But when I update my evaluation scripts and config yaml file, I face this error. How can I fix this?

Following errors occurred while validating the challenge config:
ERROR: The leaderboard with ID: 2 doesn't exist. Addition of a new leaderboard after challenge creation is not allowed.
ERROR: Challenge phase 3 doesn't exist. Addition of a new challenge phase after challenge creation is not allowed.
ERROR: Challenge phase split (leaderboard_id: 1, challenge_phase_id: 3, dataset_split_id: 2) doesn't exist. Addition of challenge phase split after challenge creation is not allowed.
ERROR: Challenge phase split (leaderboard_id: 1, challenge_phase_id: 2, dataset_split_id: 2) not found in config. Deletion of existing challenge phase split after challenge creation is not allowed.

Thanks!

[Thakor-Yashpal]

Your new config must have the exact same ids for phases and leaderboards as the original. You can't add or remove them, only edit their details (like the script path).

[Thakor-Yashpal]

I think you need to use your original YAML file as the starting point.

Find the config.yaml file you used when you first successfully created the challenge.

In that file, do not add or delete any leaderboard or challenge phase sections. You must keep the exact same number of phases and leaderboards with their original ids.

You can update the fields you intended to change, like the path to your new evaluation_script or the name of a phase.