-
Notifications
You must be signed in to change notification settings - Fork 756
[Bug] IMO-Bench-AnswerBench 评测无数据集下载链接 #2405
Description
Prerequisite
- I have searched Issues and Discussions but cannot get the expected help.
- The bug has not been fixed in the latest version.
Type
I'm evaluating with the officially supported tasks/models/datasets.
IMO-Bench-AnswerBench 评测时,在 datasets_info.py 的 DATASETS_URL 字典中缺少 imo-bench-answerbench 的下载配置。
报错信息如下:
INFO - ./data/imo-bench-answerbench.jsonl does not exist!Start Download data automatically!If you have downloaded the data before,You can specific COMPASS_DATA_CACHE to avoid downloading~
Traceback (most recent call last):
File "/u01/miniforge3/envs/open-compass/bin/opencompass", line 6, in
sys.exit(main())
^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/cli/main.py", line 349, in main
tasks = partitioner(cfg)
^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/partitioners/base.py", line 83, in call
tasks = self.partition(**model_and_dataset_args,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/partitioners/num_worker.py", line 72, in partition
dataset_size = self.get_size(dataset)
^^^^^^^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/partitioners/num_worker.py", line 145, in get_size
dataset = build_dataset_from_cfg(dataset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/utils/build.py", line 12, in build_dataset_from_cfg
return LOAD_DATASET.build(dataset_cfg)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/mmengine/registry/registry.py", line 570, in build
return self.build_func(cfg, *args, **kwargs, registry=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
obj = obj_cls(**args) # type: ignore
^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/datasets/base.py", line 19, in init
dataset = self.load(**kwargs)
^^^^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/datasets/chatml/chatml.py", line 67, in load
path = get_data_path(path, local_mode=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/utils/datasets.py", line 30, in get_data_path
return download_dataset(local_path, cache_dir)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/utils/datasets.py", line 90, in download_dataset
assert dataset_name, f'No valid url for {data_path}!\n' +
^^^^^^^^^^^^
AssertionError: No valid url for ./data/imo-bench-answerbench.jsonl!
Please make sure ./data/imo-bench-answerbench.jsonl is correct