Skip to content

[Bug] IMO-Bench-AnswerBench 评测无数据集下载链接 #2405

@zhouyujin

Description

@zhouyujin

Prerequisite

Type

I'm evaluating with the officially supported tasks/models/datasets.

IMO-Bench-AnswerBench 评测时,在 datasets_info.py 的 DATASETS_URL 字典中缺少 imo-bench-answerbench 的下载配置。

报错信息如下:

INFO - ./data/imo-bench-answerbench.jsonl does not exist!Start Download data automatically!If you have downloaded the data before,You can specific COMPASS_DATA_CACHE to avoid downloading~
Traceback (most recent call last):
File "/u01/miniforge3/envs/open-compass/bin/opencompass", line 6, in
sys.exit(main())
^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/cli/main.py", line 349, in main
tasks = partitioner(cfg)
^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/partitioners/base.py", line 83, in call
tasks = self.partition(**model_and_dataset_args,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/partitioners/num_worker.py", line 72, in partition
dataset_size = self.get_size(dataset)
^^^^^^^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/partitioners/num_worker.py", line 145, in get_size
dataset = build_dataset_from_cfg(dataset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/utils/build.py", line 12, in build_dataset_from_cfg
return LOAD_DATASET.build(dataset_cfg)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/mmengine/registry/registry.py", line 570, in build
return self.build_func(cfg, *args, **kwargs, registry=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
obj = obj_cls(**args) # type: ignore
^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/datasets/base.py", line 19, in init
dataset = self.load(**kwargs)
^^^^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/datasets/chatml/chatml.py", line 67, in load
path = get_data_path(path, local_mode=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/utils/datasets.py", line 30, in get_data_path
return download_dataset(local_path, cache_dir)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/u01/miniforge3/envs/open-compass/lib/python3.12/site-packages/opencompass/utils/datasets.py", line 90, in download_dataset
assert dataset_name, f'No valid url for {data_path}!\n' +
^^^^^^^^^^^^
AssertionError: No valid url for ./data/imo-bench-answerbench.jsonl!
Please make sure ./data/imo-bench-answerbench.jsonl is correct

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions