测试longbench报错，我该怎么在脚本中设置超过2048这个参数

[fxnie]

from mmengine.config import read_base
from opencompass.models import OpenAISDK

with read_base():
# from opencompass.configs.datasets.humaneval.humaneval_gen_8e312c import humaneval_datasets # noqa: F401, F403
# from opencompass.configs.datasets.ARC_c.ARC_c_gen import ARC_c_datasets # noqa: F401, F403
from opencompass.configs.datasets.longbench.longbench import longbench_datasets
# from opencompass.configs.datasets.leval.leval import leval_datasets

datasets = longbench_datasets

api_meta_template = dict(
round=[
dict(role='HUMAN', api_role='HUMAN'),
dict(role='BOT', api_role='BOT', generate=True),
],
reserved_roles=[dict(role='SYSTEM', api_role='SYSTEM')],
)

models = [
dict(
abbr='mamba',
type=OpenAISDK,
key='EMPTY', # API key
openai_api_base='http://0.0.0.0:6606/v1', # 服务地址
path='mamba', # 请求服务时的 model name
tokenizer_path='/workspace/mnt/cm-nfx/model/Falcon3-Mamba-7B-Instruct', # 请求服务时的 tokenizer name 或 path, 为None时使用默认tokenizer gpt-4
rpm_verbose=True, # 是否打印请求速率
meta_template=api_meta_template, # 服务请求模板
query_per_second=1, # 服务请求速率
max_out_len=1024, # 最大输出长度
max_seq_len=32768, # 最大输入长度
temperature=0.01, # 生成温度
batch_size=1, # 批处理大小
)
]

error:

/workspace/mnt/cm-nfx/opencompass/opencompass/init.py:19: UserWarning: Starting from v0.4.0, all AMOTIC configuration files currently located in ./configs/datasets, ./configs/models, and ./configs/summarizers will be migrated to the opencompass/configs/ package. Please update your configuration file paths accordingly.
_warn_about_config_migration()
2025-01-15 15:36:21.504541: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-01-15 15:36:21.651542: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
2025-01-15 15:36:21.651584: I tensorflow/compiler/xla/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2025-01-15 15:36:22.320673: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
2025-01-15 15:36:22.320755: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
2025-01-15 15:36:22.320764: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
01/15 15:36:24 - OpenCompass - INFO - Current exp folder: outputs/default/20250115_153624
01/15 15:36:24 - OpenCompass - WARNING - SlurmRunner is not used, so the partition argument is ignored.
01/15 15:36:24 - OpenCompass - INFO - Partitioned into 1 tasks.
01/15 15:36:27 - OpenCompass - INFO - Task [mamba/LongBench_2wikimqa,mamba/LongBench_hotpotqa,mamba/LongBench_musique,mamba/LongBench_multifieldqa_en,mamba/LongBench_multifieldqa_zh,mamba/LongBench_narrativeqa,mamba/LongBench_qasper,mamba/LongBench_triviaqa,mamba/LongBench_gov_report,mamba/LongBench_qmsum,mamba/LongBench_vcsum,mamba/LongBench_dureader,mamba/LongBench_lcc,mamba/LongBench_repobench-p,mamba/LongBench_passage_retrieval_en,mamba/LongBench_passage_retrieval_zh,mamba/LongBench_passage_count,mamba/LongBench_trec,mamba/LongBench_lsht,mamba/LongBench_multi_news,mamba/LongBench_samsum]
01/15 15:36:27 - OpenCompass - WARNING - Max Completion tokens for mamba is 16384
01/15 15:36:27 - OpenCompass - INFO - Try to load the data from /root/.cache/opencompass/./data/Longbench
01/15 15:36:28 - OpenCompass - INFO - Start inferencing [mamba/LongBench_2wikimqa]
01/15 15:36:28 - OpenCompass - WARNING - 'Could not automatically map /workspace/mnt/cm-nfx/model/Falcon3-Mamba-7B-Instruct to a tokeniser. Please use tiktoken.get_encoding to explicitly get the tokeniser you expect.', tiktoken encoding cannot load /workspace/mnt/cm-nfx/model/Falcon3-Mamba-7B-Instruct
01/15 15:36:28 - OpenCompass - INFO - Successfully load HF Tokenizer from /workspace/mnt/cm-nfx/model/Falcon3-Mamba-7B-Instruct
[2025-01-15 15:36:33,498] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting build dataloader
[2025-01-15 15:36:33,498] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process...
0%| | 0/200 [00:00<?, ?it/s01/15 15:36:33 - OpenCompass - INFO - Current RPM 1. | 0/1 [00:00<?, ?it/s]
01/15 15:36:33 - OpenCompass - ERROR - /workspace/mnt/cm-nfx/opencompass/opencompass/models/openai_api.py - _generate - 648 - error occurs at http://0.0.0.0:6606/v1
01/15 15:36:33 - OpenCompass - ERROR - /workspace/mnt/cm-nfx/opencompass/opencompass/models/openai_api.py - _generate - 650 - Error code: 400 - {'object': 'error', 'message': "This model's maximum context length is 2048 tokens. However, you requested 7428 tokens (7396 in the messages, 32 in the completion). Please reduce the length of the messages or completion.", 'type': 'BadRequestError', 'param': None, 'code': 400}
01/15 15:36:34 - OpenCompass - INFO - Current RPM 2.
01/15 15:36:34 - OpenCompass - ERROR - /workspace/mnt/cm-nfx/opencompass/opencompass/models/openai_api.py - _generate - 648 - error occurs at http://0.0.0.0:6606/v1
01/15 15:36:34 - OpenCompass - ERROR - /workspace/mnt/cm-nfx/opencompass/opencompass/models/openai_api.py - _generate - 650 - Error code: 400 - {'object': 'error', 'message': "This model's maximum context length is 2048 tokens. However, you requested 7428 tokens (7396 in the messages, 32 in the completion). Please reduce the length of the messages or completion.", 'type': 'BadRequestError', 'param': None, 'code': 400}
Inferencing: 0%| | 0/1 [00:01<?, ?it/s]
0%| | 0/200 [00:01<?, ?it/s]
Traceback (most recent call last):
File "/workspace/mnt/cm-nfx/opencompass/run.py", line 4, in
main()
File "/workspace/mnt/cm-nfx/opencompass/opencompass/cli/main.py", line 308, in main
runner(tasks)
File "/workspace/mnt/cm-nfx/opencompass/opencompass/runners/base.py", line 38, in call
status = self.launch(tasks)
File "/workspace/mnt/cm-nfx/opencompass/opencompass/runners/local.py", line 128, in launch
task.run(cur_model=getattr(self, 'cur_model',
File "/workspace/mnt/cm-nfx/opencompass/opencompass/tasks/openicl_infer.py", line 89, in run
self._inference()
File "/workspace/mnt/cm-nfx/opencompass/opencompass/tasks/openicl_infer.py", line 134, in _inference
inferencer.inference(retriever,
File "/workspace/mnt/cm-nfx/opencompass/opencompass/openicl/icl_inferencer/icl_gen_inferencer.py", line 153, in inference
results = self.model.generate_from_template(
File "/workspace/mnt/cm-nfx/opencompass/opencompass/models/base.py", line 201, in generate_from_template
return self.generate(inputs, max_out_len=max_out_len, **kwargs)
File "/workspace/mnt/cm-nfx/opencompass/opencompass/models/openai_api.py", line 176, in generate
results = list(
File "/opt/conda/envs/swift3/lib/python3.10/site-packages/tqdm/std.py", line 1181, in iter
for obj in iterable:
File "/opt/conda/envs/swift3/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
yield _result_or_cancel(fs.pop())
File "/opt/conda/envs/swift3/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
return fut.result(timeout)
File "/opt/conda/envs/swift3/lib/python3.10/concurrent/futures/_base.py", line 458, in result
return self.__get_result()
File "/opt/conda/envs/swift3/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/opt/conda/envs/swift3/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/workspace/mnt/cm-nfx/opencompass/opencompass/models/openai_api.py", line 655, in _generate
raise RuntimeError('Calling OpenAI API failed after retrying for '
RuntimeError: Calling OpenAI API failed after retrying for 2 times. Check the logs for details.

[fxnie]

将max_out_len设置为512也是同样的问题

[fxnie]

#1824