Bug while initializing the evaluation of Qwen2VL-7B

IndexError: index 0 is out of bounds for dimension 0 with size 0 for " if cache_position is None or (cache_position is not None and cache_position[0] == 0):

The bug is similar to QwenLM/Qwen3-VL#596. Is replacing the template_chat JSON file methodologically sound? Could it potentially compromise model performance?"

[kcz358]

Can you try pip install -U transformers? I think this issue is possibly related to transformers and not lmms-eval. I am using transformers==4.52.3 and seems I can not reproduce this error

Yes, thanks. This is due to the version of transformers, thanks.