AttributeError: 'WrappedPrefixCausalLM' object has no attribute 'generate'

[vengdeng]

Hi authors, very great paper. I try to run evaluation but found

AttributeError: 'WrappedPrefixCausalLM' object has no attribute 'generate'

It seems WrappedPrefixCausalLM don't have generate function, may I know how you resolve the issue?

Thank you very much.

[ChenMnZ]

@vengdeng What evaluation scripts are you used？

[QingshuiL]

@ChenMnZ I had the same problem when I ran eval.py.

Here's my code：

CUDA_VISIBLE_DEVICES=0,1,2,3 python eval.py \
--quant_model ./pre_quantized_models/QwQ-32B-Preview-w4a4kv4 \
--eval_tasks  gpqa \

The problem

AttributeError: 'WrappedPrefixCausalLM' object has no attribute 'generate'