normed_output = model.generate(normed_seqs, max_new_tokens=prediction_length)

[Alex19970228]

debug:
File "/home/zhanghuanhome/miniconda3/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py", line 331, in _prepare_4d_causal_attention_mask
attention_mask = attn_mask_converter.to_4d(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zhanghuanhome/miniconda3/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py", line 139, in to_4d
expanded_attn_mask = causal_4d_mask.masked_fill(expanded_attn_mask.bool(), torch.finfo(dtype).min)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: The size of tensor a (2880) must match the size of tensor b (30) at non-singleton dimension 3

[Auditor1234]

debug: File "/home/zhanghuanhome/miniconda3/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py", line 331, in _prepare_4d_causal_attention_mask attention_mask = attn_mask_converter.to_4d( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhanghuanhome/miniconda3/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py", line 139, in to_4d expanded_attn_mask = causal_4d_mask.masked_fill(expanded_attn_mask.bool(), torch.finfo(dtype).min) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: The size of tensor a (2880) must match the size of tensor b (30) at non-singleton dimension 3

May be you downloaded the pre-trained model from hf-mirror.com. I encountered this problem yet and solved it by downloading model from the website huggingface.co

[DB1929]

I also downloaded the file from hf-mirror and solved the problem by upgrading package
torch==2.0.1
transformers==4.40.1