You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We detected that you are passing `past_key_values` as a tuple of tuples. This is deprecated and will be removed in v4.47. Please convert your cache or use an appropriate `Cache` class (https://huggingface.co/docs/transformers/kv_cache#legacy-cache-format)
Traceback (most recent call last):
File "<stdin>", line 1, in<module>
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/transformers/utils/fx.py", line 1503, in symbolic_trace
traced_graph = tracer.trace(model, concrete_args=concrete_args)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/transformers/utils/fx.py", line 1326, in trace
self.graph = super().trace(root, concrete_args=concrete_args)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 822, in trace
(self.create_arg(fn(*args)),),
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 1189, in forward
outputs = self.model(
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 800, in module_call_wrapper
return self.call_module(mod, forward, args, kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/transformers/utils/fx.py", line 1190, in call_module
returnsuper().call_module(m, forward, args, kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 518, in call_module
ret_val = forward(*args, **kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 793, in forward
return _orig_module_call(mod, *args, **kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 1000, in forward
layer_outputs = decoder_layer(
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 800, in module_call_wrapper
return self.call_module(mod, forward, args, kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/transformers/utils/fx.py", line 1190, in call_module
returnsuper().call_module(m, forward, args, kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 518, in call_module
ret_val = forward(*args, **kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 793, in forward
return _orig_module_call(mod, *args, **kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 729, in forward
hidden_states, self_attn_weights, present_key_value = self.self_attn(
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 800, in module_call_wrapper
return self.call_module(mod, forward, args, kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/transformers/utils/fx.py", line 1190, in call_module
returnsuper().call_module(m, forward, args, kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 518, in call_module
ret_val = forward(*args, **kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 793, in forward
return _orig_module_call(mod, *args, **kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 646, in forward
if query_states.device.type == "cuda" and causal_mask is not None:
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/transformers/utils/fx.py", line 669, in __bool__
returnsuper().__bool__()
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/fx/proxy.py", line 447, in __bool__
return self.tracer.to_bool(self)
File "~/miniconda3/envs/pytorch-dev/lib/python3.10/site-packages/torch/fx/proxy.py", line 307, in to_bool
raise TraceError('symbolically traced variables cannot be used as inputs to control flow')
torch.fx.proxy.TraceError: symbolically traced variables cannot be used as inputs to control flow
Expected behavior
it should run without error. symbolic_trace() with ["input_ids", "attention_mask"] runs fine. However, when ["input_ids", "attention_mask", "past_key_values"] is fed as input_names, the error occurs. If using past_key_values is incorrect, it should be warned and aborted before trying to trace the model.
While a fix on a related error (#29923) is included in the released version, it seems there is still some bug.
The text was updated successfully, but these errors were encountered:
Hey! Thanks for reporting! cc @michaelbenayoun I can reproduce, but no idea how to avoid this. It's also not new, so I think past key value path was not tested!
System Info
transformers
version: 4.45.1Who can help?
No response
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
Expected behavior
it should run without error.
symbolic_trace()
with["input_ids", "attention_mask"]
runs fine. However, when ["input_ids", "attention_mask", "past_key_values"] is fed as input_names, the error occurs. If using past_key_values is incorrect, it should be warned and aborted before trying to trace the model.While a fix on a related error (#29923) is included in the released version, it seems there is still some bug.
The text was updated successfully, but these errors were encountered: