NotImplementedError #635
Unanswered
WhiteAlbum2-Analyst
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
No matter whether I run command start with "ktransformers" or "python -m ktransformers.local_chat" ,I get a NotImplementedError.
I use official image approachingai/ktransformers:0.2.1
File "/workspace/ktransformers/ktransformers/models/modeling_llama.py", line 128, in __init__ inv_freq, self.attention_scaling = self.rope_init_fn( File "/opt/conda/lib/python3.10/site-packages/transformers/modeling_rope_utils.py", line 330, in _compute_llama3_parameters if wavelen < high_freq_wavelen: File "/opt/conda/lib/python3.10/site-packages/torch/utils/_device.py", line 78, in __torch_function__ return func(*args, **kwargs) NotImplementedError: aten::_local_scalar_dense: attempted to run this operator with Meta tensors, but there was no abstract impl or Meta kernel registered. You may have run into this message while using an operator with PT2 compilation APIs (torch.compile/torch.export); in order to use this operator with those APIs you'll need to add an abstract impl.Please see the following doc for next steps: https://docs.google.com/document/d/1_W62p8WJOQQUzPsJYa7s701JXt0qf2OfLub2sbkHOaU/edit
Any advice would be appreciated
Beta Was this translation helpful? Give feedback.
All reactions