Open
Description
Hi, I am getting this issue. I am running it on following system. I followed the instructions given in README.
Windows 11 Home
Intel Core i9
32GB RAM
( I tried with Anaconda and python3.11.9 and pip both failed )
pip show accelerate:
Name: accelerate
Version: 0.34.2
Pytorch GPU CUDA version:
Is CUDA supported by this system True
CUDA version: 12.1
ID of current CUDA device: 0
Name of current CUDA device: NVIDIA GeForce RTX 4090 Laptop GPU
for pip version
pip install torch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu121
for conda version
conda install pytorch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 pytorch-cuda=12.1 -c pytorch -c nvidia
any idea how to solve this issue:
Some parameters are on the meta device because they were offloaded to the cpu.
C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\models\clip\modeling_clip.py:480: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:455.)
attn_output = torch.nn.functional.scaled_dot_product_attention(
Traceback (most recent call last):
File "C:\Users\user\Desktop\Automatizacion_Township\EAGLE\predict_demo.py", line 66, in <module>
output_ids = model.generate(
^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\Desktop\Automatizacion_Township\EAGLE\eagle\model\language_model\eagle_llama.py", line 140, in generate
) = self.prepare_inputs_labels_for_multimodal(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\Desktop\Automatizacion_Township\EAGLE\eagle\model\eagle_arch.py", line 221, in prepare_inputs_labels_for_multimodal
image_features = self.encode_images(images)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\Desktop\Automatizacion_Township\EAGLE\eagle\model\eagle_arch.py", line 160, in encode_images
image_features = self.get_model().get_vision_tower()(images)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\accelerate\hooks.py", line 170, in new_forward
output = module._old_forward(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\Desktop\Automatizacion_Township\EAGLE\eagle\model\multimodal_encoder\multi_backbone_channel_concatenation_encoder.py", line 116, in forward
feature = vision_tower(resized_x)
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\accelerate\hooks.py", line 170, in new_forward
output = module._old_forward(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\Desktop\Automatizacion_Township\EAGLE\eagle\model\multimodal_encoder\convnext_encoder.py", line 99, in forward
image_features = self._forward_images(images)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\Desktop\Automatizacion_Township\EAGLE\eagle\model\multimodal_encoder\convnext_encoder.py", line 105, in _forward_images
image_forward_outs = self.forward_features(images.to(device=self.device, dtype=self.dtype))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\Desktop\Automatizacion_Township\EAGLE\eagle\model\multimodal_encoder\convnext_encoder.py", line 86, in forward_features
x = self.vision_tower.stem(x)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\accelerate\hooks.py", line 165, in new_forward
args, kwargs = module._hf_hook.pre_forward(module, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\accelerate\hooks.py", line 364, in pre_forward
return send_to_device(args, self.execution_device), send_to_device(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\accelerate\utils\operations.py", line 175, in send_to_device
return honor_type(
^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\accelerate\utils\operations.py", line 82, in honor_type
return type(obj)(generator)
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\accelerate\utils\operations.py", line 176, in <genexpr>
tensor, (send_to_device(t, device, non_blocking=non_blocking, skip_keys=skip_keys) for t in tensor)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\accelerate\utils\operations.py", line 156, in send_to_device
return tensor.to(device, non_blocking=non_blocking)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
NotImplementedError: Cannot copy out of meta tensor; no data!
Metadata
Metadata
Assignees
Labels
No labels