FileNotFoundError: No such file or directory: "checkpoints/convnext-xxl-clip.safetensors"

[huajinghua]

hello,i have a problem:

python simple_inference_demo.py --model-path /llms/models/slowfast-video-mllm-qwen2-7b-convnext-576-frame128-s2t4/ --conv-mode qwen_1_5 --video_path "assets/catinterrupt.mp4" --question "Please describe this video in detail." --max_frames 64

/llms/anaconda3/envs/slowfast_mllm/lib/python3.10/site-packages/transformers/utils/generic.py:482: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
_torch_pytree._register_pytree_node(
.....
....
Traceback (most recent call last):
File "/llms/git/Slow-Fast-Video-Multimodal-LLM/simple_inference_demo.py", line 84, in
eval_model(args)
File "/llms/git/Slow-Fast-Video-Multimodal-LLM/simple_inference_demo.py", line 30, in eval_model
tokenizer, model, image_processor, context_len = load_pretrained_model(model_path, None, model_name, use_flash_attn=True)
File "/llms/git/Slow-Fast-Video-Multimodal-LLM/llava/model/builder.py", line 51, in load_pretrained_model
model = LlavaQwenSlowFastForCausalLM.from_pretrained(model_path, low_cpu_mem_usage=True, **kwargs)
File "/llms/anaconda3/envs/slowfast_mllm/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3886, in from_pretrained
model = cls(config, *model_args, **model_kwargs)
File "/llms/git/Slow-Fast-Video-Multimodal-LLM/llava/model/language_model/llava_qwen2_slowfast.py", line 249, in init
self.model = LlavaQwenSlowFastModel(config)
File "/llms/git/Slow-Fast-Video-Multimodal-LLM/llava/model/language_model/llava_qwen2_slowfast.py", line 41, in init
super(LlavaQwenSlowFastModel, self).init(config)
File "/llms/git/Slow-Fast-Video-Multimodal-LLM/llava/model/llava_arch.py", line 37, in init
self.vision_tower = build_vision_tower(config, delay_load=True)
File "/llms/git/Slow-Fast-Video-Multimodal-LLM/llava/model/multimodal_encoder/builder.py", line 27, in build_vision_tower
return ConvNextVisionTower(convnext_vision_tower, convnext_args)
File "/llms/git/Slow-Fast-Video-Multimodal-LLM/llava/model/multimodal_encoder/convnext_encoder.py", line 44, in init
self.load_model()
File "/llms/git/Slow-Fast-Video-Multimodal-LLM/llava/model/multimodal_encoder/convnext_encoder.py", line 53, in load_model
encoder_ckpt = safetensors.torch.load_file("checkpoints/convnext-xxl-clip.safetensors", device="cpu")
File "/llms/anaconda3/envs/slowfast_mllm/lib/python3.10/site-packages/safetensors/torch.py", line 311, in load_file
with safe_open(filename, framework="pt", device=device) as f:
FileNotFoundError: No such file or directory: "checkpoints/convnext-xxl-clip.safetensors"

where is the file："checkpoints/convnext-xxl-clip.safetensors“ ?

[flyinglynx]

Hi,
Thank you for your interest!

You can download the checkpoint from the following link

We’ve disabled the automatic checkpoint loading feature in the timm library because it isn't compatible with DeepSpeed ZeRO-3. Instead, we extracted the original checkpoint and converted it to the safetensors format for manual loading.