I think we need to update the clip modeling from huggingface during with embedding where it can leads to dtype error with flash attention

Name		Name	Last commit message	Last commit date
Latest commit History 11 Commits
.ipynb_checkpoints		.ipynb_checkpoints
.vscode		.vscode
log		log
output		output
.gitignore		.gitignore
README.md		README.md
__init__.py		__init__.py
evaluate.py		evaluate.py
finetuning.py		finetuning.py
multi-gpu-run.sh		multi-gpu-run.sh
phi3v_dataset.py		phi3v_dataset.py
prepare_dataset.py		prepare_dataset.py
quilt_1m_data_prepare.py		quilt_1m_data_prepare.py
requirement.txt		requirement.txt
run_finetuning.sh		run_finetuning.sh
single-gpu-run.sh		single-gpu-run.sh
temp.csv		temp.csv
temp.py		temp.py
utils.py		utils.py

Provide feedback