[BUG]build_model loads wrong state_dict

[Dango233]

Short in short, in Windows, when initiating model using kelip.build_model('ViT-B/32'), sometimes it loads a wrong state_dict from the huggingface cache folder. Maybe I've downloaded too much stuff from Huggingface, especially many different clips?

Now I have to add a line to load the right state_dict: model.load_state_dict(torch.load(r"C:\Users\*****\.cache\huggingface\transformers\8f525db77e15271b26656a43682a71b06f93efef9b38f2e880117210d3f3be74.f30d465dc319e53701f2bea7f3e60214c3846cd9cc0b6b27445f121d496322f6"))

Maybe rename the model will help?

[geonm]

No. It won't solve the problem you reported.

'ViT-B/32' is used to indicate the name of revision, not the whole model name.

Please check this line: https://github.com/navervision/KELIP/blob/master/kelip/model_loader.py#L108-L111

We didn't try to test KELIP with any edition of Windows.

[kobiso]

@Dango233
I had the similar problem in Ubuntu, and I fixed it by removing the huggingface cache folder and re-downloading the model checkpoints.
Can you try to remove the cache folder?
I haven't tried in Windows, tho.