What if we only use distill loss?

[ThisisBillhe]

Hi,

Thanks for your great work! Regarding table 5 in the paper, I wonder what the results will be when only using L_distill?

Looking forward to your reply.

[FelixMessi]

Hi, thanks for your question.

When using only L_distill, we found it to be competitive on the CC3M dataset. The contrastive loss becomes more beneficial as the dataset scales up, providing additional performance improvements with larger amounts of data.