[Custom data] validation loss is less than train loss, and opposite for the accuracy...

[mewcat2011]

Did I do something wrong if val loss is less than train loss for several models???
I have own dataset, and have tried transfer learning for the same data with 'Beit, Davit, Swin, EfficientViT' one by one.
But All the results show the val loss < train loss, and val accuracy > train loss.

[leondgarse]

Ya, it's common depending on the loss function and date augmentation. As you can see some results in imagenet#training using default BinaryCrossEntropyTimm loss, all sharing val loss < train loss, val accuracy > train accuracy. You may also try models like EfficientNetV1B0 as a baseline, this should be more stable.

[mewcat2011]

Thank you for your quick answer.
Do you think this kind of phenomenon could be normal for some cases like mine?

I am still confused because when I search this subject,
there are some articles which say this kind of phenomenon is the symtom of wrong training.
But couldn't resolve the problem with their solutions.

Now I see same cases in your link, and wonder why is this happened in some cases?

[leondgarse]

As long as the final val_acc could meet expect, I think this phenomenon is not an issue. For those basic imagenet training curves, the main reason is the heavy regularization and data augmentation. Like if setting --mixup_alpha 0 --cutmix_alpha 0 turning off mix augmentaion, or further --random_crop_min 1 turning random crop off, train accuracy should be higher. Previously I've tested those, but the data is lost...