Data are CenterCroped during preparation. However, if images are too small, the output cropped image is zero-padded. I have the intuition that when the model is exposed to data that include zeros, the model performance is reduced because these zeros represent a strong discontinuity:

I would like to try using mirroring instead of zero-padding and see how training goes.
Related to #1259 #1018 #1231