Training the network gives high AUC but low ACC #14
Description
Thanks for open-sourcing this solution. I am using it as the backbone to evaluate a Federated Learning approach to medical imaging for my final year individual undergraduate project.
I trained the model (not using the pre-saved weights) on a 20% sample of the data, which follows the same distribution as the overall training set, and although the AUC scores are quite good, the ACC scores are quite low for some of the observations. I was wondering if you had some insight on what might be causing this. What Acc scores were you getting for your best model?
Atelectasis - AUCROC: 0.871, Acc: 0.375
Cardiomegaly - AUCROC: 0.834, Acc: 0.670
Consolidation - AUCROC: 0.908, Acc: 0.860
Edema - AUCROC: 0.888, Acc: 0.790
Pleural Effusion - AUCROC: 0.901, Acc: 0.335
Also, I am using the CheXpert downsampled dataset (11GB). The config file I am using is similar to the provided one except I have had to reduce the batch size to 8 due to GPU memory constraints.
Thanks!