Discrepancy in AUC Scores Using Provided Pre-trained Model #49
Description
I am currently working with the pre-trained model located at "config/pre_train.pth" as indicated in the provided data. My objective is to evaluate the AUC scores for the five given findings using the dev.csv dataset. However, the AUC scores that I have obtained appear to be significantly lower than the values mentioned in the readme.md file.
According to the readme.md file, the AUC scores obtained from testing the pre-trained model on a dataset of 200 patients are higher. However, after using the provided pre-trained model and evaluating it on the same dataset, I obtained the following AUC scores:
Finding Cardiomegaly Edema Consolidation Atelectasis Pleural_Effusion
Mentioned 0.8703 0.9436 0.9334 0.9029 0.9166
Obtained 0.7727 0.8711 0.8582 0.8434 0.9113
I am seeking clarification on this discrepancy. Could you please help me understand why there is a difference between the AUC scores I have obtained and the values mentioned in the readme.md file? Is there something specific I need to consider or any steps that I might be missing? Your guidance would be greatly appreciated in resolving this issue.