Pre-trained TBX11K classifier — architecture, labels, and integration

[danielchang-0]

Following up on the TBX11K dataset (merged in #180), I'd like to train a classifier on it to include as a pre-trained model. After looking through models.py, I have a few questions so it fits the existing conventions:

1. Architecture / naming — The core models are all DenseNet121 at 224×224 (densenet121-res224-*). Should I match that and name this something like densenet121-res224-tbx11k?

2. Labels — The TBX11K pathologies (ActiveTuberculosis, ObsoletePulmonaryTuberculosis, Tuberculosis) aren't part of the standard 18-pathology default_pathologies list that the existing models align to. How would you like to handle this? A few options I can see:

   • Add the TB labels to default_pathologies
   • Give this model its own separate label list (not aligned to the standard 18)
   • Train it to predict just the Tuberculosis superclass

3. op_threshs — Each model includes per-pathology operating-point thresholds. Should I compute these on the validation split after training, the same way the existing models did?

4. Weights hosting — The weights all live on the GitHub releases page. I assume I'd train the model and send you the .pt file to upload to a release — is that right, or is there a preferred way to hand it off?

5. Evaluation — Are there metrics you'd want reported (e.g. AUC on the test split) before including it?

I'll be training on Kaggle (free GPU). Happy to follow whatever approach fits best.

[ieee8023]

Architecture / naming

I would add it as a new baseline model. Something like:

model = xrv.baseline_models.danielchang.TBClassifier() 

Labels

This doesn't need to be a subset of those labels. This model can predict its own set of labels. The other baseline models have a targets field with the names of the classes predicted:

torchxrayvision/torchxrayvision/baseline_models/xinario/__init__.py

Line 37 in f4087a0

targets: List[str] = ['Frontal', 'Lateral']

You should pick a set of tagets of these that maximizes performance on the downsteam tasks people care about. In the research papers about this dataset what classes do they predict? I think predicting three outputs: "ActiveTuberculosis", "ObsoletePulmonaryTuberculosis", Tuberculosis" as individual tasks (not a softmax over all) would be useful.

op_threshs

I think it is good to have models calibrated using this method. The validation set is good for this.

Weights hosting

Ya, you can share with me somehow (google drive etc) and I will add it, you can do this before the PR is ready and we can also delete and update it after.

Evaluation

In the TBX11K paper I see they report F1 which is also fine. I like AUC, its easy to report both. It would be good to follow the evaluation in the original paper to compare numbers directly with that. I think having a writeup about this evaluation would be nice to have to accompany this model as an md file or a pdf that is posted to arxiv.

I'd also try to find even more data and validate against that. I think you can also grab some images that should be out of the training distribution here: https://radiopaedia.org/articles/tuberculosis and run a small scale evaluation. This will help build confidence in the model because sometimes the split for train/val/test from the same pile of images doesn't test the potential distribution shifts encountered when models are used for other tasks.

I'll be training on Kaggle (free GPU)

That should work. Also google colab is an option. If you have issues with memory or job runtime I can work with you to figure out a solution.