Basic descriptive statistics of the dataset

[simonqueric]

As suggested by Maxime, I checked some basic statistics and irregularities of the dataset.
The corresponding notebook can be found here.

Here is a short summary :

• 1975 subjects
• Each subjects has multiple sequences.
• Sequence types are among the three : Sagittal T1, Sagittal T2/STIR, Axial T2
• Number of sequences per subject are between 2 and 6 (mainly 3 sequences)

Multiple NIfTI problem :

For Sagittal sequences type, there are at most 2 niftii volumes after conversion from dicom to nifti.
For Axial T2, there are often multiple nifti volumes.

[jcohenadad]

Excellent first-level analysis. Now it would be interesting to dig more. Eg:

• is there a rationale to acquire more axial scans in some patients?
• do all subjects have at least one sagittal?
• can you reliably identify the contrast from the DICOM data (T1, T2, STIR, etc.)
• what are the two NIfTI volumes from the sagittal sequence?

[simonqueric]

In the dataset, volume resolutions are very heterogeneous.
I plot below histogram of resolution for Sagittal T1w volumes :

[simonqueric]

Relevent information : the dataset is extremely unbalanced.

Here is a graph showing prevalence of each pathologies in the training dataset :
(source)

We need a strategy to avoid a model which would output only "Normal" for every patient.
Some ideas :

• Resampling minority classes Moderate and Severe
• Remove randomly some normal subjects from the training dataset
• In the loss, penalize the model when its prediction is wrong on minority classes

[simonqueric]

Correlation matrix between conditions

I computed correlation between conditions (with encoding Normal=0, Moderate=1, Severe=2).
Below, the correlation matrix :

Interistingly enough, spinal canal stenosis is correlated with subarticular stenosis.

[jcohenadad]

Very interesting indeed. I'm wondering if we should add some regularization in the loss to account for this correlation, given that if we train models independently this correlation won't be learned by the models (unless we add another branch to the architecture that does a joint optimization to account for the output of each model).