GM segmentation on external dataset: `hc-lumbar-zurich`

[Nilser3]

Description

We currently have 2 models capable of segmenting the lumbar GM:

1. lumbar-T2star-GMseg : Dataset503 performed the best model with a 3D single class configuration. r20230601

• 3D single-class model
• Trained on lumbar-vanderbilt dataset
• Single contrast model: T2star

2. seg_gm_contrast_agnostic : release r20250204

• 2D single-class model
• Trained on multi centric datasets and multi contrasts: see Train model for GM segmentation #2 (comment)

Dataset to test

• hc-lumbar-zurich
• 28 subjects
• 429 2D images (GT present)
• GT: Binary GM manual segmentations

Preliminar results

It is observed that the lumbar-T2star-GMseg method has better performance when segmenting the lumbar GM than the seg_gm_contrast_agnostic method, however in both cases there are many DICES at 0, to be investigated further.

[Nilser3]

After discussion with Gergely, it was found convenient to explore the DICES for each slice.
The images have 20 axial slices (0 for inferior and 19 for superior).

• Note, the DICE is only calculated if the 2D axial GT is present.

Here is the comparison:

It can be observed that indeed, in the lower slices the DICE are zero, and it is in these slices where the lumbar-T2star-GMseg model is more robust, however along the central slices, both methods have good results, and surprisingly in the superior slices (slice 18 and 19) the seg_gm_contrast_agnostic method is more robust, possibly because this method is a 2D model that better generalizes the classical shape of the GM (a butterfly).

Here are the inferences of both methods for more details.

[Nilser3]

Some observations from Gergely:

• lumbar-T2star-GMseg outperforms seg_gm_contrast_agnostic at both the Lumbosacral Enlargement (LSE) and the Conus Medullaris (CM), which is what I expected.
• The Dice remains surprisingly stable around 0.85 when moving from the LSE to the CM. This is very good news to me! 😊
• I think the low Dices at the upper two slices are due to the low GM/WM contrast caused by imperfect excitation profile (3D sequence). This is not a problem at all – we typically exclude these slices from any analysis.

The question now is: Is the performance of lumbar-T2star-GMseg on the Zurich data comparable to its performance on the Vanderbilt test set? Looking at this page (https://github.com/ivadomed/lumbar-T2star-GMseg?tab=readme-ov-file), the performance on the Vanderbilt set seems to be somewhat better (Dice around 0.9). Besides the different imaging parameters, I think an even larger factor could be the different GM segmentation styles in the two ground truth sets. There is at least one voxel layer at the GM/WM boundary, which some raters consider to be part of GM, while others don't.

Thanks for sending the inferences! We will look at them to identify whether there is tendency for lumbar-T2star-GMseg to either over- or undersegment the Zurich data.

[Nilser3]

The question now is: Is the performance of lumbar-T2star-GMseg on the Zurich data comparable to its performance on the Vanderbilt test set? Looking at this page (https://github.com/ivadomed/lumbar-T2star-GMseg?tab=readme-ov-file), the performance on the Vanderbilt set seems to be somewhat better (Dice around 0.9).

Indeed, the performance in the Vanderbilt set is better because the model is monocentric, in addition to the same shape of manual segmentation of the GM in the training and test set, also because in the Diagramme 2 on README.md a Dice analysis of 3D volumes of the GM of the lumbar spine was made, so having more pixels to evaluate, the Dice increases.

• Here is a 3D Dice analysis for our experiment in hc-lumbar-zurich:

[Nilser3]

seg_gm_contrast_agnostic II : New version of contrast agnostic model

• Details: Train seg_gm_contrast_agnostic model v.2.0 #13
• Model 2D ,
• New datasets added to the training: ms-barcelona-psir (cervical images) , hc-lumbar-zurich (22 subjects, lumbar images), als-basel-ramira (there are T12 and L1 images on this dataset)

Test split (N=6 subjects)

• sub-32, sub-36, sub-37, sub-39, sub-42 and sub-43 from hc-lumbar-zurich

Preliminary results

Dice Score on 2D slices (N=93)	Dice Score on 3D volumes

Dice Score by 2D slices (N=93)

Even though there are few test subjects (only 6), the seg_gm_contrast_agnostic_II model performs better than the previous models in the 93 2D slices.

Here the inferences for more details.

[jcohenadad]

wow, impressive!