Atlas based segmentation with AMU7T template

[Nilser3]

Description

AMU7T is a 3D template created in 2 contrasts acquired at 7T:

• qT1-> from 26 subjects
• t2s -> from 72 subjects
  an atlas of 50 labels is associated (WM parcels from Levi et al., NeuroImage (2015)., and GM parcels derivated from Hausman, L. Atlases of the Spinal Cord and Brainstem and the Forebrain. (Thomas, 1962).
  AMU7T was created in the space of PAM50 (related in Integration of 7T template (T2s and qT1) + atlas  PAM50#14 ), and in a resolution of 0.17x0.17x0.5mm.

@jcohenadad this issue is to discuss how to automatically segment based on this new high resolution template/atlas through SCT:

here is a t1q 7T image that was automatically segmented using Sct registration (a) and a method with Optimized parameters (b).

From ISMRM2023

Sct registration (a)

# Register 7T MP2RAGE image  (t1q) to PAM50
sct_register_to_template -i t1q.nii.gz -s t1q_sc.nii.gz -l landmarks_t1q.nii.gz -c t2s -v 1 
# Register of  AMU7T_50_labels.nii.gz to to subject space
sct_apply_transfo  -i PAM50/AMU7T/atlas/AMU7T_50_labels.nii.gz -d t1q.nii.gz -w warp_template2anat.nii.gz -o tract_from_t1q_wm.nii.gz -x nn

Optimized parameters (b)

# Register 7T MP2RAGE image  (t1q) to PAM50
sct_register_to_template -i t1q.nii.gz -s t1q_sc.nii.gz -l landmarks_t1q.nii.gz -c t2s -v 1 

# Register of AMU26_qt1.nii.gz and AMU7T_wm.nii.gz to subject space using warps 
antsApplyTransforms -d 3 -i PAM50/AMU7T/template/AMU26_qt1.nii.gz -o qT1_AMU26_ants.nii.gz -t warp_template2anat.nii.gz  -r t1q.nii.gz
antsApplyTransforms -d 3 -i PAM50/AMU7T/template/AMU7T_wm.nii.gz -o wm_AMU26_ants.nii.gz -t warp_template2anat.nii.gz  -r t1q.nii.gz

# Adjustment of the registration in the subject space with a SyN step
antsRegistration --verbose 1 --dimensionality 3 --float 0 --collapse-output-transforms 1 --output [wm_t1q_s,wm_t1q_sWarped.nii.gz] --interpolation Linear --transform SyN[0.2,3,0] --metric CC[t1q.nii.gz,qT1_AMU26_ants.nii.gz,0.2,2] --metric CC[t1q_wm.nii.gz, wm_AMU26_ants.nii.gz,0.8,2] --convergence [100x75x50,1e-6,50] --shrink-factors 4x2x1 --smoothing-sigmas 2x1x0vox -g 1x1x0 -x [t1q_sc.nii.gz, t1q_sc.nii.gz]

# Register of  AMU7T_50_labels.nii.gz to to subject space using 2 warps 
antsApplyTransforms -d 3 -i PAM50/AMU7T/atlas/AMU7T_50_labels.nii.gz  -o tract_from_t1q_wm_ants.nii.gz   -t wm_t1q_s0Warp.nii.gz -t warp_template2anat.nii.gz -r t1q.nii.gz -n MultiLabel

[joshuacwnewton]

this issue is to discuss how to automatically segment based on this new high resolution template/atlas through SCT:

Given that the goal of this issue is to create a "pure SCT" way to do step (b), and that this will involve updating sct_warp_template, I think it might be best to transfer this issue to the SCT repo for organizational purposes.

[joshuacwnewton]

In our July 6th meeting, we asked "Should AMU7T be integrated into the PAM50 repo, or should it be kept separate?"

I don't think we quite came to a definitive answer during the meeting, so here are my thoughts:

---

Looking at the associated AMU7T PR (spinalcordtoolbox/PAM50#14), I notice that the AMU7T template has a similar structure to the PAM50 template as a whole:

• AMU7t:
  
• PAM50:
  

In other words, AMU7t is more like an alternate version of PAM50 (with alternate atlas/ and template/ folders), rather than a distinct subfolder within the PAM50 template (like histology/ was).

---

Given this, I am kind of leaning towards keeping AMU7t separate from PAM50:

• Users would "opt-in" by selecting the template using the -t (template) options of sct_register_to_template and sct_warp_template, which would allow us to re-use the existing template/atlas warping code.
• Pros: Much less complicated implementation (no need for new separate options, no need for multiple nested atlas or template folders, can reuse existing functionality [which is kind of the whole purpose of having a -t option!]).
• Cons: Users would have to run sct_warp_template twice: Once to warp the PAM50 template/atlas and once to warp the AMU7t template/atlas. Also, users have to "opt-in", rather than immediately getting the warped AMU7t via the default options, which might make the AMU7T template/atlas harder for users to discover.

[jcohenadad]

Excellent arguments @joshuacwnewton. Thank you for bringing that up. In light of these arguments, I agree that it makes more sense to have AMU7t as a separate template (despite it being within the same space as the PAM50 template).

[joshuacwnewton]

I agree that it makes more sense to have AMU7t as a separate template

Great! In that case, I would say the next steps are:

• Close the current PR on the PAM50 repo
• Create a new repo containing the current files for the AMU7T template
• Try out the sct_register_to_template and sct_warp_template commands with the -t option pointing at the AMU7T folder, to see if there are any issues.
• After we make sure that the -t option works well, then opening a PR in the SCT repo to add the AMU7T template to sct_download_data

As well, there is also the separate observation you noticed during our last SCT meeting:

• Improving the alignment for the "AMU26_qt1.nii.gz" version of the AMU7T template (the blue .GIF in the PAM50 PR)

cc: @Nilser3

[Nilser3]

Thanks for your comments @joshuacwnewton @jcohenadad

I closed the PR, and created a repo for template_AMU7T

I should point out that AMU7T is a template cropped in XY plane of 19.3 x 15.8 mm and along the Z axis from C1 to C7 approximately. Here a visualization with the levels of PAM50

Some Experiences of registration:

Here 7T T1q image + SC mask with which we are going to experiment ( r = 0.3x0.3x5.0):

Exp_1. Registration using PAM50:

sct_register_to_template -i image.nii.gz -l landmarks_pal1.nii.gz -c t1

Time: 98 sec

Exp_2. Registration using original AMU7T:

sct_register_to_template -i image.nii.gz -l landmarks_pal1.nii.gz -c t1 -t ../../../template_AMU7T/

Comment: Edge problems and loss of information at different levels.

Time: 37 sec

Exp_3. Registration using AMU7T with a zero padding on the XY axis (to get the same size as PAM50 from C1 to C7 )

sct_register_to_template -i image.nii.gz -l landmarks_pal1.nii.gz -c t1 -t ../../../template_AMU7T/

Comment: a partial loss of information in C8 (comparing with Exp_1)

Time: 306 sec

Exp_4. Registration using AMU7T with a zero padding in the XY and Z axis (C1 to T1):

sct_register_to_template -i image.nii.gz -l landmarks_pal1.nii.gz -c t1 -t ../../../template_AMU7T/

Comment: Results similar to exp_3

Time: 357 sec

Exp_5. Registration using AMU7T with a zero padding on the XYZ axis (C1 to T1) + using SC mask of PAM50 (at the same resolution and dimension):

sct_register_to_template -i image.nii.gz -l landmarks_pal1.nii.gz -c t1 -t ../../../template_AMU7T/

Comment: less deformation is observed in C1 and C8, with more resemblance to Exp_1

Time: 356 sec

Use of warps

a) sct_warp_template works for template_AMU7T/atlas/... labels

sct_warp_template -d image.nii.gz -t ../../../template_AMU7T/

Comment: approximate segmentation of the CSF contour.

b) sct_apply_transfo for atlas/AMU7T_50_labels.nii.gz

sct_apply_transfo -i ../../../template_AMU7T/atlas/AMU7T_50_labels.nii.gz -d image.nii.gz -w warp_template2anat.nii.gz -x nn

Comment: atlas-based segmentation with errors

The option -t is still not working well,
I am trying to use a sct_register_multimodal between anat2template from Exp_5 and /template_AMU7T/template/AMU26_qt1.nii.gz (using WM masks) to adjust the registration,

I have also seen that in the sct_register_to_template they do a resampling to 1mm isotropic, so I will also try to improve the registration in this way.

[jcohenadad]

We should redirect as many information/context regarding the AMU7T template (eg this very informative comment #4 (comment) which will be forgotten if not placed in the appropriate repos) as possible in the dedicated repos (https://github.com/spinalcordtoolbox/template_AMU7T) instead of here. @joshuacwnewton can you please help @Nilser3 do some cleaning / transfer of issues, etc. thanks!

[jcohenadad]

Exp_1. Registration using PAM50:

you should not be using -c t1 because this is a t2w like contrast

[jcohenadad]

you should never do straightening with 5mm thick images

[Nilser3]

Thank you @jcohenadad
following your suggestion, I worked on the registration of anisotropic images in the subject space:

Registration using AMU7T in subject space:

sct_register_to_template -i image.nii.gz -s image_wm.nii.gz -l landmarks.nii.gz -c t1 -v 1   -t ../../../template_AMU7T/ -s-template-id 4
sct_apply_transfo -i ../../../template_AMU7T/template/AMU7T_wm_qt1.nii.gz  -d image.nii.gz -w  warp_template2anat.nii.gz -o wm_amu7Ttoanat.nii.gz -x nn
sct_register_multimodal -i template2anat.nii.gz -d image.nii.gz -iseg wm_amu7Ttoanat.nii.gz  -dseg image_wm.nii.gz -o amu7Ttoanat_t1q.nii.gz -m sc_dil.nii.gz -param step=1,type=seg,algo=slicereg,metric=MeanSquares:step=2,type=im,algo=syn,metric=CC:step=3,type=seg,algo=syn,metric=CC,gradStep=0.2

Time = 788 sec.

Atlas AMU7T to subject space using 2 warps

sct_apply_transfo -i ../../../template_AMU7T/atlas/AMU7T_50_labels.nii.gz  -d image.nii.gz -w warp_template2anat.nii.gz warp_template2anat2image.nii.gz  -o atlasAMU7T_to_subject.nii.gz -x nn

Register AMU7T atlas using sct_warp_template (in 2 times)

sct_warp_template -d image.nii.gz -w warp_template2anat.nii.gz -t ../../../template_AMU7T/  -ofolder warp_template_1/
sct_warp_template -d image.nii.gz -w warp_template2anat2t1q_aniso_or_cor_den.nii.gz -t warp_template_1/label/ -ofolder warp_template_2

Visualization of external regions (CSF contour, Central-zone, Anterior-fissure and Septum)

In template_AMU7T/atlas/.. the labels are binary, however, after this registration, non-binary masks were obtained, possibly due to linear interpolations when applying the 2 warps.

Metric extraction using sct_extract_metric

sct_extract_metric -i image.nii.gz -f label/atlas/AMU7T_label_353_bin.nii.gz -method wa -o WA_353_GM_R_fasciculus_dorsolateralis.csv

Weighted average metric (WA ± SD) in the non-binary GM Right fasciculus-dorsolateralis mask per slice,in a qT1 image (peak in a MS lesion).

Note:

I worked on a second version of the AMU7T atlas and template (XY zero padding = 256x256) or (45x45mm) to preserve the spinal canal and vertebral bodies on the registered images.

[jcohenadad]

sct_register_to_template -i image.nii.gz -s image_wm.nii.gz -l landmarks.nii.gz -c t1 -v 1   -t ../../../template_AMU7T/ -s-template-id 4

To help the team review your code, I would add a comment that template-id 4 refers to the WM segmentation of the template:

template_AMU7T/template/info_label.txt

Line 8 in 8d68e44

4, white matter mask (probabilistic), AMU7T_wm.nii.gz

sct_apply_transfo -i ../../../template_AMU7T/template/AMU7T_wm_qt1.nii.gz  -d image.nii.gz -w  warp_template2anat.nii.gz -o wm_amu7Ttoanat.nii.gz -x nn

Why are you using '-x nn'? And for what purpose are you bringing AMU7T_wm_qt1.nii.gz into the subject's space? (Comments would be welcome)

sct_register_multimodal -i template2anat.nii.gz -d image.nii.gz -iseg wm_amu7Ttoanat.nii.gz  -dseg image_wm.nii.gz -o amu7Ttoanat_t1q.nii.gz -m sc_dil.nii.gz -param step=1,type=seg,algo=slicereg,metric=MeanSquares:step=2,type=im,algo=syn,metric=CC:step=3,type=seg,algo=syn,metric=CC,gradStep=0.2

Why not running these registration parameters while doing sct_register_to_template and specifying a -param flag in the first place?

Sub-sections "Atlas AMU7T to subject space using 2 warps" and "Register AMU7T atlas using sct_warp_template (in 2 times)": what are these? Are they different steps for doing the template to image registration, or are they different strategies that you are comparing? Please describe what is the purpose of each sub-section.

Section "Metric extraction using sct_extract_metric": very cool result!

[vcallot]

Thank you ++ @Nilser3 @jcohenadad and the team for all the efforts for integrating the AMU7T into the SCT!

A few comments and questions :

1) - the native image resolution you are working on is 0.30.34 mm3 (ie. anisotropic, with thickness of 4mm, not 5mm)(but that's no big deal ..)

2) -

following your suggestion, I worked on the registration of anisotropic images in the subject space:

I agree anisotropic images are not adequate for the straightening. In the ISMRM abstract, Arnaud indeed registered the template to the subject image. But then, the warping to the AMU7T template was not adequate (-> could you double check that everything is going well when you go back in the template space if you adopt this "template->subject" strategy?)

Also, how many landmarks do you use when running sct_register_to_template -l landmarks? On the native anisotropic T1q images, the vertebrae can not be seen (not as clearly as with other modalities). Consequently, 2 labels may not be sufficient/adequate.

3) - when you use the option "-c t1", registration is made from (or to) the AMU26_qt1.nii.gz, right?

4) -

I worked on a second version of the AMU7T atlas and template (XY zero padding = 256x256) or (45x45mm) to preserve the spinal canal and vertebral bodies on the registered images.

could you please provide more details? Did you start from scratch with a new crop for the native image? Or something else ?

5) -

Weighted average metric (WA ± SD) in the non-binary GM Right fasciculus-dorsolateralis mask per slice,in a qT1 image (peak in a MS lesion).

Thanks for this preliminary try!
When ready with the AMU7T template registration, we will have to investigate more carefully the robustness of the registration when a lesion (or compression, etc..) is present. In this try, the red ROI (supposedly GM fasciculus-dorsolateralis) is more likely a mix of GM and lateral CST. But for a 1st try, this is very cool. Thanks again!