A reproducibility crisis has been reported across many research fields, including neuroimaging, reaching up to 70% of studies. Neuroimaging data, such as MRI, requires preprocessing to allow for inter-subject comparison, increase signal contrast and noise reduction. As manual MRI preprocessing is time consuming and requires expertise, multiple automatic preprocessing frameworks have been proposed. However, neuroimaging studies often report divergent results, even for similar populations, thus it is important to determine whether this occurs as a result of different processing tools. Two of the most used tools are FreeSurfer and the CAT12. In this study, we assessed the reproducibility between these two automatic preprocessing frameworks for structural MRI and test-retest reliability within the framework on estimating cortical thickness. Our results show that the reproducibility between the frameworks is lower at the ROI level than at the individual level. Furthermore, we found that the reproducibility was lower in paediatric samples than in adults. Finally, an acquisition site effect was also identified. Given the widespread use of these frameworks in basic and clinical neuroscience, the results of multicentric cross-sectional studies must be interpreted with caution, particularly with paediatric samples. The observed reproducibility issue might be one of the sources of discrepancies reported in neuroimaging studies. On a positive note, framework test-retest reliability within subject is high, suggesting that inconsistency of results may be less concerning in longitudinal studies.
https://www.sciencedirect.com/science/article/pii/S2666956022000617
https://cibit-uc.github.io/fs-cat12-cortical-thickness-reproducibility