11# Evaluation Workflow for BraTS 2023+
22
3- The repository contains the evaluation workflows for the [ BraTS 2023 challenge and beyond] ,
3+ The repository contains the evaluation workflows for the [ BraTS 2023 Challenge and beyond] ,
44including:
55
66* BraTS 2023
@@ -16,71 +16,72 @@ are located in the `shared` folder.
1616Source code of the metrics computations mentioned in the README are available
1717in the ` evaluation ` folder of this repo, organized into sub-folders by task.
1818
19- [ BraTS 2023 challenge and beyond ] : https://www.synapse.org/brats
19+ [ BraTS 2023 Challenge and beyond ] : https://www.synapse.org/brats
2020
21- ## BraTS 2024
2221
23- Branch: ` main `
22+ ## Metrics Overview
2423
25- BraTS 2024 is an extension to [ BraTS 2023] ( #brats-2023 ) , and will also follow the two
26- evaluation phases approach.
24+ < details >< summary >< strong > BraTS 2023</ strong ></ summary >
25+ < br />
2726
2827Metrics returned and used for ranking will depend on the task:
2928
3029** Task** | ** Metrics** | ** Ranking**
3130--|--|--
3231Segmentations | Lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity | Lesion-wise dice, lesion-wise HD95
33- Inpainting | Structural similarity index measure (SSIM), peak-signal-to-noise-ratio (PSNR), mean-square-error (MSE) | All 3 metrics
34- Augmentations | Full dice, full HD95, sensitivity, specificity | Dice mean, Dice GINI index, HD95 mean, HD95 GINI index
35- Pathology | Matthews correlation coefficient (MCC), F1, sensitivity, specificity | All 4 metrics
32+ Inpainting | Structural similarity index measure (SSIM), peak-signal-to-noise-ratio (PSNR), mean-square-error (MSE) | SSIM, PSNR, MSE
33+ Augmentations | Full dice, full HD95, sensitivity, specificity | Dice mean, dice variance, HD95 mean, HD95 variance
3634
37- ## BraTS 2023
35+ ---
3836
39- Branch: ` brats2023 `
37+ </ details >
4038
41- BraTS 2023 is split into two evaluation phases:
4239
43- * ** Validation phase:** participants submit <u >predictions files</u > (segmentation masks, t1n inferences, etc.) to be evaluated using the validation dataset
40+ <details ><summary ><strong >BraTS-GoAT 2024</strong ></summary >
41+ <br />
4442
45- * ** Test phase: ** participants submit < u >MLCube models</ u > that will generate prediction files using the test dataset
43+ Metrics returned and used for ranking are:
4644
47- Metrics returned and used for ranking will depend on the task:
45+ ** Metrics** | ** Ranking**
46+ --|--
47+ Lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity | Lesion-wise dice, lesion-wise HD95
4848
49- ** Task** | ** Metrics** | ** Ranking**
50- --|--|--
51- Segmentations | Lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity | Lesion-wise dice, lesion-wise HD95
52- Inpainting | Structural similarity index measure (SSIM), peak-signal-to-noise-ratio (PSNR), mean-square-error (MSE) | SSIM, PSNR, MSE
53- Augmentations | Full dice, full HD95, sensitivity, specificity | Dice mean, dice variance, HD95 mean, HD95 variance
49+ ---
5450
55- ## BraTS-GoAT 2024
51+ </ details >
5652
57- Branch: ` brats_goat2024 `
5853
59- Similar to BraTS 2023, BraTS-GoAT 2024 is split into two evaluation phases:
54+ <details ><summary ><strong >FeTS 2024</strong ></summary >
55+ <br />
6056
61- * ** Validation phase:** participants submit <u >segmentation predictions</u > to be evaluated using the validation dataset
6257
63- * ** Test phase:** participants submit <u >MLCube models</u > that will generate segmentation predictions using the test dataset
58+ Metrics returned are: lesion-wise dice, lesions-wise Hausdorff 95% distance
59+ (HD95), full dice, full HD95, sensitivity, specificity
6460
65- Metrics returned and used for ranking are:
61+ ** Note** : Code submission evaluations and ranking were handled by the
62+ [ FeTS-AI Task 1 infrastructure] ( https://github.com/FeTS-AI/Challenge/tree/main/Task_1 ) .
6663
67- ** Metrics ** | ** Ranking **
68- --|--
69- Lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity | Lesion-wise dice, lesion-wise HD95
64+ ---
65+
66+ </ details >
7067
71- ## FeTS 2024
7268
73- Branch: ` fets2024 `
69+ <details ><summary ><strong >BraTS 2024</strong ></summary >
70+ <br />
7471
75- FeTS 2024 has one evaluation phase facilitated by this repo :
72+ Metrics returned and used for ranking will depend on the task :
7673
77- * ** Validation phase:** participants submit <u >segmentation predictions</u > to be evaluated using the validation dataset
74+ ** Task** | ** Metrics** | ** Ranking**
75+ --|--|--
76+ Segmentations | Lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity | Lesion-wise dice, lesion-wise HD95
77+ Inpainting | Structural similarity index measure (SSIM), peak-signal-to-noise-ratio (PSNR), mean-square-error (MSE) | All 3 metrics
78+ Augmentations | Full dice, full HD95, sensitivity, specificity | Dice mean, Dice GINI index, HD95 mean, HD95 GINI index
79+ Pathology | Matthews correlation coefficient (MCC), F1, sensitivity, specificity | All 4 metrics
7880
79- Metrics returned are: lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity
81+ ---
8082
81- The ** Code submission phase ** is handled by the [ FeTS-AI Task 1 infrastructure ] .
83+ </ details >
8284
83- [ FeTS-AI Task 1 infrastructure ] : https://github.com/FeTS-AI/Challenge/tree/main/Task_1
8485
8586## Kudos 🍻
8687
0 commit comments