Skip to content

Commit 266dfb9

Browse files
authored
Update README.md
1 parent f429fcc commit 266dfb9

File tree

1 file changed

+37
-36
lines changed

1 file changed

+37
-36
lines changed

README.md

Lines changed: 37 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Evaluation Workflow for BraTS 2023+
22

3-
The repository contains the evaluation workflows for the [BraTS 2023 challenge and beyond],
3+
The repository contains the evaluation workflows for the [BraTS 2023 Challenge and beyond],
44
including:
55

66
* BraTS 2023
@@ -16,71 +16,72 @@ are located in the `shared` folder.
1616
Source code of the metrics computations mentioned in the README are available
1717
in the `evaluation` folder of this repo, organized into sub-folders by task.
1818

19-
[BraTS 2023 challenge and beyond]: https://www.synapse.org/brats
19+
[BraTS 2023 Challenge and beyond]: https://www.synapse.org/brats
2020

21-
## BraTS 2024
2221

23-
Branch: `main`
22+
## Metrics Overview
2423

25-
BraTS 2024 is an extension to [BraTS 2023](#brats-2023), and will also follow the two
26-
evaluation phases approach.
24+
<details><summary><strong>BraTS 2023</strong></summary>
25+
<br/>
2726

2827
Metrics returned and used for ranking will depend on the task:
2928

3029
**Task** | **Metrics** | **Ranking**
3130
--|--|--
3231
Segmentations | Lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity | Lesion-wise dice, lesion-wise HD95
33-
Inpainting | Structural similarity index measure (SSIM), peak-signal-to-noise-ratio (PSNR), mean-square-error (MSE) | All 3 metrics
34-
Augmentations | Full dice, full HD95, sensitivity, specificity | Dice mean, Dice GINI index, HD95 mean, HD95 GINI index
35-
Pathology | Matthews correlation coefficient (MCC), F1, sensitivity, specificity | All 4 metrics
32+
Inpainting | Structural similarity index measure (SSIM), peak-signal-to-noise-ratio (PSNR), mean-square-error (MSE) | SSIM, PSNR, MSE
33+
Augmentations | Full dice, full HD95, sensitivity, specificity | Dice mean, dice variance, HD95 mean, HD95 variance
3634

37-
## BraTS 2023
35+
---
3836

39-
Branch: `brats2023`
37+
</details>
4038

41-
BraTS 2023 is split into two evaluation phases:
4239

43-
* **Validation phase:** participants submit <u>predictions files</u> (segmentation masks, t1n inferences, etc.) to be evaluated using the validation dataset
40+
<details><summary><strong>BraTS-GoAT 2024</strong></summary>
41+
<br/>
4442

45-
* **Test phase:** participants submit <u>MLCube models</u> that will generate prediction files using the test dataset
43+
Metrics returned and used for ranking are:
4644

47-
Metrics returned and used for ranking will depend on the task:
45+
**Metrics** | **Ranking**
46+
--|--
47+
Lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity | Lesion-wise dice, lesion-wise HD95
4848

49-
**Task** | **Metrics** | **Ranking**
50-
--|--|--
51-
Segmentations | Lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity | Lesion-wise dice, lesion-wise HD95
52-
Inpainting | Structural similarity index measure (SSIM), peak-signal-to-noise-ratio (PSNR), mean-square-error (MSE) | SSIM, PSNR, MSE
53-
Augmentations | Full dice, full HD95, sensitivity, specificity | Dice mean, dice variance, HD95 mean, HD95 variance
49+
---
5450

55-
## BraTS-GoAT 2024
51+
</details>
5652

57-
Branch: `brats_goat2024`
5853

59-
Similar to BraTS 2023, BraTS-GoAT 2024 is split into two evaluation phases:
54+
<details><summary><strong>FeTS 2024</strong></summary>
55+
<br/>
6056

61-
* **Validation phase:** participants submit <u>segmentation predictions</u> to be evaluated using the validation dataset
6257

63-
* **Test phase:** participants submit <u>MLCube models</u> that will generate segmentation predictions using the test dataset
58+
Metrics returned are: lesion-wise dice, lesions-wise Hausdorff 95% distance
59+
(HD95), full dice, full HD95, sensitivity, specificity
6460

65-
Metrics returned and used for ranking are:
61+
**Note**: Code submission evaluations and ranking were handled by the
62+
[FeTS-AI Task 1 infrastructure](https://github.com/FeTS-AI/Challenge/tree/main/Task_1).
6663

67-
**Metrics** | **Ranking**
68-
--|--
69-
Lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity | Lesion-wise dice, lesion-wise HD95
64+
---
65+
66+
</details>
7067

71-
## FeTS 2024
7268

73-
Branch: `fets2024`
69+
<details><summary><strong>BraTS 2024</strong></summary>
70+
<br/>
7471

75-
FeTS 2024 has one evaluation phase facilitated by this repo:
72+
Metrics returned and used for ranking will depend on the task:
7673

77-
* **Validation phase:** participants submit <u>segmentation predictions</u> to be evaluated using the validation dataset
74+
**Task** | **Metrics** | **Ranking**
75+
--|--|--
76+
Segmentations | Lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity | Lesion-wise dice, lesion-wise HD95
77+
Inpainting | Structural similarity index measure (SSIM), peak-signal-to-noise-ratio (PSNR), mean-square-error (MSE) | All 3 metrics
78+
Augmentations | Full dice, full HD95, sensitivity, specificity | Dice mean, Dice GINI index, HD95 mean, HD95 GINI index
79+
Pathology | Matthews correlation coefficient (MCC), F1, sensitivity, specificity | All 4 metrics
7880

79-
Metrics returned are: lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity
81+
---
8082

81-
The **Code submission phase** is handled by the [FeTS-AI Task 1 infrastructure].
83+
</details>
8284

83-
[FeTS-AI Task 1 infrastructure]: https://github.com/FeTS-AI/Challenge/tree/main/Task_1
8485

8586
## Kudos 🍻
8687

0 commit comments

Comments
 (0)