Merge pull request #19 from clementpoiret/develop

v1.1.0 - Multispectrality and Better Models

Author: Clément POIRET

.github/workflows/python-app.yml

@@ -5,13 +5,12 @@ name: Python application
 
 on:
   push:
-    branches: [ master, develop ]
+    branches: [master, develop]
   pull_request:
-    branches: [ master, develop ]
+    branches: [master, develop]
 
 jobs:
   build:
-
     strategy:
       matrix:
         operating-system: ["ubuntu-latest"]
@@ -20,51 +19,51 @@ jobs:
     runs-on: ${{ matrix.operating-system }}
 
     steps:
-    - uses: actions/checkout@v2
+      - uses: actions/checkout@v2
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install Poetry
+        uses: snok/install-poetry@v1
+        with:
+          version: latest
+          virtualenvs-create: true
+          virtualenvs-in-project: false
+          virtualenvs-path: ~/.virtualenvs
+          installer-parallel: true
 
-    - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v2
-      with:
-        python-version: ${{ matrix.python-version }}
+      - name: Cache Poetry virtualenv
+        uses: actions/cache@v1
+        id: cache
+        with:
+          path: ~/.virtualenvs
+          key: poetry-${{ secrets.CACHE_VERSION }}-${{ matrix.python-version }}-${{ matrix.operating-system }}-${{ hashFiles('**/poetry.lock') }}
+          restore-keys: |
+            poetry-${{ secrets.CACHE_VERSION }}-${{ matrix.python-version }}-${{ matrix.operating-system }}-${{ hashFiles('**/poetry.lock') }}
 
-    - name: Install Poetry
-      uses: snok/install-poetry@v1
-      with:
-        version: latest
-        virtualenvs-create: true
-        virtualenvs-in-project: false
-        virtualenvs-path: ~/.virtualenvs
-        installer-parallel: true
+      - name: Install Dependencies
+        run: poetry install
+        if: steps.cache.outputs.cache-hit != 'true'
 
-    - name: Cache Poetry virtualenv
-      uses: actions/cache@v1
-      id: cache
-      with:
-        path: ~/.virtualenvs
-        key: poetry-${{ secrets.CACHE_VERSION }}-${{ matrix.python-version }}-${{ matrix.operating-system }}-${{ hashFiles('**/poetry.lock') }}
-        restore-keys: |
-          poetry-${{ secrets.CACHE_VERSION }}-${{ matrix.python-version }}-${{ matrix.operating-system }}-${{ hashFiles('**/poetry.lock') }}
+      - name: Test & Coverage
+        run: poetry run coverage run --source=hsf -m pytest
 
-    - name: Install Dependencies
-      run: poetry install
-      if: steps.cache.outputs.cache-hit != 'true'
- 
-    - name: Test & Coverage
-      run: poetry run coverage run --source=hsf -m pytest
-    
-    - name: Publish Coverage on CodeClimate
-      uses: paambaati/[email protected]
-      env:
-        CC_TEST_REPORTER_ID: ${{ secrets.CC_TEST_REPORTER_ID }}
-      with:
-        coverageCommand: poetry run coverage xml
-        coverageLocations: |
-          ${{ github.workspace }}/coverage.xml:coverage.py
-        debug: true
+      - name: Publish Coverage on CodeClimate
+        uses: paambaati/[email protected]
+        env:
+          CC_TEST_REPORTER_ID: ${{ secrets.CC_TEST_REPORTER_ID }}
+        with:
+          coverageCommand: poetry run coverage xml
+          coverageLocations: |
+            ${{ github.workspace }}/coverage.xml:coverage.py
+          debug: true
 
-    - name: Publish Coverage on Codacy
-      uses: codacy/[email protected]
-      with:
-        project-token: ${{ secrets.CODACY_PROJECT_TOKEN }}
-        api-token: ${{ secrets.CODACY_API_TOKEN }}
-        coverage-reports: ${{ github.workspace }}/coverage.xml 
+      - name: Publish Coverage on Codacy
+        uses: codacy/[email protected]
+        with:
+          project-token: ${{ secrets.CODACY_PROJECT_TOKEN }}
+          api-token: ${{ secrets.CODACY_API_TOKEN }}
+          coverage-reports: ${{ github.workspace }}/coverage.xml

LAST_CHANGELOG.md

@@ -2,5 +2,11 @@
 
 ## Software
 
-* Fixed batch_size not properly set,
-* Memory optimizations.
+* New multispectral mode to segment from both T1 and T2 images
+* Fixed ANTs overloading /tmp directory
+* Updated dependencies
+
+## Models
+
+* New single and bagging models with more data and less biases (Tails of T1w MRIs)
+* New sparse models with int8-quantized weights

README.rst

@@ -4,7 +4,7 @@ Hippocampal Segmentation Factory (HSF)
 
 Exhaustive documentation available at: `hsf.rtfd.io <https://hsf.rtfd.io/>`_
 
-**Current Models version:** 2.0.0
+**Current Models version:** 3.0.0
 
 .. list-table::
     :header-rows: 1
@@ -115,8 +115,10 @@ To date, we propose 4 different segmentation algorithms (from the fastest to the
 
 - ``single_fast``: a segmentation is performed on the whole volume by only one model,
 - ``single_accurate``: a single model segments the same volume that has been augmented 20 times through TTA,
+- ``single_sq``: like ``single_accurate``, but using int8-quantized sparse models for a fast and efficient inference,
 - ``bagging_fast``: a bagging ensemble of 5 models is used to segment the volume without TTA,
-- ``bagging_accurate``: a bagging ensemble of 5 models is used to segment the volume with TTA.
+- ``bagging_accurate``: a bagging ensemble of 5 models is used to segment the volume with TTA,
+- ``bagging_sq``: like ``bagging_accurate``, but using int8-quantized sparse models for a fast and efficient inference.
 
 Finally, ``segmentation.ca_mode`` is a parameter that allows to combine CA1, CA2 and CA3 subfields.
 It is particularly useful when you want to segment low-resolution images where it makes no sense to
@@ -132,8 +134,10 @@ Compose your configuration from those groups (group=option)
 
 * augmentation: default
 * files: default
-* roiloc: default_t2iso
-* segmentation: bagging_accurate, bagging_fast, single_accurate, single_fast
+* hardware: deepsparse, onnxruntime
+* multispectrality: default
+* roiloc: default_corot2, default_t2iso
+* segmentation: bagging_accurate, bagging_fast, bagging_sq, single_accurate, single_fast, single_sq
 
 Override anything in the config (e.g. hsf roiloc.margin=[16,2,16])
 
@@ -189,13 +193,34 @@ Fields set with ??? are mandatory.
          * max_displacement: 4
          * locked_borders: 0
 
+   multispectrality:
+   
+   * pattern: null
+   * same_space: true
+   * registration:
+     * type_of_transform: Affine
+
+   hardware:
+  
+   * engine: onnxruntime
+   * engine_settings:
+     * execution_providers:
+       - CUDAExecutionProvider
+       - CPUExecutionProvider
+     * batch_size: 1
+
 
 Changelogs
 ==========
 
 HSF
 ---
 
+**Version 1.1.0**
+
+* New optional multispectral mode de segment from both T1 AND T2 images
+* Bug fixes and optimizations
+
 **Version 1.0.1**
 
 * Fix batch size issue
@@ -226,6 +251,20 @@ HSF
 Models
 ------
 
+**Version 3.0.0**
+
+* More data (coming from the Human Connectome Project),
+* New sparse and int8-quantized models.
+
+**Version 2.1.1**
+
+* Fixed some tails in 3T CoroT2w images (MemoDev)
+
+**Version 2.1.0**
+
+* Corrected incorrect T1w labels used for training,
+* Trained on slightly more data (T1w @1.5T & 3T, T2w; Healthy, Epilepsy & Alzheimer)
+
 **Version 2.0.0**
 
 * Trained with more T1w and T2w MRIs,

docs/about/release-notes.md

@@ -16,6 +16,11 @@ Current maintainers:
 
 ## HSF
 
+### Version 1.1.0 (N/A)
+
+* New optional multispectral mode de segment from both T1 AND T2 images
+* Bug fixes and optimizations
+
 ### Version 1.0.1 (2021-12-07)
 
 * Fixed batch size issue
@@ -44,13 +49,26 @@ Current maintainers:
 
 ## Models
 
+### Version 3.0.0 (2022-04-24)
+
+* More data (coming from the Human Connectome Project),
+* New sparse and int8-quantized models.
+
+### Version 2.1.1 (2022-03-03)
+
+* Fixed some tails in 3T CoroT2w images (MemoDev)
+
+### Version 2.1.0 (N/A)
+
+* Corrected incorrect T1w labels used for training,
+* Trained on slightly more data (T1w @1.5T & 3T, T2w; Healthy, Epilepsy & Alzheimer)
+
 ### Version 2.0.0 (2021-11-12)
 
 * Trained with more T1w and T2w MRIs,
 * Trained on more hippocampal sclerosis and Alzheimer's disease cases,
 * Updated training pipeline (hyperparameter tuning),
-* `single` models are now independant from bags,
-* `bagging` have `sparse` and `sparseqat` versions for sparsification and Quantization Aware Training.
+* `single` models are now independant from bags.
 
 ### Version 1.0.0 (2021-09-24)
 

docs/index.md

@@ -5,8 +5,8 @@
     <br>
     <font size="+2"><b>Hippocampal</b> <i>Segmentation</i> Factory</font>
     <br>
-    <b>Current HSF version:</b> 1.0.1<br>
-    <b>Built-in Models version:</b> 2.0.0<br>
+    <b>Current HSF version:</b> 1.1.0<br>
+    <b>Built-in Models version:</b> 3.0.0<br>
     <b>Models in the Hub:</b> 4
 </p>
 

docs/model-hub.md

@@ -48,10 +48,10 @@ All the models are using the same `ARUnet` architecture, which will be detailed
     Test-time augmentation also allows the computation of an uncertainty map to analyze
     the quality of the resulting segmentation.
 
-<!-- ??? example "`bagging_sparse` and `bagging_sparseqat` models"
+??? example "`single_sq` and `bagging_sq` models"
 
-    Those are the two most advanced methods. Inference is done with 5 models trained
-    on different subsets of the dataset (random sampling with replacement).
+    Those are the two most advanced methods. Inference is done with 1 or 5 models
+    trained on different subsets of the dataset (random sampling with replacement).
 
     This allows to have models with different learned properties, offering a better
     segmentation, but a slower inference compared to the classic `single_*` models.
@@ -60,16 +60,16 @@ All the models are using the same `ARUnet` architecture, which will be detailed
     the computational cost of the inference while retaining an optimal sub-model
     following the lottery ticket hypothesis.
 
-    The `sparseqat` method also includes Quantization Aware Training to improve even
+    The `sq` method also includes Quantization Aware Training to improve even
     more the efficiency of the inference.
 
     Those methods are appropriate for recent hardware supporting efficient computations
     on sparse vectors. Int8 Quantization is better used on hardware supporting fast
     int8 matrix computations. For example, on a CPU supporting the AVX512-VNNI vector
-    instruction set, your best bet is to use the `bagging_sparseqat` segmentation method.
+    instruction set, your best bet is to use the `bagging_sq` segmentation method.
 
     Test-time augmentation also allows the computation of an uncertainty map to analyze
-    the quality of the resulting segmentation. -->
+    the quality of the resulting segmentation.
 
 ## Third-party Segmentation Models
 

docs/user-guide/configuration.md

@@ -22,15 +22,20 @@ conf
 │   │   onnxruntime.yaml
 │   │   deepsparse.yaml
 │
+└───multipectrality
+│   │   default.yaml
+│
 └───roiloc
 │   │   default_corot2.yaml
 │   │   default_t2iso.yaml
 │
 └───segmentation
     │   single_fast.yaml
     │   single_accurate.yaml
+    │   single_sq.yaml
     │   bagging_fast.yaml
     │   bagging_accurate.yaml
+    │   bagging_sq.yaml
 ```
 
 Groups can be selected with `group=option`. For example: `hsf segmentation=bagging_fast`
@@ -57,6 +62,26 @@ The following example will recursively search all `*T2w.nii.gz` files in the `~D
 hsf files.path="~/Datasets/MRI/" files.pattern="**/*T2w.nii.gz" files.mask_pattern="*T2w_bet_mask.nii.gz
 ```
 
+### Multispectral mode
+
+Since v1.1.0, HSF supports multispectral mode, where the segmentation is defined from a consensus between segmentations from both T1 and T2 images. Default parameters are defined in [`conf/multispectrality/default.yaml`](https://github.com/clementpoiret/HSF/blob/master/hsf/conf/multispectrality/default.yaml).
+
+- `pattern` defines how to find the alternative contrast of the subject.
+- `same_space` defines whether the alternative contrast is already in the same space as the main one. If not, a registration will be performed with the `registration.*` arguments.
+- `registration` are the parameters given to [`ants.registration`](https://antspy.readthedocs.io/en/latest/registration.html), such as `type_of_transform`.
+
+You can use the multispectral mode with the following example. For each T2w MRI, it will search a local T1w MRI in the same folder, then register the T1 to the T2 image using an affine registration (default behavior), using the meansquares metric.
+
+```sh
+hsf files.path="~/Datasets/MRI/" files.pattern="**/*T2w.nii.gz" multispectrality.pattern="T1w_hires.nii.gz" multispectrality.same_space=False +multispectrality.registration.aff_metric="meansquares"
+```
+
+!!! warning "Multispectral mode may not always be the best choice"
+    Because it comes from a consensus between T1 and T2 images, it is highly dependent on the quality of the registration.
+    If hippocampi do not overlap well, the consensus will be biased.
+
+    A good choice might be to manually register the images, perform a quality check, then use the multispectral mode while passing `same_space=True`.
+
 ### Preprocessing pipeline
 
 The preprocessing pipeline is kept as minimal as possible.
@@ -237,9 +262,9 @@ hsf hardware=deepsparse
     For example, models can be pruned (e.g. weights are removed to obtain an optimal sub-model),
     or Quantized (e.g. weights, biases and activations are quantized to 8-bit).
 
-<!-- Since HSF v1.0.0, we provide sparsified and quantized models. Therefore, to fully benefit from
+Since HSF v1.1.0, we provide sparsified and quantized models. Therefore, to fully benefit from
 DeepSparse, you can our sparsified bootstrapped models trained with Quantization Aware Training (QAT):
 
 ```sh
-hsf hardware=deepsparse segmentation=bagging_sparseqat
-``` -->
+hsf hardware=deepsparse segmentation=bagging_sq
+```

docs/user-guide/installation.md

@@ -42,10 +42,11 @@ You'll be able to to install HSF from PyPI by following the instructions:
     ONNXRuntime 1.8 requires at least CUDA 11.0.3, and cuDNN 8.0.4. For newer versions, please
     check the [ONNXRuntime documentation](https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html)
 
-    Then run the following commands:
+    Then run the following commands (please note that sometimes, you need to uninstall `onnxruntime` for the GPU to be detected correctly):
 
     ```shell
     pip install hsf
+    pip uninstall onnxruntime  # If needed
     pip install onnxruntime-gpu
     ```
 

hsf/__init__.py

@@ -1 +1 @@
-__version__ = '1.0.1'
+__version__ = '1.1.0'