PathoInsightMIL (elaborated upon HistoMIL originally by Dr. Shi Pan - Secrier Lab - UCL Genetics Institute)

PathoInsightMIL is a Python package for handling histopathology whole-slide images using multiple instance learning (MIL) techniques. You can create MIL datasets, train, cross-validate, evaluate MIL models, make MIL predictions on new slide images and perform interpretability analysis:

A sample of scripts submitted to the Sun Grid Engine scheduler executing each of the above steps in the pipeline using HistoMIL can be found at: https://github.com/awxlong/scripts_g0_arrest.

Implementation details:

HistoMIL is written in Pytorch Lightning, which provides the following benefits:

• mixed precision training
• gradient accumulation over patches
• model checkpointing for resuming crashed experiments
• logging of metrics with WandB.

We implement the following MIL algorithms:

1. TransMIL
2. TransMILRegression: TransMIL outputting regression scores instead of classification probabilities
3. TransMILMultimodal: TransMIL with multimodal fusion of clinical features
4. ABMIL
5. DSMIL
6. Transformer
7. TransformerRegression
8. TransformerMultimodal
9. AttentionMIL
10. CAMIL
11. DTFD_MIL
12. GraphTransformer
13. CLAM
14. A hybrid DTFD-MIL-TransMIL, where attention over pseudo-bags is replaced with TransMIL's Nystrom attention

There are additional jupyter notebooks in Notebooks/Data_Analysis/ concerning data analysis and plotting of results for cross-validation, testing with ensemblem, visualization of heatmaps and multimodal clinical feature importance. Notebooks are named based on the steps laid out in the above pipeline.

Installing HistoMIL

To use HistoMIL, you first need to create a conda environment with the required dependencies.

Create env with pre-defined file

You can do this by importing the env.yml file provided in this repository:

Linux user pre-requisites

1. Create conda env

conda create -n HistoMIL python=3.9

This will create a new environment named histomil, which you can activate with:

conda activate HistoMIL

Windows user pre-requisites

Windows (10+)

1. Download OpenSlide binaries from this page. Extract the folder and add bin and lib subdirectories to Windows system path. If you are using a conda environment you can also copy bin and lib subdirectories to [Anaconda Installation Path]/envs/YOUR ENV/Library/.

2. Install OpenJPEG. The easiest way is to install OpenJpeg is through conda using

conda create -n HistoMIL python=3.9

This will create a new environment named histomil, which you can activate with:

conda activate HistoMIL

C:\> conda install -c conda-forge openjpeg

macOS user pre-requisites

On macOS there are two popular package managers, homebrew and macports.

Homebrew

brew install openjpeg openslide

MacPorts

port install openjpeg openslide

Create env manually

Then install openslide and pytorch-gpu with following scripts.

conda install -c conda-forge openslide
conda install pytorch pytorch-cuda=11.7 -c pytorch -c nvidia

Next, install the required Python packages with pip:

pip install -r requirements.txt

This will install all the packages listed in requirements.txt, including HistoMIL itself.

Usage

All of the examples for using HistoMIL are included in the Notebooks/Tutorial/ directory. You can open and run these Jupyter notebooks to see how to use HistoMIL for different histopathology tasks.

Contributing

If you find a bug or want to suggest a new feature for HistoMIL, please open a GitHub issue in this repository. Pull requests are also welcome!

License

HistoMIL is released under the GNU-GPL License. See the LICENSE file for more information.