Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
8963c29
initial commit
edyoshikun Jul 26, 2025
4e64bc0
bump dependencies for the setup
edyoshikun Jul 27, 2025
aee6ec1
Commit from GitHub Actions (Build Notebooks)
edyoshikun Jul 27, 2025
aa9a2c4
update cellpose-sam implementation
edyoshikun Jul 27, 2025
fe5381d
Commit from GitHub Actions (Build Notebooks)
edyoshikun Jul 27, 2025
a2511ac
incorporate ziwen's comments
edyoshikun Jul 28, 2025
7c80056
Commit from GitHub Actions (Build Notebooks)
edyoshikun Jul 28, 2025
0c674cb
swap the data download folder to their shared mount and bumping viscy…
edyoshikun Aug 21, 2025
6bc0fe3
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 21, 2025
c7c84bf
addressing Albert's comments on cellpose 4D->2D and adding extra hint…
edyoshikun Aug 21, 2025
5f3c684
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 21, 2025
1ddb435
adding TTAs bonus section
edyoshikun Aug 21, 2025
9cda37c
adding the fluor2phase model exercise 2.5
edyoshikun Aug 21, 2025
3f381e9
fix the setup_student.sh env name
edyoshikun Aug 21, 2025
764a563
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 21, 2025
460bba9
fix references and adding pretrained fluor2phase ckpt
edyoshikun Aug 21, 2025
4c10d9e
fix imports and ruff
edyoshikun Aug 21, 2025
8605da9
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 21, 2025
b186a4d
update exercise number from 6 to 4
edyoshikun Aug 21, 2025
c1f46b6
Merge branch 'fix_for_2025' of github.com:ai-mbl/image_translation in…
edyoshikun Aug 21, 2025
10c6613
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 21, 2025
2009607
replace min-max with rescale_intensity().
edyoshikun Aug 22, 2025
232a0e2
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 22, 2025
6c8a7cf
fix tta
edyoshikun Aug 22, 2025
8c90987
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 22, 2025
4898ec9
update to aws mountoint
edyoshikun Aug 22, 2025
7c0ed89
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 22, 2025
21d2f2f
fix typos
edyoshikun Aug 22, 2025
545ab5c
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 22, 2025
b68bf0f
fixing the exercise tasks to match the solution
edyoshikun Aug 22, 2025
daa74dd
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 22, 2025
03ddb3d
style: Minor reorganization of text and comments
adjavon Aug 22, 2025
fad1ab7
chore: Remove TODO comments where there isn't a change to make
adjavon Aug 22, 2025
b2de2eb
chore: Renumber task
adjavon Aug 22, 2025
92cdeff
Commit from GitHub Actions (Build Notebooks)
adjavon Aug 22, 2025
d2d5209
moving tensors to proper device and separating tasks for readability
edyoshikun Aug 22, 2025
7463a96
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 22, 2025
70d7d1b
adding a cropping parameter to compute the metrics
edyoshikun Aug 22, 2025
3f3dc21
removing the todo for plotting.
edyoshikun Aug 22, 2025
27bbcc2
chore: More re-numbering
adjavon Aug 22, 2025
402bb0b
chore: Update word order
adjavon Aug 22, 2025
69ddbe9
Commit from GitHub Actions (Build Notebooks)
adjavon Aug 22, 2025
d27c8a2
moving tensors to proper device and separating tasks for readability
edyoshikun Aug 22, 2025
190c225
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 22, 2025
143c5b9
swap to inference_mode(), fixing unecessary filepath glob, adding hin…
edyoshikun Aug 22, 2025
5e7e878
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 22, 2025
596f8d1
adding one more hint for the SSIM in the fluor2phase
edyoshikun Aug 22, 2025
f1488c0
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 22, 2025
4d13f01
Merge branch 'fix_for_2025' into minor_suggestions
edyoshikun Aug 22, 2025
8fa457a
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 22, 2025
a361cb5
Merge pull request #13 from ai-mbl/minor_suggestions
edyoshikun Aug 22, 2025
49cc4a4
extra space to render html code
edyoshikun Aug 22, 2025
c3439fd
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 22, 2025
a561aff
remove unecessary expanduser()
edyoshikun Aug 23, 2025
c3d1842
adding Diane and Albert
edyoshikun Aug 23, 2025
cbc98a7
Commit from GitHub Actions (Build Notebooks)
edyoshikun Aug 23, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
*lightning_logs*
.vscode
data/
21 changes: 12 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Exercise 6: Image translation - Part 1
# Exercise 4: Image translation

Written by Eduardo Hirata-Miyasaki, Ziwen Liu, and Shalin Mehta, CZ Biohub San Francisco with many inputs from Diane Adjavon for the DL@MBL2024 course
Written by Eduardo Hirata-Miyasaki, Ziwen Liu, and Shalin Mehta, CZ Biohub San Francisco with many inputs from Diane Adjavon and Albert Dominguez Mantes for the AI@MBL2025 course

## Overview

Expand All @@ -10,7 +10,7 @@ This strategy solves the problem as "multi-spectral imaging", but is more compat
Virtual staining is often a step towards multiple downstream analyses: segmentation, tracking, and cell state phenotyping.

In this exercise, you will:
- Train a model to predict the fluorescence images of nuclei and plasma membranes from QPI images
- Train a model to predict the fluorescence images of nuclei and plasma membranes from QPI images using CytoLand
- Make it robust to variations in imaging conditions using data augmentions
- Segment the cells
- Use regression and segmentation metrics to evalute the models
Expand All @@ -22,7 +22,7 @@ In this exercise, you will:

### Goals

#### Part 1: Train a virtual staining model
#### Part 1: Train a virtual staining model with Cytoland

- Explore OME-Zarr using [iohub](https://czbiohub-sf.github.io/iohub/main/index.html)
and the high-content-screen (HCS) format.
Expand All @@ -36,6 +36,9 @@ In this exercise, you will:
- Compare the performance of your trained model with the _VSCyto2D_ pre-trained model.
- Evaluate the model using pixel-level and instance-level metrics.

### Part 2.5: Evaluate the model to translate fluorescence to phase
- Compare the model trained from fluorescence to phase

#### Part 3: Visualize the image transforms learned by the model and explore the model's regime of validity
- Visualize the first 3 principal componets mapped to a color space in each encoder and decoder block.
- Explore the model's regime of validity by applying blurring and scaling transforms to the input phase image.
Expand All @@ -52,7 +55,7 @@ VisCy exploits recent advances in data and metadata formats

## Setup

Make sure that you are inside of the `image_translation` folder by using the `cd` command to change directories if needed.
Create an `mkdir image_translation` folder and `cd image_translation` command into it.

Run the setup script to create the environment for this exercise and download the dataset.
```bash
Expand All @@ -61,7 +64,7 @@ sh setup_student.sh
```
Activate your environment
```bash
conda activate 06_image_translation
conda activate 04_image_translation
```

## Use vscode
Expand All @@ -80,13 +83,13 @@ jupyter lab

...and continue with the instructions in the notebook.

If `06_image_translation` is not available as a kernel in jupyter, run:
If `04_image_translation` is not available as a kernel in jupyter, run:

```
python -m ipykernel install --user --name=06_image_translation
python -m ipykernel install --user --name=04_image_translation
```

### References

- [Liu, Z. and Hirata-Miyasaki, E. et al. (2024) Robust Virtual Staining of Cellular Landmarks](https://www.biorxiv.org/content/10.1101/2024.05.31.596901v2.full.pdf)
- [Liu, Z. and Hirata-Miyasaki, E. et al. (2025) Robust Virtual Staining of Cellular Landmarks](https://www.nature.com/articles/s42256-025-01046-2)
- [Guo et al. (2020) Revealing architectural order with quantitative label-free imaging and deep learning. eLife](https://elifesciences.org/articles/55502)
Loading