Automated Segmentation of PSM and Somite Structures in 4D Light-Sheet Microscopy

EPFL CS-433 Machine Learning Project - Fall 2025

Developed in collaboration with the Oates Lab at EPFL.

---

Data Location (Oates Lab Jupyter Cluster)

All data and code are located on the RCP cluster at:

/Data/241211-Her1YFPxUtrCh+H2BCer-HIGHRES/20241211_181907_Experiment/Position 3_Settings 1/

Contents:

Folder	Description
DELIVERABLES/	This repository (notebooks, models, report)
mask_santi/	CH3 PSM masks generated by Method 1 (218 timepoints)
Ground_Truth_Masks/	Expert annotations for evaluation
t0001_Channel X.tif ...	Raw microscopy data (CH1, CH2, CH3)

Access: https://rcp-caas-prod.rcp.epfl.ch/course-cs-433-group04

---

Project Overview

This project implements automated segmentation pipelines for analyzing 4D light-sheet fluorescence microscopy (LSFM) images of zebrafish embryos. The main contributions are:

1. PSM Segmentation (CH3): Weakly supervised Random Forest achieving Dice 0.679 (180% improvement over baselines)
2. Marker-Free PSM Segmentation (CH1+CH2): Novel approach achieving Dice 0.696 without PSM-specific markers
3. Somite Segmentation: Documentation of classical method limitations without specific fluorescent markers

Workflow Overview

┌─────────────────────────────────────────────────────────────────────────┐
│                         PSM SEGMENTATION PIPELINE                        │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                          │
│  METHOD 1: With Channel 3 (PSM marker available)                        │
│  ─────────────────────────────────────────────────                      │
│  Raw CH3 → MAD Thresholding → Train RF → 3D Mask → Dice 0.679           │
│  Notebook: PSM_Segmentation.ipynb                                       │
│                                                                          │
│  METHOD 2: Without Channel 3 (marker-free)                              │
│  ──────────────────────────────────────────                             │
│  CH1+CH2 → 13 Features → Train RF (using CH3 masks as labels)           │
│         → Position + Size Filtering → 3D Mask → Dice 0.696             │
│  Notebook: PSM_From_CH1_CH2_Improved.ipynb                              │
│                                                                          │
└─────────────────────────────────────────────────────────────────────────┘

When to use each method:

• Method 1: You have Channel 3 (Her1-YFP) data → Use PSM_Segmentation.ipynb
• Method 2: You only have CH1+CH2 (no PSM marker) → Use PSM_From_CH1_CH2_Improved.ipynb

Important note about Method 2:

• If using our pre-trained model (models/rf_model_improved.joblib): You do NOT need CH3. Just run the notebook with CH1+CH2 data and it works directly.
• If training a new model from scratch: You need CH3 masks ONE TIME to train. After training, the model can be used forever with only CH1+CH2.

In other words: CH3 is only needed once to create training labels. Once the model is trained, it segments PSM using only CH1+CH2, which is the whole point of this approach.

---

Repository Structure

DELIVERABLES/
├── README.md                              # This file
├── requirements.txt                       # Python dependencies
├── ML_Proyect_2_Final_REPORT.pdf         # Final report (PDF)
│
├── MAIN NOTEBOOKS
│   ├── 01_data_exploration.ipynb          # Data quality assessment & EDA
│   ├── PSM_Segmentation.ipynb             # Main PSM pipeline (Channel 3)
│   ├── Compute_Real_Metrics.ipynb         # Quantitative evaluation vs expert GT
│   ├── Quick_Metrics.ipynb                # Fast baseline comparison
│   └── Somite_Watershed_Segmentation.ipynb # Somite segmentation (exploratory)
│
├── MARKER-FREE SEGMENTATION (CH1+CH2)
│   ├── PSM_From_Channel1.ipynb            # CH1-only approach
│   ├── PSM_From_Channel2.ipynb            # CH2-only approach
│   ├── PSM_From_CH1_CH2_Combined.ipynb    # Basic CH1+CH2 combination
│   └── PSM_From_CH1_CH2_Improved.ipynb    # Improved CH1+CH2 (best results)
│
├── models/                             # Pre-trained models
│   ├── rf_model_psm.joblib                # RF model for CH3-based segmentation
│   ├── rf_model_psm_from_ch1_ch2.joblib   # RF model for CH1+CH2 segmentation
│   ├── rf_model_improved.joblib           # Improved CH1+CH2 model
│   ├── feature_stats.joblib               # Feature statistics
│   └── psm_statistics.joblib              # PSM size/location statistics
│
├── Comparison/                         # Visual comparison images
│   ├── t0XXX_gt_image.png                 # Expert/supervisor sketches
│   └── viz_t0XXX_Channel 3.png            # Our predictions
│
└── Report Latex/                       # LaTeX source files
    ├── latex-template.tex                 # Main LaTeX source
    ├── literature.bib                     # References
    ├── figures/                           # Report figures
    └── ML_Proyect_2_Final_REPORT.pdf       # Compiled PDF

Quick Start

1. Install Dependencies

pip install -r requirements.txt

2. Data Requirements

The pipeline expects 4D LSFM data in the following format:

• Format: 16-bit OME-TIFF
• Naming: tXXXX_Channel Y.tif (e.g., t0001_Channel 3.tif)
• Dimensions: Z × Y × X (e.g., 200 × 2304 × 2304)

Channels:

• Channel 1: H2B-Cerulean (nuclei)
• Channel 2: Utrophin (membranes)
• Channel 3: Her1-YFP (PSM-specific marker)

3. Run Notebooks

For PSM segmentation with Channel 3:

1. 01_data_exploration.ipynb - Verify data quality
2. PSM_Segmentation.ipynb - Generate PSM masks
3. Compute_Real_Metrics.ipynb - Evaluate against ground truth

For marker-free segmentation (without Channel 3):

1. PSM_From_CH1_CH2_Improved.ipynb - Best marker-free approach

---

How to Reproduce / Retrain

Step-by-Step Reproduction

Option A: Use Pre-trained Models (Fast)

The models/ folder contains pre-trained Random Forest models. Just run the notebooks and they will automatically load these models.

Option B: Retrain from Scratch

1. Delete the model file you want to retrain:
   • For CH3 method: delete models/rf_model_psm.joblib
   • For CH1+CH2 method: delete psm_from_ch1_ch2_improved/rf_model_improved.joblib
2. Run the notebook - it will automatically retrain when the model file is missing

Configuration: What to Change in Notebooks

Each notebook has a CONFIGURATION cell at the top. Key parameters to modify:

Parameter	Where	Description
RAW_DIR	All notebooks	Path to your raw TIFF files
GT_MASK_DIR	CH1+CH2 notebooks	Path to CH3 masks (training labels)
TRAINING_FRAMES	Training notebooks	List of timepoints to train on
TEST_FRAMES	All notebooks	List of timepoints to evaluate
SCALE_FACTOR	All notebooks	Downsampling (0.25 = 4x smaller, faster)

Expected Directory Structure

your_data_folder/
├── t0001_Channel 1.tif
├── t0001_Channel 2.tif
├── t0001_Channel 3.tif
├── t0002_Channel 1.tif
├── ...
└── mask_santi/              # CH3 masks (for CH1+CH2 training)
    ├── mask_t0001_Channel 3.tif
    ├── mask_t0030_Channel 3.tif
    └── ...

Troubleshooting

Problem	Solution
FileNotFoundError	Check RAW_DIR path points to your data
ModuleNotFoundError	Run pip install -r requirements.txt
Out of memory	Reduce SCALE_FACTOR (e.g., 0.125 instead of 0.25)
Model not retraining	Delete the .joblib file and rerun
Poor results	Ensure training frames have good CH3 masks

Results Summary

PSM Segmentation with Channel 3

Metric	Our Method	Otsu	Hysteresis
Dice Score	0.679 ± 0.044	0.242 ± 0.043	0.160 ± 0.017
IoU	0.515 ± 0.038	0.138 ± 0.031	0.087 ± 0.012

• 180% improvement over best baseline
• Processed 218 timepoints in 7.8 hours
• Biologically consistent volume dynamics (95.4% reduction)

Marker-Free Segmentation (CH1+CH2)

Method	Mean Dice	Best Dice
CH2 only	0.258	0.400
CH1 only	0.424	0.625
CH1+CH2 basic	0.489	0.737
CH1+CH2 improved	0.696	0.827

Key innovations:

• Position features (29% importance)
• Nearby-tissue negative sampling
• Size/location filtering

Somite Segmentation (Exploratory)

Classical watershed methods proved insufficient:

• Detected regions varied from 5-10 per frame
• No temporal stability
• Requires specific fluorescent marker or supervised deep learning

Key Parameters

PSM Segmentation (CH3)

Parameter	Value
MAD multiplier (k)	6.0
Features	Raw, Gaussian (σ=1.5, 3.5, 8.0), Sobel
RF trees / depth	70 / 12
Downsampling	4× XY
Post-processing	Closing (r=8), hole fill, largest component

Marker-Free (CH1+CH2)

Parameter	Value
Features	13 (5 CH1 + 5 CH2 + ratio + position x,y)
RF trees / depth	150 / 18
Class weights	balanced
Negative sampling	Within 15px of PSM boundary

Notebooks Description

Main Pipeline

• PSM_Segmentation.ipynb: Main PSM segmentation using CH3 with weakly supervised Random Forest
• Compute_Real_Metrics.ipynb: Quantitative evaluation (Dice, IoU) against expert ground truth
• Somite_Watershed_Segmentation.ipynb: Exploratory somite segmentation with parameter analysis

Marker-Free Approach

• PSM_From_CH1_CH2_Improved.ipynb: Best marker-free method using position features and nearby-tissue sampling
• PSM_From_CH1_CH2_Combined.ipynb: Basic combination approach
• PSM_From_Channel1.ipynb / PSM_From_Channel2.ipynb: Single-channel baselines

Analysis

• 01_data_exploration.ipynb: Data quality assessment (Z-attenuation, artifacts, drift)
• Quick_Metrics.ipynb: Fast baseline comparison

Citation

If you use this code, please cite:

Rivadeneira S., Hidri Y., Maznichenko L. (2025). 
Automated Segmentation of Presomitic Mesoderm and Somite Structures 
in 4D Light-Sheet Microscopy of Zebrafish Embryos. 
EPFL CS-433 Machine Learning Project.

Acknowledgements

• Oates Lab at EPFL for microscopy data and expert annotations
• Project supervisor for creating ground truth masks for quantitative evaluation

License

This project was developed for educational purposes at EPFL.

---

Authors: Santiago Rivadeneira, Yasmine Hidri, Lev Maznichenko
Contact: {santiago.rivadeneiraquintero, yasmine.hidri, lev.maznichenko}@epfl.ch