Refactor Prediction App to include extended annotation workflow

CLAUDE.md

@@ -0,0 +1,128 @@
+# CLAUDE.md - OMERO.biomero
+
+## Project structure
+
+OMERO.biomero is a Django web app plugin for OMERO.web. The frontend is a React/webpack SPA in `webapp/`, built assets go to `omero_biomero/static/`. The backend is Python/Django in `omero_biomero/`.
+
+### Repository layout
+
+- `omero_biomero/` — Django views, URLs, tests, scripts
+- `webapp/src/` — React frontend source
+  - `webapp/src/biomero/` — Analyzer (workflow execution) app
+  - `webapp/src/biomero/annotate/` — Annotate app (annotation + training)
+  - `webapp/src/importer/` — Importer app
+  - `webapp/src/shared/` — Shared components
+- `omero_biomero/static/omero_biomero/assets/` — Built webpack output (committed)
+
+### Related repos (sibling directories)
+
+| Repo | Path | Purpose |
+|------|------|---------|
+| `omero_annotate_ai` | `../omero_annotate_ai/` | Core annotation logic (manifest, GeoJSON, OMERO persistence) |
+| `biomero` | `../biomero/` | SlurmClient library for SLURM job management |
+| `biomero-scripts` | `../biomero-scripts/` | OMERO scripts that run on the server (training, inference) |
+| `W_Segmentation-Cellpose4` | `../W_Segmentation-Cellpose4/` | Cellpose training Singularity container |
+| `NL-BIOMERO` | `../NL-BIOMERO/` | Docker compose stack for the full BIOMERO deployment |
+
+### Git remotes
+
+- `origin` — user's fork (`maartenpaul/OMERO.biomero`)
+- `upstream` — main repo (`NL-BioImaging/OMERO.biomero`)
+
+## Development workflow
+
+After making changes, always rebuild and deploy:
+
+1. **Rebuild webapp assets** (if frontend changed): `cd webapp && yarn build`
+2. **Deploy to OMERO web container**: `bash omero-init.sh` (from project root)
+
+Both steps are required for frontend changes — yarn build compiles React/webpack assets into `omero_biomero/static/`, and `omero-init.sh` installs packages and restarts OMERO web in the Docker container (`nl-biomero-omeroweb-1`).
+
+For backend-only changes (Python files under `omero_biomero/`), only `bash omero-init.sh` is needed since packages are installed in editable mode.
+
+### Biomero (SlurmClient) package
+
+If the `biomero` library itself has changed (local repo at `../biomero/`), run:
+
+```bash
+bash ../NL-BIOMERO/biomeroworker/biomeroworker-init.sh
+cd ../NL-BIOMERO && docker compose -f docker-compose-dev.yml restart biomeroworker
+```
+
+The biomero package is volume-mounted into the biomeroworker container but needs `pip install -e` to be picked up. The omeroweb container does NOT mount biomero — it uses the pip-installed version from its Docker image.
+
+### omero_annotate_ai package
+
+Local repo at `../omero_annotate_ai/`. Installed in editable mode in the omeroweb container. After changes, run `bash omero-init.sh` to pick them up.
+
+Tests: `cd ../omero_annotate_ai && pixi run -e dev python -m pytest tests/ -p no:napari -v`
+
+### SLURM scripts (biomero-scripts)
+
+The `biomero-scripts` repo is volume-mounted at `../biomero-scripts/` -> `/opt/omero/server/OMERO.server/lib/scripts/biomero` on the omeroserver. Changes are picked up on next script load — no rebuild needed.
+
+### Training container (W_Segmentation-Cellpose4)
+
+To rebuild and deploy the training container:
+
+```bash
+cd ../W_Segmentation-Cellpose4
+docker build --no-cache -t cellpose4-train .
+docker save cellpose4-train -o ~/cellpose4-train.tar
+docker cp ~/cellpose4-train.tar slurmctld:/data/cellpose4-train.tar
+docker exec slurmctld singularity build --force /data/my-scratch/singularity_images/workflows/cellpose4/w_segmentation-cellpose4_latest.sif docker-archive:/data/cellpose4-train.tar
+docker exec slurmctld rm /data/cellpose4-train.tar
+rm ~/cellpose4-train.tar
+```
+
+Use `--no-cache` when `train.py` or `entrypoint.sh` changed — Docker caches COPY layers aggressively.
+
+**Important**: Singularity containers are read-only. All write paths in `train.py` must use the `--outfolder` (bound to `/data`) not `/tmp`. The `--writable-tmpfs` flag does NOT work on this SLURM cluster (fuse-overlayfs broken).
+
+## Architecture notes
+
+### Storage model (annotation consolidation)
+
+Two storage artifacts per annotation set:
+
+1. **JSON Manifest** — `AnnotationConfig` from `omero_annotate_ai` serialized to JSON. Stored as a `FileAnnotation` on the dataset/plate with namespace `omero.biomero.manifest.{set_id}`. Contains workflow config, unit list with progress, feature types, and cached channel presentation per image.
+
+2. **GeoJSON per image** — One `FileAnnotation` per image per annotation set with namespace `omero.biomero.annotations.{set_id}`. Contains all polygon annotations (across patches) with channel presentation at the top level. Patches are identified by `patch` property on each feature.
+
+**set_id** is a timestamp + random suffix (e.g. `20260401_143022_483_a7f2`), stable across updates.
+
+**What's NOT stored on annotation save**: ROI objects, label TIFFs. Label masks are generated at training time from GeoJSON.
+
+### Data flow
+
+- **ConfigureTab** creates the manifest via `saveManifest` API
+- **AnnotateTab** reads units from the manifest, saves GeoJSON per image via `saveAnnotateAnnotation`, updates manifest to mark units processed
+- **PreviewViewer** loads annotation overlays from GeoJSON by set_id namespace
+- **AnnotateApp** manages `setId` and `manifest` state, computes progress from `manifest.annotations`
+
+### Key backend endpoints (annotate)
+
+| Endpoint | Method | Purpose |
+|----------|--------|---------|
+| `save_manifest` | POST | Create/update JSON manifest |
+| `load_manifest` | GET | Load manifest by set_id |
+| `list_manifests` | GET | List annotation sets for a container |
+| `delete_manifest` | POST | Delete manifest + GeoJSON files |
+| `save_annotation` | POST | Save GeoJSON + update manifest progress |
+| `fetch_annotation` | GET | Load GeoJSON by set_id namespace |
+| `add_patch` | POST | Add random patch unit to manifest |
+
+All manifest/GeoJSON persistence logic lives in `omero_annotate_ai.omero.omero_functions` — the Django views are thin wrappers.
+
+### Channel normalization
+
+Image-level normalization (not per-patch) to avoid contrast issues in dark patches. Stored as `channel_presentation` on `ImageAnnotation` in the manifest (cached) and in the GeoJSON file (source of truth). The `ImageChannelControls` component uses 0-100% scales that are converted to absolute pixel values for the OMERO render URL.
+
+### Training pipeline
+
+Training flow: UI (TrainingBiomeroTab) -> Django (training_views.py) -> OMERO Script Service -> SLURM_Run_Training.py (biomero-scripts) -> sbatch (custom inline script) -> Singularity container (W_Segmentation-Cellpose4)
+
+- `SLURM_Run_Training.py` builds its own sbatch script (not the inference job script) with proper `/data` and models binds
+- After training completes, the script persists the model to the SLURM models dir via SSH and uploads results to OMERO
+- SLURM dev cluster nodes have 5GB RAM limit — don't request more in sbatch
+- Model files on SLURM need `chmod 644` to be readable inside Singularity containers

omero-init.sh

@@ -6,17 +6,19 @@ CONTAINER_NAME="nl-biomero-omeroweb-1"
 
 # Command to execute inside the container
 # COMMAND1="/usr/local/bin/entrypoint.sh"
-COMMAND0="chmod a+w /opt/omero/web/OMERO.web/var/static"
-COMMAND0B="git config --global --add safe.directory /opt/omero/web/OMERO.biomero"
+COMMAND0="chmod a+w /opt/omero/web/OMERO.web/var/static && chmod a+rw /opt/omero/web/OMERO.web/var/slurm-config.ini && touch /opt/omero/web/OMERO.web/var/importer-config.json && chmod a+rw /opt/omero/web/OMERO.web/var/importer-config.json"
 
-COMMAND1="/opt/omero/web/venv3/bin/python -m pip install -e /opt/omero/web/OMERO.biomero"
+# Mark volume-mounted repos as safe for git (needed by setuptools_scm)
+COMMAND_GIT="git config --global --add safe.directory /opt/omero/web/OMERO.biomero && git config --global --add safe.directory /opt/omero/web/omero_annotate_ai && git config --global --add safe.directory /opt/omero/web/OMERO.forms"
+
+COMMAND1="/opt/omero/web/venv3/bin/python -m pip install -e /opt/omero/web/omero_annotate_ai[all] -e /opt/omero/web/OMERO.biomero"
 COMMAND2="/opt/omero/web/venv3/bin/omero-biomero-setup"
 
 COMMAND3="/opt/omero/web/venv3/bin/omero web stop || true; rm -f /opt/omero/web/OMERO.web/var/django.pid"
 COMMAND4="/opt/omero/web/OMERO.biomero/startup.sh"
 
 docker exec --user root "$CONTAINER_NAME" sh -c "$COMMAND0"
-docker exec --user root "$CONTAINER_NAME" sh -c "$COMMAND0B"
+docker exec --user root "$CONTAINER_NAME" sh -c "$COMMAND_GIT"
 docker exec --user root "$CONTAINER_NAME" sh -c "$COMMAND1"
 docker exec --user root "$CONTAINER_NAME" sh -c "$COMMAND2"
 docker exec --user omero-web "$CONTAINER_NAME" sh -c "$COMMAND3"

omero-update-assets.sh

@@ -25,7 +25,7 @@ if [ $? -eq 0 ]; then
   # require an OMERO.web restart after in-place updates.
   echo "Restarting OMERO.web to refresh cached static metadata..."
   docker exec "$CONTAINER_NAME" bash -lc "/opt/omero/web/venv3/bin/omero web restart"
-  
+
   # Ensure readable permissions
   docker exec "$CONTAINER_NAME" bash -c "chmod -R a+rX ${DEST_DIR}"