Migrate Cellpose Segmentation Workflows to Cellpose 4

[falconsmilie]

Summary

This PR migrates scPortrait’s Cellpose segmentation workflow support toward Cellpose 4 compatibility while adding a stronger test safety net around the existing workflow behaviour.

The main change is the introduction of a narrow Cellpose backend adapter that owns Cellpose-specific model loading and eval-call compatibility. The segmentation workflows continue to own image preparation, channel selection, post-processing, finalisation, filtering, mask matching, and saving.

This should make the Cellpose 4 migration easier to maintain without spreading Cellpose-version-specific logic across the workflow layer.

Why this change was needed

Cellpose 4 changed the supported model-loading API and channel/eval semantics compared with the previous Cellpose 3-era integration. The previous workflow code called Cellpose APIs directly, which made the dependency upgrade risky because model loading, eval parameters, workflow orchestration, and post-processing were tightly coupled.

This PR separates those responsibilities and adds tests around the behaviours most likely to regress during the migration.

Main changes

Test harness and workflow characterisation

• Added reusable lightweight Cellpose test helpers:
  • DummyFileHandler
  • FakeCellposeModel
  • workflow builder helpers
  • deterministic synthetic input image helpers
• Added smoke-style workflow tests for DAPI, cytosol, and cytosol-only Cellpose workflows.
• Added eval-call contract tests for current workflow behaviour.
• Expanded channel-selection tests across DAPI, cytosol, and cytosol-only workflows.
• Added post-processing/finalisation tests for:
  • edge-label removal
  • output stacking order
  • dtype conversion
  • size-filter orchestration
  • mask-matching orchestration
  • saved output structure

Model-loading and backend migration

• Added _cellpose_backend.py with:
  • CellposeBackend
  • CellposeModelSpec
  • CellposeEvalParameters
• Refactored Cellpose workflows to route model loading and eval calls through the backend adapter.
• Updated model loading to use Cellpose 4-compatible CellposeModel(pretrained_model=...) behaviour.
• Added deterministic tests for pretrained, custom, cache, and CP4-native model resolution paths.
• Normalised downloaded/custom model references with os.fspath(...) before passing them to Cellpose.
• Improved legacy model-cache fallback and error handling.

Validation and robustness fixes

• Fixed _load_model(...) validation so invalid config fails clearly when neither model nor model_path is configured.
• Fixed _check_input_image_dtype(...) so invalid image dtypes raise ValueError instead of silently constructing an error object.
• Fixed _check_seg_dtype(...) so it checks mask.dtype, emits a real warning, and converts to the configured segmentation dtype when needed.
• Hardened backend mask extraction so malformed Cellpose eval outputs fail early with clear errors.
• Fixed cytosol size-filter parameter wiring so cytosol filtering no longer accidentally reuses nucleus thresholds/confidence settings.

Cellpose 4 dependency and integration smoke tests

• Updated the Cellpose dependency to cellpose>=4.1,<5.
• Adjusted the NumPy requirement for Python 3.13 compatibility.
• Added opt-in pytest markers:
  • integration
  • cellpose
• Added maintainer-safe real Cellpose smoke/sanity paths for:
  • pretrained-name loading
  • custom-model-path loading
• Smoke tests are skipped by default and require explicit environment-variable opt-in.

Documentation

• Updated Cellpose segmentation documentation for Cellpose 4.
• Documented:
  • supported Cellpose version
  • model usage
  • model_path usage
  • parameter forwarding behaviour
  • channel behaviour for DAPI, cytosol, and cytosol-only workflows
  • expected differences from previous Cellpose behaviour

Additional compatibility fixes

• Added fallback label rendering for installed spatialdata-plot versions that do not support the method argument.
• Normalised combined Phenix XML paths to POSIX-style strings for improved cross-platform compatibility.

Testing

The branch adds or updates tests covering:

• Cellpose test helpers
• model-loading validation
• dtype validation
• backend model loading and eval forwarding
• backend malformed-output handling
• channel-selection policies
• eval-call contracts
• post-processing and finalisation
• deterministic model-cache/load behaviour
• optional real Cellpose integration smoke tests

Recommended test commands:

pytest tests/unit_tests/pipeline -q

pytest tests/unit_tests --disable-warnings

Focused Cellpose test run:

pytest tests/unit_tests/pipeline/test_cellpose_backend.py tests/unit_tests/pipeline/test_cellpose_model_loading.py tests/unit_tests/pipeline/test_cellpose_eval_contract.py tests/unit_tests/pipeline/test_cellpose_postprocessing.py tests/unit_tests/pipeline/test_segmentation_channel_selection.py -q

Optional real Cellpose smoke test:

SCPORTRAIT_RUN_CELLPOSE_INTEGRATION=1 pytest tests -m "cellpose or integration" -q

Notes for reviewers

The intended responsibility boundary is:

• backend owns Cellpose API details
• workflows own image preparation, channel policy, orchestration, post-processing, finalisation, and saving
• model cache code owns cache/download resolution
• tests should avoid real Cellpose downloads unless explicitly opted in

The real Cellpose smoke tests intentionally assert only structural output properties. They do not attempt to validate scientific segmentation quality or exact label values.

Areas to review carefully

Please pay special attention to:

• pretrained model-name resolution and cache fallback behaviour
• Cellpose 4 CellposeModel(pretrained_model=...) usage
• eval-kwargs filtering and forwarding
• channel mapping for DAPI/cytosol/cytosol-only workflows
• post-processing stack order and dtype conversion
• cytosol size-filter threshold wiring
• whether any Cellpose-version-specific logic has leaked outside the backend/cache boundary

Scope note

This PR includes a small amount of adjacent compatibility work for plotting and combined Phenix XML path handling.

[falconsmilie]

I'm running pre-commit locally, no idea why it fails on the server (and is being run under a unit test job?).

The pre-commit.ci action should work now (i ticked the box).

I figure the build will succeed now, if you can restart it.