|
| 1 | +# CLAUDE.md |
| 2 | + |
| 3 | +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +shrimPy is a Python framework for high-throughput smart microscopy that synchronizes data collection using hardware triggering and performs intelligent acquisition tasks like autofocus and autoexposure. The framework is designed to support multiple microscope platforms (mantis, iSIM, Dragonfly) through a modular, extensible architecture built on pymmcore-plus. |
| 8 | + |
| 9 | +Current status: Alpha version, actively restructuring from mantis-only to multi-microscope support (branch: `215-restructure-repository-for-multi-microscope-support`). |
| 10 | + |
| 11 | +## Common Development Commands |
| 12 | + |
| 13 | +### Setup |
| 14 | +```bash |
| 15 | +# Install in development mode with dev dependencies |
| 16 | +pip install -e ".[dev]" |
| 17 | + |
| 18 | +# Or using make |
| 19 | +make setup-develop |
| 20 | + |
| 21 | +# Install pre-commit hooks (required for contributors) |
| 22 | +pre-commit install |
| 23 | +``` |
| 24 | + |
| 25 | +### Code Quality |
| 26 | +```bash |
| 27 | +# Format code (black + isort) |
| 28 | +make format |
| 29 | + |
| 30 | +# Check formatting without modifying files |
| 31 | +make check-format |
| 32 | + |
| 33 | +# Lint with flake8 |
| 34 | +make lint |
| 35 | + |
| 36 | +# Run all pre-commit hooks |
| 37 | +make pre-commit |
| 38 | +``` |
| 39 | + |
| 40 | +### Testing |
| 41 | +```bash |
| 42 | +# Run all tests |
| 43 | +make test |
| 44 | + |
| 45 | +# Or directly with pytest |
| 46 | +python -m pytest . --disable-pytest-warnings |
| 47 | + |
| 48 | +# Run specific test file |
| 49 | +pytest shrimpy/tests/test_mantis_logger.py |
| 50 | +``` |
| 51 | + |
| 52 | +### Running the Mantis GUI |
| 53 | +```bash |
| 54 | +# Launch the GUI-based acquisition interface |
| 55 | +python -m shrimpy.mantis.launch_mantis_gui |
| 56 | +``` |
| 57 | + |
| 58 | +### Demo Mode Acquisition (Legacy) |
| 59 | +The legacy CLI is archived but provides a pattern for programmatic acquisition: |
| 60 | +```bash |
| 61 | +shrimpy acquire mantis \ |
| 62 | + --config-filepath examples/acquisition_settings/example_mda_sequence.yaml \ |
| 63 | + --output-dirpath ./YYYY_MM_DD_experiment/acquisition_name \ |
| 64 | + --mm-config-filepath path/to/MMConfig_Demo.cfg |
| 65 | +``` |
| 66 | + |
| 67 | +## Architecture |
| 68 | + |
| 69 | +### Microscope Module Structure |
| 70 | +``` |
| 71 | +shrimpy/ |
| 72 | +├── mantis/ # Label-free + Light-sheet microscope (fully implemented) |
| 73 | +│ ├── mantis_engine.py # MDAEngine subclass (~455 lines) |
| 74 | +│ ├── mantis_acquisition_widget.py # Qt GUI (~815 lines) |
| 75 | +│ ├── mantis_logger.py # Logging configuration |
| 76 | +│ ├── launch_mantis_gui.py # GUI entry point |
| 77 | +│ └── archive/ # Historical implementations (pycromanager, old pymmcore-plus) |
| 78 | +│ |
| 79 | +├── isim/ # iSIM microscope (placeholder for future implementation) |
| 80 | +├── viewer/ # Data visualization (placeholder) |
| 81 | +├── cli/ # Command-line interface (in transition, currently empty) |
| 82 | +└── tests/ # Unit tests |
| 83 | +``` |
| 84 | + |
| 85 | +### Key Design Patterns |
| 86 | + |
| 87 | +#### 1. Engine Abstraction Pattern |
| 88 | +Each microscope implements a custom `MDAEngine` subclass: |
| 89 | +```python |
| 90 | +class MantisEngine(MDAEngine): |
| 91 | + def setup_sequence(sequence: MDASequence) -> SummaryMetaV1: |
| 92 | + # Configure hardware before acquisition starts |
| 93 | + # - Set ROI, focus device, initialization settings |
| 94 | + # - Configure hardware sequencing |
| 95 | + # - Setup autofocus parameters |
| 96 | + |
| 97 | + def setup_event(event: MDAEvent): |
| 98 | + # Prepare for each acquisition event |
| 99 | + # - Configure TriggerScope if using hardware sequencing |
| 100 | + |
| 101 | + def _set_event_xy_position(event: MDAEvent): |
| 102 | + # Custom XY positioning with intelligent stage movement |
| 103 | + # - Variable speed (2.0 mm/s short, 5.75 mm/s long distances) |
| 104 | + # - Post-movement autofocus engagement with retry logic |
| 105 | + # - Stage settlement waiting |
| 106 | +``` |
| 107 | + |
| 108 | +To add a new microscope: |
| 109 | +1. Create `shrimpy/<microscope_name>/` directory |
| 110 | +2. Subclass `MDAEngine` in `<microscope_name>_engine.py` |
| 111 | +3. Override `setup_sequence()`, `setup_event()`, and positioning methods as needed |
| 112 | +4. Define microscope-specific metadata schema |
| 113 | +5. Create Qt widget for GUI (optional) |
| 114 | + |
| 115 | +#### 2. Metadata Propagation Pattern |
| 116 | +Configuration is passed through MDASequence metadata: |
| 117 | +```python |
| 118 | +sequence = MDASequence.from_file('config.yaml') |
| 119 | +sequence.metadata = { |
| 120 | + 'mantis': { |
| 121 | + 'roi': [x, y, width, height], |
| 122 | + 'z_stage': 'AP Galvo', |
| 123 | + 'initialization_settings': [[device, property, value], ...], |
| 124 | + 'setup_hardware_sequencing_settings': [...], |
| 125 | + 'autofocus': { |
| 126 | + 'enabled': True, |
| 127 | + 'stage': 'ZDrive', |
| 128 | + 'method': 'PFS', # Nikon Perfect Focus System |
| 129 | + 'wait_after_correction': 0.5, |
| 130 | + 'wait_before_acquire': 0.1, |
| 131 | + }, |
| 132 | + } |
| 133 | +} |
| 134 | +``` |
| 135 | + |
| 136 | +#### 3. Logging Pattern |
| 137 | +Each microscope module uses a separate logger instance: |
| 138 | +```python |
| 139 | +from shrimpy.mantis.mantis_logger import configure_mantis_logger, get_mantis_logger |
| 140 | + |
| 141 | +# During acquisition setup |
| 142 | +logger = configure_mantis_logger(save_dir, 'acquisition_name') |
| 143 | +# Creates dual handlers: |
| 144 | +# - Console: INFO level |
| 145 | +# - File: DEBUG level (saved to logs/ subdirectory) |
| 146 | + |
| 147 | +# Also captures pymmcore-plus logger to same file |
| 148 | +``` |
| 149 | + |
| 150 | +Use `logger.debug()` for detailed diagnostics (file only) and `logger.info()` for user-facing messages (console + file). |
| 151 | + |
| 152 | +### Configuration Files |
| 153 | + |
| 154 | +Acquisitions are configured using YAML files that define MDASequence parameters plus microscope-specific metadata. Examples in `examples/acquisition_settings/`. |
| 155 | + |
| 156 | +**Key Configuration Sections:** |
| 157 | +- `time_plan`: Timepoint intervals and loops |
| 158 | +- `channels`: Channel configurations |
| 159 | +- `z_plan`: Z-stack range and step size |
| 160 | +- `stage_positions`: XY positions (optional) |
| 161 | +- `metadata.mantis`: Mantis-specific settings (ROI, autofocus, hardware sequencing) |
| 162 | + |
| 163 | +See `examples/acquisition_settings/example_mda_sequence.yaml` for a minimal example. |
| 164 | + |
| 165 | +### Widget Composition (Qt GUI) |
| 166 | +``` |
| 167 | +MantisAcquisitionWidget (main container) |
| 168 | +├── ImagePreview (from pymmcore-widgets) |
| 169 | +├── CustomCameraRoiWidget (workaround for camera snap issues) |
| 170 | +├── StageWidget (XY and Z stage control) |
| 171 | +├── MDAWidget (standard multi-dimensional acquisition configuration) |
| 172 | +└── MantisSettingsWidget |
| 173 | + ├── TriggerScopeSettingsWidget (hardware triggering) |
| 174 | + └── MicroscopeSettingsWidget (focus device, autofocus, hardware sequencing) |
| 175 | +``` |
| 176 | + |
| 177 | +Widgets communicate via Qt signals/slots. Settings are propagated to MDASequence metadata before acquisition starts. |
| 178 | + |
| 179 | +## Key Dependencies |
| 180 | + |
| 181 | +- **pymmcore-plus** (0.17.0): Python bindings for Micro-Manager with MDA engine |
| 182 | +- **pymmcore-widgets**: Qt widgets for microscope control |
| 183 | +- **useq-schema**: Multi-dimensional acquisition sequence specification |
| 184 | +- **PyYAML**: Configuration parsing |
| 185 | +- **numpy**: Numerical operations |
| 186 | +- **qtpy**: Qt abstraction layer (PyQt5/6, PySide2/6) |
| 187 | + |
| 188 | +Optional (for analysis, not in core package): |
| 189 | +- **biahub**: Image analysis library (deskewing, reconstruction, registration) |
| 190 | +- **iohub**: OME-Zarr conversion and metadata management |
| 191 | +- **recOrder**: Phase and orientation reconstruction |
| 192 | +- **VisCy**: Virtual staining |
| 193 | + |
| 194 | +## Code Style |
| 195 | + |
| 196 | +- **Formatter**: black with line length 95, Python 3.11, skip string normalization (`-S`) |
| 197 | +- **Import sorting**: isort (black profile) |
| 198 | +- **Linter**: flake8 (disabled: C, R, W, import-error, unsubscriptable-object) |
| 199 | +- **Pre-commit hooks**: Automatically run style checks on commit |
| 200 | + |
| 201 | +Run `make format` before committing. The pre-commit hooks will catch violations. |
| 202 | + |
| 203 | +## Testing |
| 204 | + |
| 205 | +- Framework: pytest |
| 206 | +- Test location: `shrimpy/tests/` |
| 207 | +- Ignore: `scripts/`, `**/archive/` (configured in pyproject.toml) |
| 208 | +- Run with: `make test` or `pytest . --disable-pytest-warnings` |
| 209 | + |
| 210 | +Current tests focus on logging infrastructure. Add tests for new microscope engines in `shrimpy/tests/test_<microscope>_*.py`. |
| 211 | + |
| 212 | +## Current Development Focus |
| 213 | + |
| 214 | +**Active restructuring** (branch `215-restructure-repository-for-multi-microscope-support`): |
| 215 | +- Transitioning from mantis-only to multi-microscope framework |
| 216 | +- Archiving legacy CLI and V1/V2 acquisition engines |
| 217 | +- Establishing iSIM placeholder for future work |
| 218 | +- Maintaining GUI-first approach with programmatic API |
| 219 | + |
| 220 | +**What's stable:** |
| 221 | +- Mantis acquisition engine (MantisEngine) |
| 222 | +- GUI-based acquisition workflow |
| 223 | +- Logging infrastructure |
| 224 | +- Configuration via YAML + metadata |
| 225 | + |
| 226 | +**What's in flux:** |
| 227 | +- CLI interface (currently empty, being redesigned) |
| 228 | +- Cross-microscope abstractions |
| 229 | +- iSIM implementation |
| 230 | + |
| 231 | +## Important Implementation Notes |
| 232 | + |
| 233 | +### Mantis-Specific Behavior |
| 234 | +- **Autofocus**: Engages Nikon PFS after XY stage movements with retry logic (up to 3 attempts, 0.5s wait between) |
| 235 | +- **Stage speed**: Variable speed based on distance (2.0 mm/s for <2000 µm, 5.75 mm/s for longer moves) |
| 236 | +- **Hardware sequencing**: TriggerScope DAC/TTL control for synchronized imaging |
| 237 | +- **Dual-arm imaging**: Label-free and light-sheet acquired on separate Micro-Manager instances |
| 238 | + |
| 239 | +### Extending to New Microscopes |
| 240 | +When adding iSIM or other microscopes: |
| 241 | +1. Study `shrimpy/mantis/mantis_engine.py` as the reference implementation |
| 242 | +2. Override only the methods that differ from default MDAEngine behavior |
| 243 | +3. Document microscope-specific metadata schema in docstrings |
| 244 | +4. Create separate logger instance following mantis_logger pattern |
| 245 | +5. Keep archived code in `archive/` subdirectory for reference |
| 246 | + |
| 247 | +### Data Output |
| 248 | +Raw data follows OME-Zarr or NDTiff format. Reconstruction workflows handled by separate biahub library. See `docs/data_structure.md` for details. |
0 commit comments