Applications: Cytoland refactor by alxndrkalinin · Pull Request #379 · mehta-lab/VisCy

alxndrkalinin · 2026-02-27T21:36:10Z

Summary

Refactors the virtual staining translation module into a standalone viscy-translation application package, syncs with app-dynaclr (new DynaCLR features, data infrastructure, evaluation framework), and ports PR #280 sliding window prediction.

Translation application (Phase 26)

Extracted viscy/translation/ into applications/translation/ (viscy-translation package):

Engine: VSUNet, FcmaeUNet, AugmentedPredictionVSUNet LightningModules
Shared components extracted to viscy-utils: HCSPredictionWriter, MixedLoss
Evaluation: SegmentationMetrics2D stays app-specific
Lightning CLI: python -m viscy_translation fit --config fit.yml
21 tests: engine smoke tests, training integration (6 tests: VSUNet/FcmaeUNet with synthetic + real data), config class_path validation, inference reproducibility (HPC-gated), sliding window prediction

PR #280 port: sliding window volume prediction

Enhanced _blend_in in prediction_writer.py to support both torch.Tensor (5D) and np.ndarray (4D)
Added predict_sliding_windows() to AugmentedPredictionVSUNet for in-memory Z-sliding inference
Extracted _predict_with_tta() helper; made transforms optional (default identity)
Added getattr guard for out_stack_depth (clear error for 2D models)

app-dynaclr sync (PR #381 + #384)

Merged 108 commits from app-dynaclr:

Multi-experiment training infrastructure (ExperimentRegistry, MultiExperimentIndex, MultiExperimentDataModule)
New viscy-data modules: cell_index, collection, sampler, schemas, channel_dropout
Foundation models: DINOv3, OpenPhenom; contrastive loss: NTXentHCL
DynaCLR evaluation framework: linear classifiers, dimensionality reduction, pseudotime analysis
New applications: airtable, qc
Code review fixes: assert→ValueError, _components/ deleted, organelle→marker rename, ArrowString compat

Bug fixes

Fix operator precedence in evaluation.py
Fix source overwrite in AugmentedPredictionVSUNet TTA loop
Fix unbound return_target in FcmaeUNet.forward_fit_task
Add weights_only=True to torch.load; fix mutable default model_config
Fix numpy indexing in _blend_in; respect overwrite flag in EmbeddingWriter
Port UNeXt2Decoder Sequence mutation fix to components/blocks.py
Port UNeXt2Stem ZeroDivisionError guards to components/stems.py
Validate every_n_epochs >= 1 in EmbeddingSnapshotCallback
Fix CPU/GPU pos_weight device mismatch in ClassificationModule
Fix None norm_meta crashing default_collate in SlidingWindowDataset
Apply torch.sigmoid before binary_accuracy/binary_f1_score (logits→probs)
Rename ULTRACK_INDEX_COLUMNS → INDEX_COLUMNS; standardize select.py imports

Test results

Suite	Result
viscy-models	95 passed, 1 xfailed
viscy-data	169 passed, 2 skipped
translation	21 passed
dynaclr	141 passed, 4 skipped

See deferred items for tracked improvements.

- 24 tests covering all 8 migrated model architectures - Each model tested for parameter count, top-level prefixes, and sentinel keys - Guards COMPAT-01: state dict keys must match for checkpoint loading - Tests import from top-level viscy_models package (validates public API) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Add package dimension to test matrix (viscy-transforms, viscy-models) - Use cross-platform --cov=src/ instead of named package coverage - Matrix now produces 18 jobs (3 OS x 3 Python x 2 packages) - check job automatically aggregates all test results via alls-green Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…e complete) - Add 10-01-SUMMARY.md with execution results - Update STATE.md: phase 10 complete, v1.1 milestone done, 100% progress Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…tone complete) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Move conv_block_2d.py and conv_block_3d.py from unet/_layers/ to _components/ alongside all other shared building blocks. All reusable layers now live in one place. unet/_layers/ retained as backward- compatible re-export shim. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…ject.toml - Add pyproject.toml with hatchling build, uv-dynamic-versioning, all base deps - Declare optional dependency groups: triplet, livecell, mmap, all - Add PEP 561 py.typed marker and tests/__init__.py - Configure pattern-prefix for independent versioning Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Copy all type definitions from viscy/data/typing.py into _typing.py - Add INDEX_COLUMNS from viscy/data/triplet.py for shared access - Update typing_extensions.NotRequired to typing.NotRequired (Python >=3.11) - Create __init__.py with full re-export of all public types - Add README.md required by hatchling build Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…roject.toml - Add viscy-data to root dependencies list - Register viscy-data as workspace source in [tool.uv.sources] - Verified editable install and full import chain works Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

… update - Add 06-01-SUMMARY.md documenting viscy-data package creation - Update STATE.md with plan position, metrics, and decisions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Extract _ensure_channel_list, _search_int_in_str, _collate_samples, _read_norm_meta from hcs.py - Extract _scatter_channels, _gather_channels, _transform_channel_wise from triplet.py - Update imports to use viscy_data._typing instead of viscy.data.typing - Add __all__ listing all 7 utility functions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Add 06-02-SUMMARY.md documenting utility extraction - Update STATE.md: Phase 6 complete, progress 80% Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…scy-data - Copy select.py with well/FOV filtering utilities (no internal viscy imports) - Copy distributed.py with ShardedDistributedSampler (no internal viscy imports) - Copy segmentation.py with viscy.data.typing -> viscy_data._typing import update - Add missing docstrings to satisfy ruff D rules Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Copy HCSDataModule, SlidingWindowDataset, MaskTestDataset from main - Replace viscy.data.typing imports with viscy_data._typing - Remove 4 utility function definitions (now in _utils.py) - Add import from viscy_data._utils for shared utilities - Remove unused re and collate_meta_tensor imports - Add missing docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Copy GPUTransformDataModule, CachedOmeZarrDataset, CachedOmeZarrDataModule - Rewire viscy.data.distributed -> viscy_data.distributed - Rewire viscy.data.hcs utility imports -> viscy_data._utils - Rewire viscy.data.select -> viscy_data.select - Rewire viscy.data.typing -> viscy_data._typing - Add missing docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Add 07-01-SUMMARY.md documenting migration of 5 core modules - Update STATE.md: phase 7 plan 1 of 4, decisions, metrics Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Rewire all imports from viscy.data to viscy_data prefix - Add lazy import for tensordict with clear error message - Add docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Rewire imports from viscy.data to viscy_data prefix - Add lazy imports for pycocotools, tifffile, torchvision - Add import guards in LiveCellDataset and LiveCellTestDataset __init__ - Add docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Rewire viscy.data.distributed to viscy_data.distributed - Rewire viscy.data.hcs._collate_samples to viscy_data._utils._collate_samples - Preserve all 6 public classes without structural changes - Add docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Rewire imports from viscy.data to viscy_data prefix - Add lazy import for pandas in cell_classification.py with clear error message - Import _transform_channel_wise from viscy_data._utils (not triplet.py) - Import INDEX_COLUMNS and AnnotationColumns from viscy_data._typing - Add docstrings for ruff D compliance

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…ems.py Copy the in_stack_depth and out_channels divisibility checks from _components/stems.py to components/stems.py before the upcoming merge deletes _components/. Without these guards, invalid parameters cause a silent ZeroDivisionError. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

7 commits from app-dynaclr (PR #384): - Delete duplicate _components/ directory (fixes ported to components/) - Replace assert with ValueError in conv_block_2d/3d - Add type annotations to StemDepthtoChannels.compute_stem_channels - Narrow except Exception to (OSError, ValueError) in HCSDataModule - Fix ArrowStringArray compatibility in embedding_writer and test conftest - Rename organelle→marker channel type across configs and tests - Add new analysis scripts and SLURM configs for linear classifiers Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

alxndrkalinin · 2026-03-18T00:30:44Z

Deferred items from code review

The following issues were identified during Copilot and Claude code reviews. All actionable bugs have been resolved in recent merges (PR #381, #384, PR #280 port). These remaining items are acknowledged as out-of-scope improvements to address in future work.

Resolved since last review:

_components/ dead code -- directory deleted in PR Sync app-dynaclr code review fixes into app-cytoland #384; all fixes ported to components/
UNeXt2Decoder.forward mutation -- fixed in components/blocks.py
UNeXt2Stem ZeroDivisionError -- validation guards ported to components/stems.py

Architecture / dependency hygiene:

embedding_snapshot.py imports viscy_data at runtime but viscy-utils does not declare viscy-data as a dependency. Works because applications depend on both, but architecturally incorrect.
embedding_snapshot.py uses EmbeddingWriter._collect_data_provenance (private API). Should be extracted to a public utility function.
embedding_snapshot.py re-opens zarr store after writing for obsm/uns updates. Could be optimized by extending write_embedding_dataset() to accept optional obsm/uns payloads.
wandb and anndata promoted to core dependencies in viscy-utils but are only used in optional evaluation code (evaluation/linear_classifier.py). Every consumer (including viscy-translation) now installs them unconditionally. wandb is heavy (~50 MB, requires credentials). Should be moved back to optional extras.
Root pyproject.toml sets pythonpath = ["applications/dynaclr/tests"] globally, leaking dynaclr's helpers.py module into all root-level pytest sessions. No conflict today but fragile. Should be scoped to dynaclr only (e.g., via conftest.py sys.path hook or moving helpers into a proper subpackage).

Minor code quality:

classification.py:102 uses deprecated nn.functional.sigmoid in predict_step. Should use torch.sigmoid.
qc_metrics.py uses print instead of logging. Should switch when CLI interface is finalized.
annotation.py convert() has no unit tests for overwrite semantics or obsm construction.
annotation.py uses np.column_stack on xarray coords without explicit .data conversion (works but fragile).

Design choices (not bugs):

qc_metrics.py hardcodes focus-specific naming (all_focus_values, z_focus_*). Acceptable while QC app is focus-only; revisit when more metric types are added.
test_engine.py depends on external grep executable. Acceptable for Linux-only HPC project.
cli.py isdisjoint logic was flagged as inverted -- verified correct.
annotation.py reads projections as coords, features as data_vars -- by design (EmbeddingWriter stores 1D projections as coords).

alxndrkalinin · 2026-03-18T00:36:06Z

@edyoshikun should be ready for your final review

…nal dependency

… pytest Dependency fixes (all were missing from pyproject.toml): - viscy-utils: add wandb, anndata, tensorboard as core deps; remove redundant optional-dependencies.anndata group - viscy-data: promote pandas and pyarrow from optional/test to core deps - viscy-models: add pytorch-metric-learning as core dep - dynaclr: add statsmodels to eval and test dep groups Test infrastructure: - Extract shared test helpers/constants from conftest.py into helpers.py so tests can import them under --import-mode=importlib (from conftest import broke when running pytest from repo root) - Remove pythonpath=["tests"] from dynaclr pyproject.toml (was a workaround for running from the app dir; root config now handles it) - Add pythonpath=["applications/dynaclr/tests"] to root pytest config - Move pd.options.future.infer_string=False into pytest_configure hook so imports stay at top level (fixes ruff E402) Bug fixes: - hcs.py: scope propagate=False to viscy_data.hcs.cache logger instead of viscy_data parent, which was breaking caplog in downstream tests - Remove test_small_group_emits_warning: tested log wording not behavior - Remove test_nan_gene_name_to_ntc: tested pandas fillna not module code Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

--import-mode=importlib does not use package-style imports for test files, so __init__.py in test directories is unnecessary and can cause import collisions. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

- Wrap open_ome_zarr() in context managers in index.py and cell_index.py - Raise RuntimeError in _sample_positives() when no positive found instead of silent self-positive fallback - Raise FileNotFoundError when tracking CSV is missing (fail fast before training) - Raise ValueError when multiple CSVs exist in a tracks dir - Update test_empty_tracks_empty_anchors → test_missing_tracking_csv_raises Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

edyoshikun

we can merge to the staging and then delete .planning

Remove old Hydra output logs and cache files from the defunct dynacell application. This directory was already excluded from the uv workspace. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

…es + dataset notebook (#449) * VisCy Modular: Transforms and Monorepo Skeleton (#356) * refactor: restructure viscy into uv workspace monorepo with viscy-transforms subpackage BREAKING CHANGE: - Import path changed: from viscy.transforms import X → from viscy_transforms import X - Removed legacy modules: viscy.utils, viscy.cli, viscy.unet, viscy.evaluation - Removed applications/, examples/, and docs/ directories Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org> * docs: updated readme with citations / examples from main Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org> * docs: added symlinking uv cache on hpc systems Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org> * add the jupyter and ipykernel to a optional visual group * re organize the transforms * add jupyternotebook back * build: updated some dep groups Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org> * build: added matplotlib back in Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org> * build: add ruff and prek to the dev dep group * docs: correct the docstring for the transform that doesn't exist lol Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org> --------- Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org> Co-authored-by: Sricharan Reddy Varra <sricharan.varra@biohub.org> Co-authored-by: Eduardo Hirata-Miyasaki <edhiratam@gmail.com> Co-authored-by: Alexandr Kalinin <alxndrkalinin@users.noreply.github.com> * docs: map existing codebase * Modularizing the model's folder (#365) * add planning roadmap * docs: start milestone v1.1 Models * docs: define milestone v1.1 requirements * docs: create milestone v1.1 roadmap (5 phases) * docs(package-scaffold-shared-components): research phase domain * docs(06): create phase plan - package scaffold and shared components * feat(06-01): create viscy-models package scaffold - Add pyproject.toml with hatchling build, torch/timm/monai/numpy deps - Create src layout with _components, unet, contrastive, vae subpackages - Add PEP 561 py.typed marker - Add test scaffolding with device fixture Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(06-01): register viscy-models in workspace - Add viscy-models to root dependencies and uv sources - Update lockfile with timm and viscy-models dependencies - Verified: uv sync, import, and pytest collection all succeed Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(06-01): complete package scaffold plan - Add 06-01-SUMMARY.md with execution results - Update STATE.md with position, metrics, and decisions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(06-02): extract shared components into _components/ module - stems.py: UNeXt2Stem, StemDepthtoChannels from v0.3.3 unext2.py - heads.py: PixelToVoxelHead, UnsqueezeHead, PixelToVoxelShuffleHead - blocks.py: icnr_init, _get_convnext_stage, UNeXt2UpStage, UNeXt2Decoder - __init__.py: re-exports all 8 public components - Zero imports from unet/, vae/, or contrastive/ - All attribute names preserved for state dict compatibility * feat(06-03): migrate ConvBlock2D and ConvBlock3D to unet/_layers/ - Copy ConvBlock2D from v0.3.3 source to snake_case file - Copy ConvBlock3D from v0.3.3 source to snake_case file - Preserve register_modules/add_module pattern for state dict key compatibility - Update _layers/__init__.py with public re-exports - Fix docstring formatting for ruff D-series compliance * test(06-03): add tests for ConvBlock2D and ConvBlock3D - 6 tests for ConvBlock2D: forward pass, state dict keys, residual, filter steps, instance norm - 4 tests for ConvBlock3D: forward pass, state dict keys, dropout registration, layer order - All 10 tests verify shape, naming patterns, and module registration * test(06-02): add forward-pass tests for all _components - test_stems.py: UNeXt2Stem shape, StemDepthtoChannels shape + mismatch error - test_heads.py: PixelToVoxelHead, UnsqueezeHead, PixelToVoxelShuffleHead shapes - test_blocks.py: icnr_init, _get_convnext_stage, UNeXt2UpStage, UNeXt2Decoder - 10 tests total, all passing on CPU * docs(06-03): complete UNet ConvBlock layers plan - SUMMARY.md with migration details and self-check - STATE.md updated: phase 6 plan 3/3, decisions recorded * docs(06-02): complete shared components extraction plan - SUMMARY.md documents 8 extracted components with 10 tests - STATE.md updated with decisions from 06-02 execution * docs(phase-6): complete phase execution and verification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(07-core-unet-models): research phase domain * docs(07): create phase plan for core UNet models * feat(07-01): migrate UNeXt2 model class to viscy-models - Copy UNeXt2 class (~70 lines) from monolithic unext2.py - Update imports to use viscy_models._components (stems, heads, blocks) - Preserve all attribute names for state dict compatibility - Export UNeXt2 from viscy_models.unet public API Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(07-01): add 6 UNeXt2 forward-pass tests and fix deconv tuple bug - Add tests: default, small backbone, multichannel, diff stack depths, deconv, stem validation - Fix deconv decoder tuple bug in UNeXt2UpStage (trailing comma created tuple not module) - Mark deconv test xfail: original code has channel mismatch in deconv forward path - All 26 tests pass (25 passed, 1 xfailed) with no regressions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(07-01): complete UNeXt2 migration plan - Add 07-01-SUMMARY.md with execution results and deviation documentation - Update STATE.md: phase 7, plan 1/2 complete, new decisions logged Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-02): migrate FullyConvolutionalMAE to viscy-models - Copy FCMAE and all helper classes/functions to unet/fcmae.py - Replace old viscy imports with viscy_models._components imports - Remove duplicated PixelToVoxelShuffleHead (import from _components.heads) - Fix mutable list defaults to tuples (encoder_blocks, dims) - Export both UNeXt2 and FullyConvolutionalMAE from unet/__init__.py * test(07-02): migrate 11 FCMAE tests to viscy-models - Copy all 11 test functions with zero logic changes - Update imports from viscy.unet.networks.fcmae to viscy_models.unet.fcmae - Import PixelToVoxelShuffleHead from viscy_models._components.heads - All 37 tests pass across full suite (no regressions) * docs(07-02): complete FCMAE migration plan (Phase 7 complete) - Add 07-02-SUMMARY.md with execution results - Update STATE.md: Phase 7 complete, 12 plans total, decisions logged * docs(phase-7): complete phase execution and verification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-8): research representation models migration * docs(08): create phase plan for representation models * feat(08-02): migrate BetaVae25D and BetaVaeMonai to viscy-models - Add BetaVae25D with VaeUpStage, VaeEncoder, VaeDecoder helpers - Add BetaVaeMonai wrapping MONAI VarAutoEncoder - Fix VaeDecoder mutable list defaults to tuples (COMPAT-02) - Change VaeEncoder pretrained default to False - Preserve all attribute names for state dict compatibility Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(08-01): migrate ContrastiveEncoder and ResNet3dEncoder to viscy-models - Add ContrastiveEncoder with convnext/resnet50 backbone support via timm - Add ResNet3dEncoder with MONAI ResNetFeatures backend - Fix ResNet50 bug: use encoder.num_features instead of encoder.head.fc.in_features - Add pretrained parameter (default False) for pure nn.Module semantics - Preserve state dict attribute names (stem, encoder, projection) - Share projection_mlp utility between both encoder classes * test(08-01): add 5 forward-pass tests for contrastive models - 3 tests for ContrastiveEncoder: convnext_tiny, resnet50, custom stem - 2 tests for ResNet3dEncoder: resnet18, resnet10 - Verify embedding and projection output shapes - ResNet50 test uses in_stack_depth=10 for valid stem channel alignment * test(08-02): add forward-pass tests for BetaVae25D and BetaVaeMonai - 2 BetaVae25D tests: resnet50 and convnext_tiny backbones - 2 BetaVaeMonai tests: 2D and 3D spatial configurations - Verify SimpleNamespace output with recon_x, mean, logvar, z - Fix ResNet50 expected spatial dims (64x64 not 128x128) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(08-01): complete contrastive model migration plan - Add 08-01-SUMMARY.md with execution results - Update STATE.md to Phase 8, plan 1/2 * docs(08-02): complete VAE migration plan (Phase 8 complete) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-8): complete phase execution and verification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(09): research legacy UNet models migration * docs(09): create phase plan for legacy UNet models * feat(09-01): migrate Unet2d and Unet25d to viscy-models - Copy Unet2d from v0.3.3 with import path update to viscy_models.unet._layers - Copy Unet25d from v0.3.3 with import path update to viscy_models.unet._layers - Fix mutable default num_filters=[] to num_filters=() in both models - Add module docstrings and __all__ exports - Update unet/__init__.py to export all 4 models (UNeXt2, FCMAE, Unet2d, Unet25d) - Preserve register_modules/add_module pattern for state dict compatibility - All 45 existing tests still pass Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(09-01): add pytest tests for Unet2d and Unet25d - 12 tests for Unet2d: default forward, variable depth, multichannel, residual, task mode, dropout, state dict keys, custom num_filters - 11 tests for Unet25d: default Z-compression, preserved depth, variable depth, multichannel, residual, task mode, state dict keys with skip_conv_layer, custom filters - Fix list(num_filters) conversion in both models for tuple default compatibility - Total test suite: 68 passed, 1 xfailed, 0 failures Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(09-01): complete legacy UNet migration plan - SUMMARY.md with task commits, deviations, and self-check - STATE.md updated to Phase 9 complete (15 plans total) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-9): complete phase execution and verification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(10): create phase plan for public API and CI integration * feat(10-01): add top-level re-exports for all 8 model classes - Import UNeXt2, FullyConvolutionalMAE, Unet2d, Unet25d from unet subpackage - Import ContrastiveEncoder, ResNet3dEncoder from contrastive subpackage - Import BetaVae25D, BetaVaeMonai from vae subpackage - Update __all__ with all 8 classes in alphabetical order Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(10-01): add state dict key compatibility regression tests - 24 tests covering all 8 migrated model architectures - Each model tested for parameter count, top-level prefixes, and sentinel keys - Guards COMPAT-01: state dict keys must match for checkpoint loading - Tests import from top-level viscy_models package (validates public API) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(10-01): add viscy-models to CI test matrix - Add package dimension to test matrix (viscy-transforms, viscy-models) - Use cross-platform --cov=src/ instead of named package coverage - Matrix now produces 18 jobs (3 OS x 3 Python x 2 packages) - check job automatically aggregates all test results via alls-green Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(10-01): complete public API & CI integration plan (v1.1 milestone complete) - Add 10-01-SUMMARY.md with execution results - Update STATE.md: phase 10 complete, v1.1 milestone done, 100% progress Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-10): complete phase execution and verification (v1.1 milestone complete) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: consolidate ConvBlock2D/3D into _components Move conv_block_2d.py and conv_block_3d.py from unet/_layers/ to _components/ alongside all other shared building blocks. All reusable layers now live in one place. unet/_layers/ retained as backward- compatible re-export shim. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * remove the _layers * update the readme * update the main readme * fix description in toml * changing ruff formatting , dosctrings and imports * renaming folder to components * updatet planning docs * update to components * numpy docstring * add claude.md and contributing.md --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * modularizing data package (#366) * add planning roadmap * docs: start milestone v1.1 Extract viscy-data * docs: complete viscy-data project research * docs: define milestone v1.1 requirements * docs: create milestone v1.1 roadmap (4 phases) * docs(06-package-scaffolding-and-foundation): create phase plan * feat(06-01): create viscy-data package directory structure with pyproject.toml - Add pyproject.toml with hatchling build, uv-dynamic-versioning, all base deps - Declare optional dependency groups: triplet, livecell, mmap, all - Add PEP 561 py.typed marker and tests/__init__.py - Configure pattern-prefix for independent versioning Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(06-01): add type definitions and package init with re-exports - Copy all type definitions from viscy/data/typing.py into _typing.py - Add INDEX_COLUMNS from viscy/data/triplet.py for shared access - Update typing_extensions.NotRequired to typing.NotRequired (Python >=3.11) - Create __init__.py with full re-export of all public types - Add README.md required by hatchling build Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(06-01): integrate viscy-data as workspace dependency in root pyproject.toml - Add viscy-data to root dependencies list - Register viscy-data as workspace source in [tool.uv.sources] - Verified editable install and full import chain works Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(06-01): complete package scaffolding plan with summary and state update - Add 06-01-SUMMARY.md documenting viscy-data package creation - Update STATE.md with plan position, metrics, and decisions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(06-02): extract shared utility functions into _utils.py - Extract _ensure_channel_list, _search_int_in_str, _collate_samples, _read_norm_meta from hcs.py - Extract _scatter_channels, _gather_channels, _transform_channel_wise from triplet.py - Update imports to use viscy_data._typing instead of viscy.data.typing - Add __all__ listing all 7 utility functions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(06-02): complete utility module extraction plan - Add 06-02-SUMMARY.md documenting utility extraction - Update STATE.md: Phase 6 complete, progress 80% Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-6): complete phase execution * docs(07-code-migration): create phase plan * feat(07-01): migrate select.py, distributed.py, segmentation.py to viscy-data - Copy select.py with well/FOV filtering utilities (no internal viscy imports) - Copy distributed.py with ShardedDistributedSampler (no internal viscy imports) - Copy segmentation.py with viscy.data.typing -> viscy_data._typing import update - Add missing docstrings to satisfy ruff D rules Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-01): migrate hcs.py to viscy-data with utility import rewiring - Copy HCSDataModule, SlidingWindowDataset, MaskTestDataset from main - Replace viscy.data.typing imports with viscy_data._typing - Remove 4 utility function definitions (now in _utils.py) - Add import from viscy_data._utils for shared utilities - Remove unused re and collate_meta_tensor imports - Add missing docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-01): migrate gpu_aug.py to viscy-data with dependency rewiring - Copy GPUTransformDataModule, CachedOmeZarrDataset, CachedOmeZarrDataModule - Rewire viscy.data.distributed -> viscy_data.distributed - Rewire viscy.data.hcs utility imports -> viscy_data._utils - Rewire viscy.data.select -> viscy_data.select - Rewire viscy.data.typing -> viscy_data._typing - Add missing docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(07-01): complete core data module migration plan - Add 07-01-SUMMARY.md documenting migration of 5 core modules - Update STATE.md: phase 7 plan 1 of 4, decisions, metrics Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-03): migrate mmap_cache.py and ctmc_v1.py to viscy-data - Rewire all imports from viscy.data to viscy_data prefix - Add lazy import for tensordict with clear error message - Add docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-03): migrate livecell.py with lazy optional dependency imports - Rewire imports from viscy.data to viscy_data prefix - Add lazy imports for pycocotools, tifffile, torchvision - Add import guards in LiveCellDataset and LiveCellTestDataset __init__ - Add docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-03): migrate combined.py as-is with import rewiring - Rewire viscy.data.distributed to viscy_data.distributed - Rewire viscy.data.hcs._collate_samples to viscy_data._utils._collate_samples - Preserve all 6 public classes without structural changes - Add docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-02): migrate cell_classification.py and cell_division_triplet.py - Rewire imports from viscy.data to viscy_data prefix - Add lazy import for pandas in cell_classification.py with clear error message - Import _transform_channel_wise from viscy_data._utils (not triplet.py) - Import INDEX_COLUMNS and AnnotationColumns from viscy_data._typing - Add docstrings for ruff D compliance * docs(07-03): complete optional dependency module migration plan Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(07-02): complete specialized module migration plan - Add 07-02-SUMMARY.md documenting triplet, classification, and cell division module migration - Update STATE.md with position, decisions, and metrics Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-04): add complete public API exports to viscy_data __init__.py - Export all 45 public names (17 types, 2 utilities, 26 DataModules/Datasets/enums) - Eager imports from all 13 modules (lazy guards handled internally by each module) - Comprehensive __all__ list for IDE autocompletion and star-import support - Ruff-sorted import ordering passes all lint checks Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(07-04): complete public API exports plan - phase 7 fully done - 07-04-SUMMARY.md documenting 45 public exports and full package verification - STATE.md updated: phase 7 complete (4/4 plans), 12 total plans done Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-7): complete code migration execution * docs(08-test-migration-and-validation): create phase plan * test(08-01): add conftest.py with HCS OME-Zarr fixtures for viscy-data - Copy all 6 fixtures and _build_hcs helper from main branch conftest - Replace legacy np.random.rand with np.random.default_rng (NPY002) - No viscy import changes needed (only uses third-party libs) - Provides preprocessed_hcs_dataset, small_hcs_dataset, tracks fixtures Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(08-02): complete smoke tests plan - phase 8 test migration done - Created 08-02-SUMMARY.md documenting 52 smoke tests for viscy_data - Updated STATE.md: phase 8 complete, 14 total plans executed - DATA-TST-02 satisfied: import, __all__, optional dep messages, no legacy namespace Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(08-01): migrate test_hcs, test_triplet, test_select to viscy-data package - Update imports from viscy.data.X to viscy_data - Add BatchedCenterSpatialCropd to _utils.py (fixes batch dim handling) - Fix triplet.py to use BatchedCenterSpatialCropd instead of CenterSpatialCropd - Add tensorstore to test dependency group for triplet tests - All 19 tests pass across 3 test files Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(08-01): complete data test migration plan summary - Create 08-01-SUMMARY.md documenting test migration and bug fixes - Update STATE.md with BatchedCenterSpatialCropd decision revision Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-8): complete test migration and validation * docs(09-ci-integration): create phase plan * feat(09-01): add viscy-data CI test jobs to GitHub Actions workflow - Add test-data job with 3x3 matrix (3 OS x 3 Python) for viscy-data - Add test-data-extras job (ubuntu-latest, Python 3.13) for extras validation - Update check job needs to aggregate all test jobs: test, test-data, test-data-extras Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(09-01): complete CI integration plan - Add 09-01-SUMMARY.md documenting viscy-data CI jobs - Update STATE.md: phase 9 complete, v1.0 milestone complete Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-9): complete CI integration - milestone v1.1 done * chore: complete v1.1 milestone — Extract viscy-data Delivered: viscy-data package with 15 modules, 45 public exports, optional dependency groups, 71 tests, and tiered CI. Archives: - milestones/v1.1-ROADMAP.md - milestones/v1.1-REQUIREMENTS.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add missing pandas guards, restore conftest fixture, remove no-op CI filter - Add `if pd is None` guard in ClassificationDataModule.setup() and TripletDataModule._align_tracks_tables_with_positions() to raise helpful ImportError instead of AttributeError when pandas is absent - Fix ClassificationDataset error message to suggest `pip install pandas` instead of `pip install 'viscy-data[triplet]'` (classification doesn't need tensorstore) - Restore `num_timepoints` parameter on `_build_hcs()` and add `temporal_hcs_dataset` fixture from upstream commit 44b25b9 - Remove no-op `-m "not slow"` from test-data-extras CI job (no tests use @pytest.mark.slow) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add CLAUDE.md and update CONTRIBUTING.md Add CLAUDE.md with project-specific instructions for Claude Code sessions. Update CONTRIBUTING.md with ruff config centralization warning and numpy docstring convention note. Synced from 71009b5. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update uv.lock after rebase onto modular-viscy-staging Regenerate lockfile to include viscy-data workspace dependencies alongside viscy-models from the updated base branch. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * port changes from tests-zarrv3 branch * ruff * add additional test for the main dataloaders * redundant tets 3. I think test 2 alreaady takes care of this. * rename INDEX_COLUMNS * remove unused LABEL classes * fix(livecell): assign transform result and avoid mutable defaults LiveCellTestDataset.__getitem__ discarded the return value of self.transform(sample), so MONAI transforms had no effect. Also replace mutable default lists in LiveCellDataModule.__init__ with None to prevent cross-instance state sharing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cell_classification): raise ValueError, tighten val_fovs type, fix mutable default - `raise (f"Unknown stage: {stage}")` raised a string instead of an exception — use `ValueError`. - `val_fovs: list[str] | None` was unconditionally indexed in `setup()` — remove the `None` option since it's always required. - `_subset(..., exclude_timepoints=[])` used a mutable default — replace with `None`. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(gpu_aug): remove _filter_fit_fovs override so exclude_fovs is applied CachedOmeZarrDataModule accepted exclude_fovs but its local _filter_fit_fovs override only filtered wells, silently ignoring excluded FOVs. Remove the override so the SelectWell mixin's implementation (which filters both wells and FOVs) is used. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: rename select.py to _select.py (private module) The module contains mostly private helpers (_filter_wells, _filter_fovs) and a mixin dataclass (SelectWell). Renaming to _select.py signals it is internal implementation detail. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: update package descriptions to "AI x Imaging" and CLAUDE.md Update viscy-data description from "virtual staining microscopy" to "AI x Imaging tasks" in README, pyproject.toml, and __init__.py. Add viscy-models test example to CLAUDE.md. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ctmc_v1): add missing prefetch_factor attribute CTMCv1DataModule.__init__ did not set self.prefetch_factor, causing an AttributeError when train_dataloader() or val_dataloader() was called (inherited from GPUTransformDataModule). Set it to None. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add functional tests for ctmc, livecell, segmentation, classification Add unit tests for the four data modules that previously only had import smoke tests: - test_ctmc_v1.py: setup, val subsample ratio, batch shape - test_livecell.py: dataset/datamodule with mock TIFF + COCO data - test_segmentation.py: paired pred/target datasets, z-slice - test_cell_classification.py: annotation CSV, FOV split, timepoint exclusion Also adds shared fixtures to conftest.py (single_channel_hcs_pair, segmentation_hcs_pair, classification_hcs_dataset). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(livecell): add missing prefetch_factor; add dataloader iteration tests Add prefetch_factor attribute to LiveCellDataModule (same fix as 97455eb for CTMC). Add batch iteration + shape validation to classification and livecell datamodule tests to match coverage patterns in test_hcs/test_triplet. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: delete leftover select.py after rename to _select.py Commit 5b132f9 renamed select.py to _select.py but did not remove the original file. Nothing imports from it. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use explicit positions[0] instead of leaked loop variable CachedOmeZarrDataset and MmappedDataset both built self.channels using the loop variable `position` after iterating, implicitly depending on the last element. Use positions[0] to be explicit and avoid UnboundLocalError on empty input. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(livecell): handle empty annotations and correct return type Guard torch.stack against empty annotation lists in LiveCellTestDataset.__getitem__ when load_labels=True, returning properly shaped empty tensors instead of crashing. Fix _parse_image_names return type annotation: list[Path] -> list[str]. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(segmentation): defer open_ome_zarr from __init__ to setup() Store only paths in __init__ and open OME-Zarr stores in setup("test"), consistent with other DataModules in the package and Lightning conventions for resource lifecycle. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * print to logger * refactor(triplet): remove BatchedCenterSpatialCropd from viscy-data The transform already exists in viscy-transforms and viscy-data should not depend on it. Replace the final crop with a shape validation check in on_after_batch_transfer() and require initial_yx_patch_size to match the desired output size. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(docs): convert Sphinx-style docstrings to numpy style in _utils.py Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(init): lazy-load submodules via PEP 562 __getattr__ Replace eager imports of all DataModule/Dataset submodules with on-demand loading. Modules with optional dependencies (triplet, livecell, mmap_cache) are no longer imported at `import viscy_data` time. Add __init__.pyi stub for type-checker/IDE support. Also split CI test-data job to run without --all-extras so the base package is validated independently from optional dependencies. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: guard None dereferences and replace mutable default arguments - cell_classification.py: guard _read_norm_meta() returning None - hcs.py: guard MaskTestDataset with ground_truth_masks=None, add missing array_key parameter, replace mutable default [] with None - triplet.py, cell_division_triplet.py: replace mutable default [] with None for normalizations/augmentations parameters Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: correct return type annotations in _utils.py _gather_channels and _transform_channel_wise return Tensor, not list[Tensor]. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(combined): check stage before calling dm.setup() Move the unsupported-stage guard to the top of setup() in ConcatDataModule and CachedConcatDataModule so constituent data modules are not set up for stages that will be rejected. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add tests for cell_division_triplet, mmap_cache, and HCS test stage - test_cell_division_triplet.py: 11 smoke tests for dataset and datamodule - test_mmap_cache.py: 5 smoke tests (skipped when tensordict missing) - test_hcs.py: add setup("test") coverage for MaskTestDataset Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Eduardo Hirata-Miyasaki <edhiratam@gmail.com> Co-authored-by: Alexandr Kalinin <alxndrkalinin@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Applications: Cytoland refactor (#379) * test(10-01): add state dict key compatibility regression tests - 24 tests covering all 8 migrated model architectures - Each model tested for parameter count, top-level prefixes, and sentinel keys - Guards COMPAT-01: state dict keys must match for checkpoint loading - Tests import from top-level viscy_models package (validates public API) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(10-01): add viscy-models to CI test matrix - Add package dimension to test matrix (viscy-transforms, viscy-models) - Use cross-platform --cov=src/ instead of named package coverage - Matrix now produces 18 jobs (3 OS x 3 Python x 2 packages) - check job automatically aggregates all test results via alls-green Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(10-01): complete public API & CI integration plan (v1.1 milestone complete) - Add 10-01-SUMMARY.md with execution results - Update STATE.md: phase 10 complete, v1.1 milestone done, 100% progress Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-10): complete phase execution and verification (v1.1 milestone complete) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: consolidate ConvBlock2D/3D into _components Move conv_block_2d.py and conv_block_3d.py from unet/_layers/ to _components/ alongside all other shared building blocks. All reusable layers now live in one place. unet/_layers/ retained as backward- compatible re-export shim. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * remove the _layers * update the readme * docs: start milestone v1.1 Extract viscy-data * update the main readme * docs: complete viscy-data project research * docs: define milestone v1.1 requirements * docs: create milestone v1.1 roadmap (4 phases) * docs(06-package-scaffolding-and-foundation): create phase plan * feat(06-01): create viscy-data package directory structure with pyproject.toml - Add pyproject.toml with hatchling build, uv-dynamic-versioning, all base deps - Declare optional dependency groups: triplet, livecell, mmap, all - Add PEP 561 py.typed marker and tests/__init__.py - Configure pattern-prefix for independent versioning Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(06-01): add type definitions and package init with re-exports - Copy all type definitions from viscy/data/typing.py into _typing.py - Add INDEX_COLUMNS from viscy/data/triplet.py for shared access - Update typing_extensions.NotRequired to typing.NotRequired (Python >=3.11) - Create __init__.py with full re-export of all public types - Add README.md required by hatchling build Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(06-01): integrate viscy-data as workspace dependency in root pyproject.toml - Add viscy-data to root dependencies list - Register viscy-data as workspace source in [tool.uv.sources] - Verified editable install and full import chain works Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(06-01): complete package scaffolding plan with summary and state update - Add 06-01-SUMMARY.md documenting viscy-data package creation - Update STATE.md with plan position, metrics, and decisions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(06-02): extract shared utility functions into _utils.py - Extract _ensure_channel_list, _search_int_in_str, _collate_samples, _read_norm_meta from hcs.py - Extract _scatter_channels, _gather_channels, _transform_channel_wise from triplet.py - Update imports to use viscy_data._typing instead of viscy.data.typing - Add __all__ listing all 7 utility functions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(06-02): complete utility module extraction plan - Add 06-02-SUMMARY.md documenting utility extraction - Update STATE.md: Phase 6 complete, progress 80% Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-6): complete phase execution * docs(07-code-migration): create phase plan * feat(07-01): migrate select.py, distributed.py, segmentation.py to viscy-data - Copy select.py with well/FOV filtering utilities (no internal viscy imports) - Copy distributed.py with ShardedDistributedSampler (no internal viscy imports) - Copy segmentation.py with viscy.data.typing -> viscy_data._typing import update - Add missing docstrings to satisfy ruff D rules Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-01): migrate hcs.py to viscy-data with utility import rewiring - Copy HCSDataModule, SlidingWindowDataset, MaskTestDataset from main - Replace viscy.data.typing imports with viscy_data._typing - Remove 4 utility function definitions (now in _utils.py) - Add import from viscy_data._utils for shared utilities - Remove unused re and collate_meta_tensor imports - Add missing docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-01): migrate gpu_aug.py to viscy-data with dependency rewiring - Copy GPUTransformDataModule, CachedOmeZarrDataset, CachedOmeZarrDataModule - Rewire viscy.data.distributed -> viscy_data.distributed - Rewire viscy.data.hcs utility imports -> viscy_data._utils - Rewire viscy.data.select -> viscy_data.select - Rewire viscy.data.typing -> viscy_data._typing - Add missing docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(07-01): complete core data module migration plan - Add 07-01-SUMMARY.md documenting migration of 5 core modules - Update STATE.md: phase 7 plan 1 of 4, decisions, metrics Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-03): migrate mmap_cache.py and ctmc_v1.py to viscy-data - Rewire all imports from viscy.data to viscy_data prefix - Add lazy import for tensordict with clear error message - Add docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-03): migrate livecell.py with lazy optional dependency imports - Rewire imports from viscy.data to viscy_data prefix - Add lazy imports for pycocotools, tifffile, torchvision - Add import guards in LiveCellDataset and LiveCellTestDataset __init__ - Add docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-03): migrate combined.py as-is with import rewiring - Rewire viscy.data.distributed to viscy_data.distributed - Rewire viscy.data.hcs._collate_samples to viscy_data._utils._collate_samples - Preserve all 6 public classes without structural changes - Add docstrings for ruff D compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-02): migrate cell_classification.py and cell_division_triplet.py - Rewire imports from viscy.data to viscy_data prefix - Add lazy import for pandas in cell_classification.py with clear error message - Import _transform_channel_wise from viscy_data._utils (not triplet.py) - Import INDEX_COLUMNS and AnnotationColumns from viscy_data._typing - Add docstrings for ruff D compliance * docs(07-03): complete optional dependency module migration plan Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(07-02): complete specialized module migration plan - Add 07-02-SUMMARY.md documenting triplet, classification, and cell division module migration - Update STATE.md with position, decisions, and metrics Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(07-04): add complete public API exports to viscy_data __init__.py - Export all 45 public names (17 types, 2 utilities, 26 DataModules/Datasets/enums) - Eager imports from all 13 modules (lazy guards handled internally by each module) - Comprehensive __all__ list for IDE autocompletion and star-import support - Ruff-sorted import ordering passes all lint checks Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(07-04): complete public API exports plan - phase 7 fully done - 07-04-SUMMARY.md documenting 45 public exports and full package verification - STATE.md updated: phase 7 complete (4/4 plans), 12 total plans done Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-7): complete code migration execution * docs(08-test-migration-and-validation): create phase plan * test(08-01): add conftest.py with HCS OME-Zarr fixtures for viscy-data - Copy all 6 fixtures and _build_hcs helper from main branch conftest - Replace legacy np.random.rand with np.random.default_rng (NPY002) - No viscy import changes needed (only uses third-party libs) - Provides preprocessed_hcs_dataset, small_hcs_dataset, tracks fixtures Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(08-02): complete smoke tests plan - phase 8 test migration done - Created 08-02-SUMMARY.md documenting 52 smoke tests for viscy_data - Updated STATE.md: phase 8 complete, 14 total plans executed - DATA-TST-02 satisfied: import, __all__, optional dep messages, no legacy namespace Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(08-01): migrate test_hcs, test_triplet, test_select to viscy-data package - Update imports from viscy.data.X to viscy_data - Add BatchedCenterSpatialCropd to _utils.py (fixes batch dim handling) - Fix triplet.py to use BatchedCenterSpatialCropd instead of CenterSpatialCropd - Add tensorstore to test dependency group for triplet tests - All 19 tests pass across 3 test files Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(08-01): complete data test migration plan summary - Create 08-01-SUMMARY.md documenting test migration and bug fixes - Update STATE.md with BatchedCenterSpatialCropd decision revision Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-8): complete test migration and validation * docs(09-ci-integration): create phase plan * feat(09-01): add viscy-data CI test jobs to GitHub Actions workflow - Add test-data job with 3x3 matrix (3 OS x 3 Python) for viscy-data - Add test-data-extras job (ubuntu-latest, Python 3.13) for extras validation - Update check job needs to aggregate all test jobs: test, test-data, test-data-extras Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(09-01): complete CI integration plan - Add 09-01-SUMMARY.md documenting viscy-data CI jobs - Update STATE.md: phase 9 complete, v1.0 milestone complete Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-9): complete CI integration - milestone v1.1 done * chore: complete v1.1 milestone — Extract viscy-data Delivered: viscy-data package with 15 modules, 45 public exports, optional dependency groups, 71 tests, and tiered CI. Archives: - milestones/v1.1-ROADMAP.md - milestones/v1.1-REQUIREMENTS.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * harmonize the planning between the modular-data and modular-models * viscy-utils package * add applications/dynaclr * update the monorepo uv * moving files around * update planning * docs: start milestone v2.1 DynaCLR Integration Validation * docs: define milestone v2.1 requirements * docs: create milestone v2.1 roadmap (2 phases) * docs(18-training-validation): create phase plan * feat(18-01): add training integration tests for ContrastiveModule - Add fast_dev_run tests for TripletMarginLoss and NTXentLoss code paths - Add parametrized config class_path resolution tests for fit.yml and predict.yml - Add tensorboard as test dependency for TensorBoardLogger in integration tests - Fix workspace exclude to skip non-package application directories - Use 2D-compatible synthetic data shapes (1,1,4,4) for render_images compatibility Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(18-01): complete training integration tests plan Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(phase-18): complete phase execution * docs(19-inference-reproducibility): create phase plan * chore(19-01): add anndata test dependency and HPC conftest fixtures - Add anndata to dynacrl test dependency group - Create conftest.py with HPC path constants, skip markers, and fixtures - Update uv.lock Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(19-01): add inference reproducibility integration tests - Create test_inference_reproducibility.py with 2 HPC integration tests - test_checkpoint_loads_into_modular_contrastive_module (INFER-01) - test_predict_embeddings_and_exact_match (INFER-02 + INFER-03) - Fix lazy imports in EmbeddingWriter to avoid unconditional umap import - Fix anndata nullable string compatibility in write_embedding_dataset - Tests skip gracefully when HPC paths or GPU unavailable Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(19-01): complete inference reproducibility plan - Add 19-01-SUMMARY.md with execution results and deviation documentation - Update STATE.md: Phase 19 complete, v2.1 milestone finished Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add seed_everything(42) to all integration tests Ensures reproducibility by seeding all tests consistently. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(phase-19): complete phase execution * restructure the examples folder and ruff * update readme.me hallucination * update the readmes * - Add `viscy` console script in viscy-utils pointing to viscy_utils.cli:main - Add jsonargparse[signatures] dependency for LightningCLI - Add 4 CLI smoke tests (help, subcommands, fit --help, predict --help) - Replace conda/anaconda with uv in SLURM scripts - Update SLURM scripts to use `viscy fit/predict` instead of old monolith * add the CLI for running training and prediction * default embedding writer to None * import within the function * ruff * dynaclr typo * rename folder to dynaclr * add the classifiers here * docs: start milestone v2.2 Composable Sampling Framework * docs: define milestone v2.2 requirements * docs: create milestone v2.2 roadmap (6 phases) * docs(20): capture phase context * docs(20): create phase plan for experiment configuration * test(20-01): add failing tests for ExperimentConfig and ExperimentRegistry - 19 test cases covering config creation, defaults, channel maps, validation errors, YAML loading, tau-range conversion, and lookups - All tests fail with ModuleNotFoundError (module not yet implemented) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(20-01): implement ExperimentConfig and ExperimentRegistry - ExperimentConfig dataclass with all fields and defaults - ExperimentRegistry with fail-fast validation at __post_init__: empty check, duplicate names, source_channel membership, channel count consistency, interval_minutes positivity, condition_wells non-empty, data_path existence, zarr channel match - channel_maps: per-experiment source position -> zarr index mapping - from_yaml classmethod for YAML config loading - tau_range_frames for hours-to-frames conversion with warning - get_experiment lookup by name with KeyError - All 19 tests pass Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(20-01): clean up imports and exclude stale dynacrl workspace member - Fix ruff I001 (import sorting) and F401 (unused import) in test file - Exclude applications/dynacrl (typo) from uv workspace to unblock builds Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(20-01): complete ExperimentConfig/ExperimentRegistry plan - SUMMARY.md with TDD execution results, self-check passed - STATE.md updated with position, decisions, session continuity Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(20-02): add explicit deps and top-level experiment API exports - Add iohub>=0.3a2 and pyyaml as explicit dependencies in dynaclr pyproject.toml - Re-export ExperimentConfig and ExperimentRegistry from dynaclr __init__.py - Both classes now importable via `from dynaclr import ExperimentConfig, ExperimentRegistry` Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(20-02): add example multi-experiment YAML configuration - Demonstrate positional channel alignment across 2 experiments - SEC61 (30min interval, ER) and TOMM20 (15min interval, mito) - Show condition_wells with infected/uninfected/mock conditions - Include comments explaining channel alignment and tau_range conversion Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(20-02): complete package wiring and example config plan - SUMMARY.md with execution results and self-check - STATE.md updated: Phase 20 complete, 20/25 phases (80%) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(phase-20): complete phase execution Phase 20 Experiment Configuration verified (11/11 must-haves). ExperimentConfig + ExperimentRegistry with TDD, package wiring, example YAML. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(21): create phase plan for Cell Index & Lineage * test(21-01): add failing tests for MultiExperimentIndex - 17 test cases covering CELL-01 (unified tracks), CELL-02 (lineage), CELL-03 (border clamping) - All fail with ModuleNotFoundError (dynaclr.index not yet implemented) - Test fixtures create mini OME-Zarr stores with tracking CSVs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(21-01): implement MultiExperimentIndex with lineage and border clamping - Unified tracks DataFrame from all experiments with enriched columns - Lineage reconstruction linking daughters to root ancestor via parent_track_id - Border clamping: retains border cells with shifted patch origins instead of exclusion - All 23 tests pass Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(21-01): fix lint issues and export MultiExperimentIndex - Remove unused variable (F841) in test_global_track_id_unique_across_experiments - Use .to_numpy() instead of .values (PD011) in test_exclude_fovs_filter - Export MultiExperimentIndex from dynaclr __init__.py Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(21-01): complete MultiExperimentIndex plan summary and state update - 21-01-SUMMARY.md with full execution documentation - STATE.md updated for 21-01 completion, decisions, session continuity Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(21-02): add failing tests for valid anchors, properties, and summary - 8 tests for valid_anchors: basic validity, subset check, end-of-track exclusion, lineage continuity, different tau ranges, empty tracks, gap handling, self-exclusion - 9 tests for properties/summary: experiment_groups, condition_groups, summary() - All 17 new tests fail with TypeError (tau_range_hours not yet accepted) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(21-02): implement valid_anchors, experiment_groups, condition_groups, summary - Add tau_range_hours parameter to MultiExperimentIndex.__init__ - _compute_valid_anchors: per-experiment tau conversion, lineage-based lookup - experiment_groups/condition_groups properties returning index arrays - summary() with experiment counts, observation counts, per-experiment breakdowns - All 40 tests pass (23 existing + 17 new) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(21-02): complete valid anchors plan - SUMMARY.md with self-check passed - STATE.md updated: Phase 21 complete, ready for Phase 22 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(22): research batch sampling phase domain * docs(22): create phase plan for batch sampling * test(22-01): add failing tests for FlexibleBatchSampler - Experiment-aware batching: single-experiment restriction, all experiments appear - Condition balancing: 2-condition and 3-condition proportional tests - Leaky mixing: zero leak, 20% leak injection, no-effect when not experiment-aware - Small group fallback: no crash, warning emission - Determinism: same seed/epoch reproduces, set_epoch changes sequence - Sampler protocol: yields list[int], correct __len__ - DDP partitioning: disjoint interleaved batches across ranks Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(22-01): implement FlexibleBatchSampler with experiment-aware, condition-balanced, leaky mixing - FlexibleBatchSampler(Sampler[list[int]]) with cascade batch construction - experiment_aware=True restricts each batch to a single experiment - condition_balanced=True balances condition representation per batch - leaky > 0.0 injects cross-experiment samples into restricted batches - Deterministic via np.random.default_rng(seed + epoch) - DDP support via interleaved batch partitioning across ranks - Small group fallback to replacement sampling with logged warning - Pre-computed group indices at __init__ for O(1) lookup - Fix lint issues in test file (import sorting, .values -> .to_numpy(), nunique -> len(unique)) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(22-01): export FlexibleBatchSampler from viscy_data package - Add FlexibleBatchSampler to viscy_data.__init__.py public API - Place import in alphabetically correct position for ruff isort compliance - Add to __all__ exports under Utilities section Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(22-01): complete FlexibleBatchSampler core plan - Create 22-01-SUMMARY.md with TDD execution results - Update STATE.md: plan 01/02 complete, decisions, session continuity Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(22-02): add failing tests for temporal enrichment, DDP coverage, validation - 6 temporal enrichment tests (focal concentration, global_fraction edge cases, validation) - 5 DDP disjoint coverage tests (interleaving, coverage, epoch reproducibility) - 3 validation guard tests (missing experiment/condition/hpi columns) - 2 package import tests (import, __all__) - All 9 new feature tests fail as expected (RED) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(22-02): implement temporal enrichment, validation guards, DDP coverage - Add temporal_enrichment, temporal_window_hours, temporal_global_fraction params - Implement _enrich_temporal: focal/global sampling from experiment pool - Add column validation guards for experiment/condition/hpi columns - Conditional precomputation: only groupby columns when feature enabled - Fix stale smoke test __all__ count (45 -> 46) from Plan 01 - All 35 sampler tests pass, 107 total viscy-data tests pass Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(22-02): complete temporal enrichment + DDP plan - 22-02-SUMMARY.md with all metrics, decisions, deviations - STATE.md advanced to Phase 23, progress 22/25 (88%) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(phase-22): complete batch sampling phase execution Phase 22 verified: FlexibleBatchSampler with all 5 SAMP requirements. 5/5 must-haves passed. 35 tests, 107 full suite pass. No regressions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(23): create phase plan for Loss & Augmentation * test(23-01): add failing tests for NTXentHCL - 12 test cases covering subclass, beta=0 equivalence, hard negatives, gradients, temperature effect, edge cases, defaults, and CUDA - All fail with ModuleNotFoundError (dynaclr.loss not yet created) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(23-02): add failing tests for ChannelDropout and variable tau sampling - 11 tests for ChannelDropout: zeros, probability bounds, eval mode, per-sample, dtype, input safety, multi-channel, CUDA - 7 tests for sample_tau: range, exponential decay, uniform, single value, determinism, return type * feat(23-02): implement ChannelDropout and variable tau sampling - ChannelDropout nn.Module: per-sample channel zeroing on (B,C,Z,Y,X) tensors - sample_tau: exponential decay weighted sampling for temporal offsets * feat(23-01): implement NTXentHCL with hard-negative concentration - NTXentHCL subclasses NTXentLoss from pytorch_metric_learning - beta=0.0 delegates to parent for exact numerical equivalence - beta>0 applies exp(beta*sim) reweighting on negatives in denominator - Normalized weights preserve loss magnitude across beta values - All 11 tests pass (1 CUDA test skipped on macOS) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(23-02): add ChannelDropout and sample_tau to package exports - Export ChannelDropout from viscy_data top-level - Export sample_tau from dynaclr top-level - Include NTXentHCL export added by linter * docs(23-02): complete ChannelDropout and tau sampling plan - Summary with TDD metrics, decisions, self-check - STATE.md updated for Phase 23 completion Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(23-01): complete NTXentHCL loss plan - Created 23-01-SUMMARY.md with TDD execution results - Updated STATE.md with HCL implementation decisions - Self-check passed: all artifacts and commits verified Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(phase-23): complete loss & augmentation phase execution Phase 23 verified: NTXentHCL (3/3 LOSS reqs), ChannelDropout (AUG-01), sample_tau (AUG-03). AUG-02 wiring deferred to Phase 24 by design. 30 tests pass across loss, channel_dropout, tau_sampling. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(24): create phase plan * test(24-01): add failing tests for MultiExperimentTripletDataset - 7 test cases covering __getitems__ return format, norm_meta, lineage-aware positive sampling, division event traversal, channel remapping, predict mode, and dataset length - All tests fail with ModuleNotFoundError (RED phase) * feat(24-01): implement MultiExperimentTripletDataset with lineage-aware sampling - __getitems__ returns batch dicts with anchor/positive Tensors (B,C,Z,Y,X) - Lineage-aware positive sampling via pre-built (experiment, lineage_id) lookup - Division events traversed naturally via shared lineage_id - Per-experiment channel remapping using registry.channel_maps - Tensorstore I/O with SLURM-aware context and per-FOV caching - Predict mode returns anchor + TrackingIndex dicts - Exponential decay tau sampling with fallback to full range scan * refactor(24-01): add MultiExperimentTripletDataset to package exports - Export from dynaclr.__init__ for public API access * docs(24-01): complete MultiExperimentTripletDataset plan - SUMMARY.md with TDD commits, decisions, self-check - STATE.md updated: position 24-01, decisions, session continuity * update uv * test(24-02): add failing tests for MultiExperimentDataModule - 6 test cases covering hyperparameter exposure, experiment-level split, FlexibleBatchSampler wiring, val dataloader, transforms, ChannelDropout - RED phase: all tests fail with ModuleNotFoundError Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(24-02): implement MultiExperimentDataModule with experiment-level split - MultiExperimentDataModule composes FlexibleBatchSampler + Dataset + ChannelDropout + ThreadDataLoader with collate_fn=lambda x: x - Train/val split by whole experiments via val_experiments parameter - All sampling, augmentation, and loss hyperparameters exposed as __init__ params - on_after_batch_transfer applies normalizations + augmentations + final crop + ChannelDropout with proper norm_meta handling for all-None case - 6 TDD tests passing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(24-02): add MultiExperimentDataModule to dynaclr package exports - Import MultiExperimentDataModule from dynaclr.datamodule - Add to __all__ for top-level importability Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(24-02): complete MultiExperimentDataModule plan - Summary with TDD commits, decisions, and deviation documentation - STATE.md updated: Phase 24 complete, 96% progress, ready for Phase 25 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(phase-24): complete dataset & datamodule phase execution Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(25): create phase plan * docs(phase-25): complete integration phase plan Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(25-01): add end-to-end multi-experiment integration tests - Create test_multi_experiment_fast_dev_run: 2 experiments with different channel sets (GFP vs RFP), fast_dev_run with NTXentHCL loss - Create test_multi_experiment_fast_dev_run_with_all_sampling_axes: experiment_aware + condition_balanced + temporal_enrichment enabled - Synthetic data helpers for multi-channel HCS OME-Zarr creation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(25-01): add multi-experiment YAML config and class_path validation test - Create multi_experiment_fit.yml with MultiExperimentDataModule, NTXentHCL loss, all sampling axes, generic channel names (ch_0/ch_1) - Add test_multi_experiment_config_class_paths_resolve validating all class_path entries in the config resolve to importable Python classes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(25-01): complete integration plan - milestone v2.2 complete - Add 25-01-SUMMARY.md documenting end-to-end integration validation - Update STATE.md: phase 25/25 complete, progress 100%, milestone v2.2 done Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(phase-25): complete integration phase execution — v2.2 milestone shipped Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * add the smoothness and dynamic range comparison * add the applications/qc * bug qc metrics exposing the device * add batch predict * adding cli for reduce dimensionality composable * add example configs for model comparision and smoothness * add the biological annotations to the zattrs * adding airtable logic * harmonize and remove duplication between airtable and qc. moving most things to airtable * cleanup readme for airtable * add callback to store embeddings every n epochs and store metadata to the anndata.uns * fix the apply-linear classifiers to make sure we use the model and version. * Exclude untracked applications/dynacell from uv workspace Local debris directory (hydra outputs, pycache) has no pyproject.toml and breaks uv lock when matched by applications/* glob. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(26): capture phase context * docs(state): record phase 26 context session * docs(26): create phase plans * feat(26-01): extract HCSPredictionWriter to viscy-utils callbacks - Create prediction_writer.py with HCSPredictionWriter, _pad_shape, _resize_image, _blend_in - Use TYPE_CHECKING guard for viscy_data imports (HCSDataModule, Sample) - Add numpy-style docstrings to all functions and class - Re-export HCSPredictionWriter from callbacks __init__.py - Fix pre-existing INDEX_COLUMNS -> ULTRACK_INDEX_COLUMNS in embedding_snapshot.py and embedding_writer.py - Add missing docstrings to embedding_snapshot.py public methods Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(26-01): extract MixedLoss to viscy-utils losses submodule - Create losses/ submodule with mixed_loss.py containing MixedLoss class - Uses ms_ssim_25d from viscy_utils.evaluation.metrics internally - Convert docstrings to numpy-style - Re-export MixedLoss from losses __init__.py Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(26-01): create translation application scaffold with workspace registration - Create applications/translation/ with src layout following dynaclr pattern - Add pyproject.toml with hatchling build, uv-dynamic-versioning, and workspace deps - Add README.md required by hatchling readme field - Add __main__.py delegating to viscy_utils.cli.main for LightningCLI entry point - Add example YAML configs (fit.yml, predict.yml) with HCSPredictionWriter callback - Create empty tests/__init__.py - Register viscy-translation in root pyproject.toml workspace sources - Update uv.lock with new workspace member Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(26-01): complete shared infra extraction + app scaffold plan - Create 26-01-SUMMARY.md with execution results - Update STATE.md with plan progress, decisions, session info - Update ROADMAP.md marking 26-01 as complete Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(26-02): migrate translation engine and evaluation modules - Copy engine.py with VSUNet, FcmaeUNet, AugmentedPredictionVSUNet, MaskedMSELoss - Copy evaluation.py with SegmentationMetrics2D - Update all imports to new package paths (viscy_data, viscy_models, viscy_utils) - Remove MixedLoss class from engine.py (now imported from viscy_utils.losses) - Update __init__.py with top-level re-exports Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(26-02): add translation engine test suite - Import tests for all public exports (VSUNet, FcmaeUNet, etc.) - VSUNet init and forward pass smoke tests with synthetic data - State dict key regression test for checkpoint compatibility - MixedLoss integration test (from viscy_utils.losses) - FcmaeUNet init test - No old import paths grep test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(26-02): complete engine migration plan - SUMMARY.md with 2 task commits, 2 auto-fixed deviations - STATE.md updated: phase 26 complete, 20/25 phases (80%) - ROADMAP.md updated with plan progress Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(26): complete UAT - 8 passed, 0 issues * add the pseudotime evals * re-structure pseudotime folder * add the linear classifier evals and restructure folder path * add evaluations to dynaclr package * cli and linear classifier init * fix(translation): address engine and evaluation bugs - Fix operator precedence bug in evaluation.py boolean condition - Fi…

edyoshikun and others added 30 commits February 13, 2026 11:05

docs(10-01): complete public API & CI integration plan (v1.1 mileston…

6657ee8

…e complete) - Add 10-01-SUMMARY.md with execution results - Update STATE.md: phase 10 complete, v1.1 milestone done, 100% progress Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

docs(phase-10): complete phase execution and verification (v1.1 miles…

2c067ab

…tone complete) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

remove the _layers

719c940

update the readme

b79c746

docs: start milestone v1.1 Extract viscy-data

aa54a91

update the main readme

ee63b38

docs: complete viscy-data project research

e1ea1cd

docs: define milestone v1.1 requirements

01a9488

docs: create milestone v1.1 roadmap (4 phases)

26db307

docs(06-package-scaffolding-and-foundation): create phase plan

29af85c

docs(06-01): complete package scaffolding plan with summary and state…

aef7948

… update - Add 06-01-SUMMARY.md documenting viscy-data package creation - Update STATE.md with plan position, metrics, and decisions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

docs(06-02): complete utility module extraction plan

0c4bd6e

- Add 06-02-SUMMARY.md documenting utility extraction - Update STATE.md: Phase 6 complete, progress 80% Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

docs(phase-6): complete phase execution

3f96d6e

docs(07-code-migration): create phase plan

9682fa6

docs(07-01): complete core data module migration plan

9d20020

- Add 07-01-SUMMARY.md documenting migration of 5 core modules - Update STATE.md: phase 7 plan 1 of 4, decisions, metrics Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

docs(07-03): complete optional dependency module migration plan

1a3589c

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

alxndrkalinin and others added 2 commits March 17, 2026 16:43

alxndrkalinin requested a review from edyoshikun March 18, 2026 00:22

edyoshikun marked this pull request as ready for review March 18, 2026 04:02

edyoshikun and others added 12 commits March 17, 2026 21:05

remove ed notes

6b1ece1

missing pyarrow in dynaclr install. dpeend on visc-data triplet optio…

2e356fd

…nal dependency

fix troubleshooting.md for MultiExperiment setup

f6ccd39

add wandndb as core dependency

740c179

remove the data.log

05a3153

docs(dynaclr): add tracking note for anndata ArrowString zarr bug

8914980

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

chore(dynaclr): minor cleanup in inspect_dataloader and test formatting

f2195e0

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

add package and applications folder rules for CLAUDE.md

f21feee

context managers rule

30546ef

edyoshikun mentioned this pull request Mar 18, 2026

Applications: DynaCLR refactor #370

Closed

edyoshikun added 2 commits March 18, 2026 16:50

remove __version.py

0f60167

delete gsd markdowns

bd71c22

edyoshikun approved these changes Mar 18, 2026

View reviewed changes

chore: remove stale applications/dynacell directory

6f802fa

Remove old Hydra output logs and cache files from the defunct dynacell application. This directory was already excluded from the uv workspace. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

alxndrkalinin merged commit 09043d1 into modular-viscy-staging Mar 19, 2026

edyoshikun mentioned this pull request Mar 19, 2026

Harmonize planning doc #371

Closed

This was referenced Mar 19, 2026

Add Predict Volume PR to applications/cytoland #374

Closed

Address deferred review cleanup on staging #386

Merged

This was referenced Apr 7, 2026

viscy-airtable package #358

Closed

Airtable metadata #382

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Applications: Cytoland refactor#379

Applications: Cytoland refactor#379
alxndrkalinin merged 292 commits into
modular-viscy-stagingfrom
app-cytoland

alxndrkalinin commented Feb 27, 2026 •

edited

Loading

Uh oh!

alxndrkalinin commented Mar 18, 2026 •

edited

Loading

Uh oh!

alxndrkalinin commented Mar 18, 2026

Uh oh!

edyoshikun left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

alxndrkalinin commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Translation application (Phase 26)

PR #280 port: sliding window volume prediction

app-dynaclr sync (PR #381 + #384)

Bug fixes

Test results

Uh oh!

alxndrkalinin commented Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deferred items from code review

Uh oh!

alxndrkalinin commented Mar 18, 2026

Uh oh!

edyoshikun left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

alxndrkalinin commented Feb 27, 2026 •

edited

Loading

alxndrkalinin commented Mar 18, 2026 •

edited

Loading