OME-Zarr v0.5 (Zarr v3) support #301
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* wip * wip: 2025-02-27 * Undo reformatting * Remove unused imports * Fix miscellaneous failing tests. * Get rename_well working. * sort import * Read MMStack with Zarr v3 (#293) * use fsspec as a wrapper for ZarrTiffStore * update deprecated readers and tests * document the helper function * add back the binary check after a flag * use kwarg for file mode in reader * restore dtype attribute * restore opening zarr with dask * stringify path name * remove redundant dtype * add rich for zarr v3 print tree * skip tests that require ome-zarr-py * fix file list checking it doesn't matter what the order is * fix commas * stringify path names * `test_rename_well` no longer broken, remove mark * from_positions is actually broken this is not related to ome-zarr-py * write to stdout pallets/click#2874 * remove unused import * non-zero return code with no args pallets/click#1489 --------- Co-authored-by: Ziwen Liu <[email protected]>
ziw-liu
added
enhancement
* wip * wip: 2025-02-27 * Undo reformatting * Remove unused imports * Fix miscellaneous failing tests. * Get rename_well working. * sort import * Read MMStack with Zarr v3 (#293) * use fsspec as a wrapper for ZarrTiffStore * update deprecated readers and tests * document the helper function * add back the binary check after a flag * use kwarg for file mode in reader * restore dtype attribute * restore opening zarr with dask * stringify path name * remove redundant dtype * add rich for zarr v3 print tree * skip tests that require ome-zarr-py * fix file list checking it doesn't matter what the order is * fix commas * stringify path names * detect ome key in reader * allow multiscales to not have version * wip: allow reading v0.5 images * normalize store path * update docstring * remove unused import * remove zarr-python synchronizer zarr-developers/zarr-python#1596 * allow empty zarr examples * add example json files * fixture for example v0.5 HCS dataset * unify ome container unwrapping * run PR tests for all target branches * test with absolute paths zarr-python 3.0.8 breakage * avoid big endian test cases * avoid yanked versions * fix zarr-python breakage * revert keyword argument name * fix version detection after zarr-python breakage * test with absolute path * Revert "allow empty zarr examples" This reverts commit 77146dd. * Test reading NGFF v0.5 with acquire-zarr * check binary values * simplify file mode switch Co-authored-by: Alan Liddell <[email protected]> * remove redundant deletion --------- Co-authored-by: Alan Liddell <[email protected]> Co-authored-by: Alan Liddell <[email protected]>
Merged
* fix member type hint * fix shape type hints * update imports * fix typo * stop allowing bytes as paths * fix index type hints * remove broken support for 0.1 * multiscales is required * fix shape type hints * fix version type hints * separate metadata dumping for positions * add version testing * fix compressor options for v3 * fix create zeros for zarr v3 * add more tests * fix test fixture * fix default value type * add overloads for open_ome_zarr * add return type hint for wells * centralize serialization logic of ome keys * always create zeros first * more version tests * fuzz test sharding * cast arguments to str * log validation error * fix version dumping * dump version for images metadata * test round trip with ngff-zarr * replace default identity transform with concrete scales * fix scale length * replace exact shard size with ratios * update test case * ensure dimension names * fix dimension separator for v2 * fix plate versioning * add example to convert to new version * fix format * make acquire zarr testing optional since it cannot be installed on Bruno * skip fixture with test * fix tensorstore for zarr3 * test acquire-zarr in CI * document aqz incompatibility with Bruno * fix aquire-zarr version
2 tasks
Update acquire-zarr OME v0.5 fixture / aqz test to reflect new config API and downsampling behavior Co-authored-by: Alan Liddell <[email protected]>
* wip * wip: 2025-02-27 * Undo reformatting * Remove unused imports * Fix miscellaneous failing tests. * Get rename_well working. * sort import * Read MMStack with Zarr v3 (#293) * use fsspec as a wrapper for ZarrTiffStore * update deprecated readers and tests * document the helper function * add back the binary check after a flag * use kwarg for file mode in reader * restore dtype attribute * restore opening zarr with dask * stringify path name * remove redundant dtype * add rich for zarr v3 print tree * skip tests that require ome-zarr-py * fix file list checking it doesn't matter what the order is * fix commas * stringify path names * `test_rename_well` no longer broken, remove mark * from_positions is actually broken this is not related to ome-zarr-py * enable tests against ome-zarr-py * from_positions is actually broken this is not related to ome-zarr-py * replace almost equal assertion in the ngff-zarr test * unskip ome-zarr-py tests * consolidate import paths * bump test requirement --------- Co-authored-by: Alan Liddell <[email protected]> Co-authored-by: Alan Liddell <[email protected]>
* update signature type hints * clarify the type of position keys * expose sharding in plate creation * isort * fix default version * fix example code block format * fix type hints * utility to split indices by shards * separate apply and save * simplify random number generation * wip: batched writing in time * match storage keys with values * removed unused argument * fix import * fix string formatting * add more version testing * use tensorstore instead * control tensorstore concurrency * memory management * isort * format * remove platform check * warning if shards is specified for 0.4 * set shards to none for 0.4 * Update acquire-zarr OME v0.5 fixture / aqz test to reflect new config API and downsampling behavior * add notes about upstream issues * add example of sharded plate * explicitly add layout in open_ome_zarr * Fix tensorstore empty array handling (#326) * Fix tensorstore empty array handling - Add validation for empty arrays in _save_transformed before tensorstore write - Skip write operations for empty arrays with warning messages - Add comprehensive error handling with detailed diagnostics for tensorstore failures - Improve error messages to include array shapes, sizes, and tensorstore details This resolves the ValueError: Error aligning dimensions issue when empty arrays are passed to tensorstore write operations. * Add empty results check to prevent tensorstore alignment errors Adds validation in apply_transform_to_tczyx_and_save() to check for empty results dictionary before calling _save_transformed(). When no valid time points are available, logs diagnostic message and skips write operation instead of attempting to write empty arrays to tensorstore, which causes alignment dimension mismatches. * Revert "Fix tensorstore empty array handling" This reverts commit 65c9ddb. * better handling of output_time_indices * style * bugfix and better type hints * better messaging * style * raise error if attempting sharding along channel dimension --------- Co-authored-by: Alan Liddell <[email protected]> Co-authored-by: Ivan Ivanov <[email protected]>
srivarra
reviewed
Sep 8, 2025
Collaborator
srivarra
left a comment
srivarra
left a comment
There was a problem hiding this comment.
LGTM to me, just a couple of minor suggestions, make changes to them if you see fit.
mattersoflight
approved these changes
Sep 9, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #257.
This is a staging branch. Please target PRs here and avoid directly pushing commits.
We will temporarily enable merge commits on main when merging this.