Skip to content

Zip store#512

Draft
will-moore wants to merge 2 commits intoome:masterfrom
will-moore:zip_store
Draft

Zip store#512
will-moore wants to merge 2 commits intoome:masterfrom
will-moore:zip_store

Conversation

@will-moore
Copy link
Copy Markdown
Member

NB: this is currently 1 commit on top of #476.

Experimenting with what it would take to support ZipStore, as requested in #363.
Also informed by #478 (comment)

Currently, write_multiscale() fails to write any metadata attributes to the top-level group when using a ZipStore, because write_multiscales_metadata() happens after the group is created and it is not updatable in a ZipStore.

The experimental approach taken here is to generate all the metadata before the group creation happens. This allows us to write the attributes when the group is created.

If the group name (string) ending with ".zip" then we create a zip store:

Limited testing with:

import numpy as np
from ome_zarr.writer import write_image

size_xy = 128
size_z = 10
rng = np.random.default_rng(0)
data = rng.poisson(lam=10, size=(size_z, size_xy, size_xy)).astype(np.uint8)

write_image(data, group="zip_test.zarr.zip", axes="zyx",
            storage_options={"chunks": (1, 64, 64), "shards": (5, size_xy, size_xy)})

@melonora
Copy link
Copy Markdown
Contributor

@will-moore please also see zarr-developers/zarr-python#3612. Though not as neat (would require first writing the data) after that it should create the zip without really having to worry about the metadata as it just gets copied to a zipstore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants