Skip to content

Use auto sharding from zarr #2148

@ilan-gold

Description

@ilan-gold

Please describe your wishes and possible alternatives to achieve the desired result.

https://zarr.readthedocs.io/en/stable/api/zarr/index.html#zarr.create_array has the shards kwargs whose type (should be linked) actually apparently includes the auto option. This is far simpler that using write_dispatched. We should

  1. Add a setting to https://anndata.readthedocs.io/en/stable/generated/anndata.settings.html for auto sharding
  2. Decide how this should interact with https://anndata.readthedocs.io/en/stable/generated/anndata.settings.html#anndata.settings.zarr_write_format since you actually can write zarr v2 sharded data (I think error out is the clear choice given possible read compatibility issues in other languages, to say the least)
  3. Update the zarr-v3 guide accordingly https://anndata.readthedocs.io/en/stable/tutorials/zarr-v3.html

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions