Thoughts on using OME-Zarr v0.5 / Zarr v3?

[lubianat]

Hi!

I am seeing that Zarr datasets (perhaps all) on the portal use .zattrs and "version": "0.4" + Zarr v2.

The newer Zarr v3 specification (https://zarr-specs.readthedocs.io/en/latest/v3/core/) has some new things, including differences in how metadata is represented (e.g. zarr.json files).

Current version of OME-Zarr (https://ngff.openmicroscopy.org/0.5/) is tied to the v3 standard.

I am curious if there are discussions on updating the CryoET Data Portal pipelines so data is available as Zarr v3

Cheers!

[uermel]

Hi @lubianat, thanks for your feedback!

Your observation is correct: currently all data on the cryoET data portal is stored in OME-Zarr v0.4 and Zarr v2. We are of course aware of the update to the OME-Zarr spec and the release of Zarr v3 earlier this year. Discussions on updating to Zarr v3/OME-Zarr v0.5 are in progress, but regrettably I can't give you an exact timeline for that migration at the moment. Should this happen, I will give an update in this discussion post.

In the meantime it would be helpful for us to know if there are specific benefits to your work if the migration were to happen. Is this causing compatibility issues with tooling you are currently using?

[lubianat]

Thank you for the quick reply! Good that discussions are happening.

No incompatibility found here, I am starting to work with Josh and others at German BioImaging and generally interested in finding resources using OME-Zarr (plus taking a look at the versions around).

I am particularly interested in the use of RO-Crate in OME-Zarr, though. It is still not part of the spec, but it has been used in the OME 2024 NGFF Challenge (see e.g. the metadata retrieved on https://ome.github.io/ome-ngff-validator/?source=https://uk1s3.embassy.ebi.ac.uk/idr/share/ome2024-ngff-challenge/idr0004/P150.ome.zarr). It may eventually become part of the spec (some discussions for OME-Zarr 0.7, but not yet concrete).

The metadata for CryoET is super well organized, it could be a good fit for RO-Crate compliance somewhere in the future.

Anyways, thank you again!