zarr ZipStore does not support append and so cannot be used during acquisition (in v2 or v3). However, use of DirectoryStore on sensors produces prohibitively large numbers of files; this can then only be addressed with manual zipping after acquisition, which produces files that can be read with ZipStore. A different store would need to be identified to streamline that process.
Version 2 of zarr supports an SQLiteStore backend that might be useful for this, but not currently version 3 [1]. Zarr v3 is packaged with fewer store backends than z2, but a proof of concept has been written for in version 3 [2]. We could consider using that store, or investigating other backends, which would also require a custom backend.
References:
[1] zarr-developers/zarr-python#3319
[2] https://github.com/auxym/zarr-sqlite
zarr
ZipStoredoes not support append and so cannot be used during acquisition (in v2 or v3). However, use ofDirectoryStoreon sensors produces prohibitively large numbers of files; this can then only be addressed with manual zipping after acquisition, which produces files that can be read withZipStore. A different store would need to be identified to streamline that process.Version 2 of zarr supports an
SQLiteStorebackend that might be useful for this, but not currently version 3 [1]. Zarr v3 is packaged with fewer store backends than z2, but a proof of concept has been written for in version 3 [2]. We could consider using that store, or investigating other backends, which would also require a custom backend.References:
[1] zarr-developers/zarr-python#3319
[2] https://github.com/auxym/zarr-sqlite