Add v3 consolidated metadata support to withConsolidated

`withConsolidated` previously only supported Zarr v2 `.zmetadata`. This
adds support for v3 consolidated metadata (inline
`consolidated_metadata` in the root `zarr.json`), which is implemented
in zarr-python but not yet accepted into the Zarr v3 spec.

A new `format` option controls which format(s) to try. It accepts a
single format string or an array to try in order. When omitted, auto-
detection uses the existing version counter heuristic from `open`.

```ts
await withConsolidated(store);                         // auto-detect
await withConsolidated(store, { format: "v2" });       // v2 only
await withConsolidated(store, { format: "v3" });       // v3 only
await withConsolidated(store, { format: ["v3", "v2"] }); // try v3, fall back to v2
```

Author: manzt

.changeset/add-v3-consolidated.md

@@ -0,0 +1,14 @@
+---
+"zarrita": minor
+---
+
+Add v3 consolidated metadata support to `withConsolidated`. The v3 format reads `consolidated_metadata` from the root `zarr.json`, matching zarr-python's implementation. Note that v3 consolidated metadata is not yet part of the official Zarr v3 spec and should be considered experimental.
+
+A new `format` option controls which format(s) to try, accepting a single string or an array for fallback ordering. When omitted, format is auto-detected using the store's version history.
+
+```ts
+await withConsolidated(store);                            // auto-detect
+await withConsolidated(store, { format: "v2" });          // v2 only
+await withConsolidated(store, { format: "v3" });          // v3 only
+await withConsolidated(store, { format: ["v3", "v2"] });  // try v3, fall back to v2
+```

docs/cookbook.md

@@ -88,17 +88,16 @@ const group = await zarr.create(store, {
 group; // zarr.Group
 ```
 
-## Open a Consolidated Store
+## Open a Consolidated Store <Badge type="tip" text="v2 & v3" />
 
-Zarr v2 allows
-[consolidating metadata](https://zarr.readthedocs.io/en/stable/tutorial.html#consolidating-metadata)
-for an entire hierarchy at the store root (under `/.zmetadata`). Metadata
-consolidation is particularly useful when interacting with remote stores, where
-each metadata fetch incurs a network request and hence, latency.
+Consolidated metadata stores the entire hierarchy's metadata in a single
+location, avoiding per-node network requests. This is particularly useful
+with remote stores where each metadata fetch incurs latency.
 
 The `withConsolidated` helper wraps an existing [store](/packages/storage),
 proxying metadata requests with the consolidated metadata, thereby minimizing
-network requests.
+network requests. It supports both Zarr v2 (`.zmetadata`) and v3
+(`consolidated_metadata` in root `zarr.json`).
 
 ```js{3}
 import * as zarr from "zarrita";
@@ -113,15 +112,29 @@ let foo = await zarr.open(root.resolve("foo"), { kind: "array" });
 ```
 
 The store returned from `withConsolidated` is **readonly** and adds
-`.contents()` list the known contents of the hierarchy:
+`.contents()` to list the known contents of the hierarchy:
 
 ```js
 store.contents(); // [{ path: "/", kind: "group" }, { path: "/foo", kind: "array" }, ...]
 ```
 
+By default, the format is auto-detected. You can explicitly specify the
+format with the `format` option, or provide an array to try formats in order:
+
+```js
+// v2 only
+let store = await zarr.withConsolidated(rawStore, { format: "v2" });
+
+// v3 only
+let store = await zarr.withConsolidated(rawStore, { format: "v3" });
+
+// try v3 first, fall back to v2
+let store = await zarr.withConsolidated(rawStore, { format: ["v3", "v2"] });
+```
+
 ::: tip
 
-The `withConsolidated` helper errors out if v2 consolidated metadata is absent.
+The `withConsolidated` helper errors out if consolidated metadata is absent.
 Use `tryWithConsolidated` for uncertain cases; it leverages consolidated
 metadata if available.
 
@@ -133,6 +146,15 @@ let store = await zarr.tryWithConsolidated(
 
 :::
 
+::: warning
+
+Zarr v3 consolidated metadata (`format: "v3"`) targets the experimental
+consolidated metadata implemented in zarr-python, which is not yet part of the
+official Zarr v3 specification. See [zarr-specs#309](https://github.com/zarr-developers/zarr-specs/pull/309)
+for the ongoing spec discussion.
+
+:::
+
 
 ## Batch and Cache Range Requests
 

fixtures/v3/data.zarr/consolidated/1d.chunked.i2/c/0

@@ -0,0 +1,33 @@
+{
+  "shape": [
+    4
+  ],
+  "data_type": "int16",
+  "chunk_grid": {
+    "name": "regular",
+    "configuration": {
+      "chunk_shape": [
+        2
+      ]
+    }
+  },
+  "chunk_key_encoding": {
+    "name": "default",
+    "configuration": {
+      "separator": "/"
+    }
+  },
+  "fill_value": 0,
+  "codecs": [
+    {
+      "name": "bytes",
+      "configuration": {
+        "endian": "little"
+      }
+    }
+  ],
+  "attributes": {},
+  "zarr_format": 3,
+  "node_type": "array",
+  "storage_transformers": []
+}

fixtures/v3/data.zarr/consolidated/1d.chunked.i2/c/1

@@ -0,0 +1,35 @@
+{
+  "shape": [
+    2,
+    2
+  ],
+  "data_type": "int16",
+  "chunk_grid": {
+    "name": "regular",
+    "configuration": {
+      "chunk_shape": [
+        2,
+        2
+      ]
+    }
+  },
+  "chunk_key_encoding": {
+    "name": "default",
+    "configuration": {
+      "separator": "/"
+    }
+  },
+  "fill_value": 0,
+  "codecs": [
+    {
+      "name": "bytes",
+      "configuration": {
+        "endian": "little"
+      }
+    }
+  ],
+  "attributes": {},
+  "zarr_format": 3,
+  "node_type": "array",
+  "storage_transformers": []
+}

fixtures/v3/data.zarr/consolidated/1d.chunked.i2/zarr.json

@@ -0,0 +1,33 @@
+{
+  "shape": [
+    4
+  ],
+  "data_type": "int16",
+  "chunk_grid": {
+    "name": "regular",
+    "configuration": {
+      "chunk_shape": [
+        4
+      ]
+    }
+  },
+  "chunk_key_encoding": {
+    "name": "default",
+    "configuration": {
+      "separator": "/"
+    }
+  },
+  "fill_value": 0,
+  "codecs": [
+    {
+      "name": "bytes",
+      "configuration": {
+        "endian": "little"
+      }
+    }
+  ],
+  "attributes": {},
+  "zarr_format": 3,
+  "node_type": "array",
+  "storage_transformers": []
+}

fixtures/v3/data.zarr/consolidated/2d.contiguous.i2/c/0/0

@@ -0,0 +1,8 @@
+{
+  "attributes": {
+    "description": "A nested group"
+  },
+  "zarr_format": 3,
+  "consolidated_metadata": null,
+  "node_type": "group"
+}