Skip to content

Commit 7f03182

Browse files
committed
Edit release notes and docs (#408)
1 parent 6b25b93 commit 7f03182

4 files changed

Lines changed: 184 additions & 212 deletions

File tree

.github/release-notes/v0.7.md

Lines changed: 57 additions & 61 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,20 @@
11
# v0.7.0
22

33
**zarrita** aims to be a delightful little library for working with Zarr in
4-
TypeScript. Since the v0.5 rewrite, real-world data and use cases have moved
5-
faster than the library, and our initial abstractions began forcing users to
6-
reach past the public API to get their work done
4+
TypeScript. Since the v0.5 rewrite, its abstractions had started forcing users
5+
past the public API to get real work done
76
([#349](https://github.com/manzt/zarrita.js/issues/349),
87
[#310](https://github.com/manzt/zarrita.js/issues/310),
98
[#325](https://github.com/manzt/zarrita.js/issues/325),
10-
[#352](https://github.com/manzt/zarrita.js/issues/352)). You filed issues. It
11-
took us some time to work out the right shape for the fixes. Thank you for your
12-
patience.
13-
14-
**v0.7 focuses on extensibility and correctness.** **Extensibility**, because
15-
most real use cases weren't about writing new stores (you still can, and
16-
`FetchStore` covers the common case). They were about *layering behavior*
17-
over an existing store: caching, request batching, auth, virtual-format
18-
translation. People were already doing that through subclassing and
19-
hand-rolled proxies. v0.7 makes layering simple and enjoyable.
20-
**Correctness**, because the other gaps were places where the library
21-
silently got the wrong answer on real data.
9+
[#352](https://github.com/manzt/zarrita.js/issues/352)). v0.7 addresses that
10+
on two fronts: **extensibility** and **correctness**.
11+
12+
Most real use cases weren't about writing new stores — they were about
13+
*layering behavior* over an existing store: caching, request batching, auth,
14+
virtual-format translation. People were doing that through subclassing and
15+
hand-rolled proxies. v0.7 makes layering a first-class operation. The other
16+
gaps were places where the library silently returned wrong answers on real
17+
data.
2218

2319
> ⚠️ **Heads up**: v0.7 has one hard break (`zarr.create` options are now
2420
> camelCase) and one deprecation (`FetchStore`'s `overrides` option, in
@@ -27,26 +23,25 @@ silently got the wrong answer on real data.
2723
2824
## Extensions
2925

30-
Before v0.7, zarrita had blessed extension points for custom codecs and custom
31-
stores (implementing `AsyncReadable` from scratch), but nothing for **layering
32-
behavior on an existing store**. Adding these kinds of features to a
33-
`FetchStore`, for example, meant subclassing it, wrapping it in a hand-rolled
34-
`Proxy`, or smuggling per-call state through the `AsyncReadable<Options>`
35-
generic. Intercepting at the *chunk* layer (instead of the byte layer) meant
36-
replacing `zarr.Array` with a bare `Proxy` entirely; there was no extension
37-
point there.
26+
Before v0.7, zarrita had extension points for custom codecs and custom stores
27+
(implementing `AsyncReadable` from scratch), but nothing for **layering
28+
behavior on an existing store**. Adding features to a `FetchStore` meant
29+
subclassing it, wrapping it in a hand-rolled `Proxy`, or smuggling per-call
30+
state through the `AsyncReadable<Options>` generic. Intercepting at the
31+
*chunk* layer meant replacing `zarr.Array` with a bare `Proxy` entirely;
32+
there was no extension point there at all.
3833

39-
v0.7 introduces two symmetric extension points, one per layer:
34+
v0.7 introduces two extension points, one per layer:
4035

41-
| Layer | Intercepts | Define | Compose |
36+
| Layer | Intercepts | Primitive | Composer |
4237
| --------- | ------------------------- | --------------------------- | ------------------ |
4338
| Transport | `store.get(key, range)` | `zarr.defineStoreExtension` | `zarr.extendStore` |
4439
| Data | `array.getChunk(coords)` | `zarr.defineArrayExtension` | `zarr.extendArray` |
4540

4641
Store extensions handle paths and bytes. Array extensions handle chunk
47-
coordinates. A factory receives the inner value and user options and returns an
48-
object of method overrides; everything else is delegated through a `Proxy`, so
49-
consumers never notice the wrapper. Compose extensions in a pipeline:
42+
coordinates. A factory receives the inner value and user options and returns
43+
method overrides; everything else is delegated through a `Proxy`. Compose
44+
extensions in a pipeline:
5045

5146
```ts
5247
let store = await zarr.extendStore(
@@ -59,22 +54,20 @@ let store = await zarr.extendStore(
5954
let arr = await zarr.open(store, { kind: "array" });
6055
```
6156

62-
Three store extensions ship in the box, built on the new primitive rather
63-
than bolted alongside it:
57+
Three store extensions ship built on the new primitive:
6458

6559
- **`zarr.withConsolidatedMetadata`**: short-circuits metadata reads from a
66-
pre-fetched consolidated blob. Now reads v3 `consolidated_metadata`
67-
from the root `zarr.json`, matching zarr-python. A `format` option
68-
picks v2, v3, or a fallback order; auto-detects by default. *v3
69-
consolidated metadata is not yet part of the official spec; treat
70-
it as experimental.*
71-
- **`zarr.withRangeCoalescing`**: microtask-tick range batcher. Concurrent
72-
`getRange` calls within a microtask are grouped by path, coalesced
73-
across a byte-gap threshold, and issued as a single request per group.
74-
Big win for many-small-chunk workloads.
60+
pre-fetched consolidated blob. Now reads v3 `consolidated_metadata` from
61+
the root `zarr.json`, matching zarr-python. A `format` option picks v2,
62+
v3, or a fallback order; auto-detects by default. *v3 consolidated
63+
metadata is not yet part of the official spec; treat it as experimental.*
64+
- **`zarr.withRangeCoalescing`**: groups concurrent `getRange` calls on the
65+
same path within a microtask, merges adjacent ranges under a byte-gap
66+
threshold, and issues one request per group. Cuts HTTP round-trips for
67+
many-small-chunk reads.
7568
- **`zarr.withByteCaching`**: byte cache over `get` and `getRange`, with an
76-
optional `keyFor` for cache-policy narrowing. The cache container is
77-
any object implementing `has`/`get`/`set`; a plain `Map` is the default.
69+
optional `keyFor` for narrowing the policy. The cache container is any
70+
object implementing `has`/`get`/`set`; a plain `Map` is the default.
7871

7972
### Virtual-format adapters
8073

@@ -129,21 +122,25 @@ cover every failure mode reachable from `zarr.open`, `zarr.get`, and
129122
| `InvalidSelectionError` | bad rank, out-of-bounds, zero step, dimension-name mismatch, scalar-shape mismatch | |
130123
| `UnsupportedError` | capability limit (sharded set, unimplemented codec encode paths, missing `DataView.prototype.getFloat16`) | |
131124

125+
`zarr.isZarritaError` is variadic and narrows to the matching union. No
126+
tags matches any zarrita error; one or more tags narrows to just those:
127+
132128
```ts
133129
try {
134130
await zarr.open(store, { kind: "array" });
135-
} catch (e) {
136-
if (zarr.isZarritaError(e, "NotFoundError")) {
137-
// e.path, e.found available
138-
} else if (zarr.isZarritaError(e, "UnknownCodecError")) {
139-
// e.codec is the unregistered codec name; register and retry
131+
} catch (err) {
132+
if (zarr.isZarritaError(err, "NotFoundError")) {
133+
err; // NotFoundError; err.path, err.found available
134+
} else if (zarr.isZarritaError(err, "UnknownCodecError", "CodecPipelineError")) {
135+
err; // UnknownCodecError | CodecPipelineError
136+
} else if (zarr.isZarritaError(err)) {
137+
err; // AnyZarritaError (catch-all for the hierarchy)
140138
}
141139
}
142140
```
143141

144-
Classes are exported, but the docs steer callers toward `zarr.isZarritaError`
145-
so `instanceof` checks aren't load-bearing. The class hierarchy can change
146-
later without breaking tag-based call sites.
142+
Prefer `zarr.isZarritaError` over `instanceof` so class-hierarchy changes
143+
stay non-breaking.
147144

148145
### Quantized data: `cast_value`, `scale_offset`, `fixedscaleoffset`
149146

@@ -161,12 +158,12 @@ though the bytes on disk are quantized.
161158

162159
### Fill values, scalars, and browser autodetect
163160

164-
`NaN`, `Infinity`, and `-Infinity` now round-trip correctly as fill values per
165-
the Zarr v3 spec (previously they silently serialized as `null`, and missing
166-
chunks filled with `0`). `get` and `set` work for scalar arrays (`shape=[]`).
167-
`zarr.open`'s version autodetection no longer fails in browsers when a server
168-
returns a non-JSON response for a v2 metadata key. Each was a
169-
silent-wrong-answer bug on real data. Not anymore.
161+
`NaN`, `Infinity`, and `-Infinity` now round-trip correctly as fill values
162+
per the Zarr v3 spec (previously they silently serialized as `null`, and
163+
missing chunks filled with `0`). `get` and `set` work for scalar arrays
164+
(`shape=[]`). `zarr.open`'s version autodetection no longer fails in
165+
browsers when a server returns a non-JSON response for a v2 metadata key.
166+
Each was a silent-wrong-answer bug on real data.
170167

171168
## Ergonomics
172169

@@ -206,9 +203,9 @@ const controller = new AbortController();
206203
await zarr.get(arr, null, { signal: controller.signal });
207204
```
208205

209-
Removing the `Options` generic on `AsyncReadable` (the same change that
210-
unblocked array extensions) let `signal` become a plain field on `GetOptions`
211-
instead of opaque per-call state threaded through the store.
206+
Dropping the `Options` generic on `AsyncReadable` (the same change that
207+
unblocked array extensions) let `signal` become a plain field on
208+
`GetOptions` instead of opaque per-call state.
212209

213210
### Named-dimension selection
214211

@@ -259,6 +256,5 @@ Thank you to everyone who contributed to v0.7:
259256
[@kylebarron](https://github.com/kylebarron),
260257
[@thewtex](https://github.com/thewtex),
261258
and [@Wietze](https://github.com/Wietze).
262-
A special thanks to the downstream maintainers (at vole-core, vizarr,
263-
and the growing collection of virtual-zarr adapters) whose real
264-
workloads shaped the extension API long before it existed as code.
259+
Thanks also to downstream maintainers at vole-core, vizarr, and the
260+
virtual-zarr adapters, whose workloads shaped the extension API.

docs/cookbook.md

Lines changed: 39 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -94,15 +94,15 @@ Consolidated metadata stores the entire hierarchy's metadata in a single
9494
location, avoiding per-node network requests. This is particularly useful
9595
with remote stores where each metadata fetch incurs latency.
9696

97-
The `withConsolidated` helper wraps an existing [store](/packages/storage),
98-
proxying metadata requests with the consolidated metadata, thereby minimizing
99-
network requests. It supports both Zarr v2 (`.zmetadata`) and v3
100-
(`consolidated_metadata` in root `zarr.json`).
97+
The `withConsolidatedMetadata` extension wraps an existing
98+
[store](/packages/storage), short-circuiting metadata requests from the
99+
consolidated blob instead of going to the network. It supports both Zarr
100+
v2 (`.zmetadata`) and v3 (`consolidated_metadata` in root `zarr.json`).
101101

102102
```js{3}
103103
import * as zarr from "zarrita";
104104
105-
let store = await zarr.withConsolidated(
105+
let store = await zarr.withConsolidatedMetadata(
106106
new zarr.FetchStore("https://localhost:8080/data.zarr")
107107
);
108108
@@ -111,7 +111,7 @@ let root = await zarr.open(store, { kind: "group" });
111111
let foo = await zarr.open(root.resolve("foo"), { kind: "array" });
112112
```
113113

114-
The store returned from `withConsolidated` is **readonly** and adds
114+
The store returned from `withConsolidatedMetadata` is **readonly** and adds
115115
`.contents()` to list the known contents of the hierarchy:
116116

117117
```js
@@ -123,23 +123,23 @@ format with the `format` option, or provide an array to try formats in order:
123123

124124
```js
125125
// v2 only
126-
let store = await zarr.withConsolidated(rawStore, { format: "v2" });
126+
let store = await zarr.withConsolidatedMetadata(rawStore, { format: "v2" });
127127

128128
// v3 only
129-
let store = await zarr.withConsolidated(rawStore, { format: "v3" });
129+
let store = await zarr.withConsolidatedMetadata(rawStore, { format: "v3" });
130130

131131
// try v3 first, fall back to v2
132-
let store = await zarr.withConsolidated(rawStore, { format: ["v3", "v2"] });
132+
let store = await zarr.withConsolidatedMetadata(rawStore, { format: ["v3", "v2"] });
133133
```
134134

135135
::: tip
136136

137-
The `withConsolidated` helper errors out if consolidated metadata is absent.
138-
Use `tryWithConsolidated` for uncertain cases; it leverages consolidated
139-
metadata if available.
137+
`withConsolidatedMetadata` throws if consolidated metadata is absent. Use
138+
`withMaybeConsolidatedMetadata` when you don't know up front; it uses
139+
consolidated metadata if available, and otherwise passes through.
140140

141141
```js
142-
let store = await zarr.tryWithConsolidated(
142+
let store = await zarr.withMaybeConsolidatedMetadata(
143143
new zarr.FetchStore("https://localhost:8080/data.zarr"),
144144
);
145145
```
@@ -156,48 +156,51 @@ for the ongoing spec discussion.
156156
:::
157157

158158

159-
## Batch and Cache Range Requests
159+
## Coalesce and Cache Range Requests
160160

161161
When reading chunked data over HTTP, many small range requests can be
162-
expensive due to per-request latency. The `withRangeBatching` helper wraps a
163-
store so that concurrent `getRange()` calls within the same microtask tick are
164-
automatically merged into fewer, larger fetches. Results are cached in an LRU
165-
cache (assumes immutable data).
162+
expensive due to per-request latency. zarrita ships two composable store
163+
extensions for this: `withRangeCoalescing` merges concurrent range reads
164+
into fewer fetches, and `withByteCaching` caches the results. Either works
165+
on its own; together they give you batched-and-cached reads.
166166

167-
```js{3-5}
167+
```js
168168
import * as zarr from "zarrita";
169169

170-
let store = zarr.withRangeBatching(
170+
let store = await zarr.extendStore(
171171
new zarr.FetchStore("https://localhost:8080/data.zarr"),
172+
(s) => zarr.withRangeCoalescing(s),
173+
(s) => zarr.withByteCaching(s),
172174
);
173175

174176
let arr = await zarr.open(store, { kind: "array" });
175177
```
176178

177-
Adjacent byte ranges separated by less than `coalesceSize` bytes (default
178-
32 KB) are coalesced into a single request. You can tune this along with the
179-
LRU cache capacity:
179+
`withRangeCoalescing` groups concurrent `getRange()` calls on the same path
180+
within a microtask tick, merges adjacent byte ranges separated by less than
181+
`coalesceSize` bytes (default 32 KB), and issues one fetch per group. When
182+
multiple callers each pass their own `AbortSignal` and their requests land
183+
in the same group, the signals are merged so the shared fetch aborts as
184+
soon as any caller aborts.
180185

181186
```js
182-
let store = zarr.withRangeBatching(
187+
let store = zarr.withRangeCoalescing(
183188
new zarr.FetchStore("https://localhost:8080/data.zarr"),
184189
{
185-
coalesceSize: 65536, // merge ranges within 64 KB of each other
186-
cacheSize: 512, // keep up to 512 entries in the LRU cache
190+
coalesceSize: 65_536, // merge ranges within 64 KB of each other
191+
onFlush(report) {
192+
// { path, groupCount, requestCount, bytesFetched }
193+
console.log(report);
194+
},
187195
},
188196
);
189197
```
190198

191-
When multiple callers each pass their own `AbortSignal` and their requests
192-
land in the same batch, the signals are merged via `AbortSignal.any`: the
193-
shared request aborts as soon as any one caller aborts.
194-
195-
You can inspect batching statistics via `store.stats`:
196-
197-
```js
198-
store.stats;
199-
// { hits: 12, misses: 4, batchedRequests: 16, mergedRequests: 4 }
200-
```
199+
`withByteCaching` is policy-agnostic: by default it caches every `get()`
200+
and `getRange()` response in an unbounded `Map`, but you can pass any
201+
`ByteCache`-compatible container (for example, an LRU) or a `keyFor`
202+
function to narrow the policy. See the
203+
[store extensions reference](./store-extensions.md) for the full API.
201204

202205

203206
## Read Data with SharedArrayBuffer <Badge type="tip" text="v2 & v3" />

0 commit comments

Comments
 (0)