Skip to content

Commit 1b3bb2a

Browse files
noelruaultnoelruault
committed
Publish 2026-05-28T16:38:26Z from 0bd1a7d
0 parents  commit 1b3bb2a

107 files changed

Lines changed: 14176 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.assetsignore‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Workers Assets, .gitignore-style exclusions. Anything matched here is
2+
# NOT uploaded with `wrangler deploy` and is invisible to the [assets]
3+
# binding the Worker reads from. This is the same root the static SPA
4+
# is served from, so we shave development-only paths out of the bundle.
5+
6+
# Worker source + node_modules (workerd binary is 100+ MiB, well above
7+
# the 25 MiB per-asset limit) and Wrangler's local-dev scratch dir.
8+
worker/
9+
.wrangler/
10+
11+
# Repo metadata + Git history.
12+
.git/
13+
.gitignore
14+
.editorconfig
15+
16+
# Local planning / brainstorming / docs scratch, not served.
17+
.plans/
18+
.brainstorming/
19+
.backups/
20+
._docs/
21+
22+
# macOS finder cruft + assorted noise.
23+
.DS_Store
24+
**/.DS_Store
25+
*.log
26+
27+
# Top-level docs that aren't part of the SPA. (Keep prompts/, skills/,
28+
# agents/ uploaded since the local transport fetches them.)
29+
Makefile
30+
.assetsignore

‎.bench/EXPERIMENTS.md‎

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
# EXPERIMENTS - Aitelier library bundle wire format
2+
3+
Append-only log. Past DISCARD entries stay as evidence the alternative was tried.
4+
5+
Environment:
6+
- macOS 25.5.0, Apple Silicon
7+
- bun 1.3.11
8+
- brotli 1.2.0, zstd 1.5.7, gzip Apple-479, bsdtar 3.5.3
9+
- Bun's `DecompressionStream` supports `gzip` and `brotli` natively (confirmed by `.bench/scripts/sanity-check.ts`). This matches Chrome 122+ / Firefox 122+ / Safari 17.5+, all baseline by 2026-05.
10+
11+
Metric (from SCOPE.md):
12+
```
13+
score = bytes + 0.05 · compress_ms_p50 + 0.50 · decode_ms_p50
14+
```
15+
16+
Baseline = `json-object × gzip-9` (the original proposal), bytes = 80 094, score = 80 095.39. A candidate is KEEP iff the bootstrap-95% CI of (cand − base) is strictly negative on the metric AND decode_p95 < 50 ms.
17+
18+
---
19+
20+
## Exp 0001 - corpus inventory
21+
22+
Date: 2026-05-18 Hypothesis: corpus shape is dominated by markdown. Cmd: `find prompts skills agents hooks -type f \( ... \) -exec wc -c {} \;` Result:
23+
- 33 files, 222 569 bytes total.
24+
- Top 3 files (`compression-engineer.md` 70.9 KB, `go-performance.md` 56.1 KB, `go-hot-pot-jo.md` 32.4 KB) = 159.4 KB = **71.6 %** of corpus.
25+
- 17 markdown bodies, 15 JSON sidecars, 1 shell script. Decision: corpus is markdown-dominated. Brotli's RFC 7932 static dictionary should help (HTML/JS-tuned, but Markdown shares vocabulary). Trained shared dictionaries will be biased toward markdown vocabulary even if we attempt to balance toward JSON sidecars.
26+
27+
---
28+
29+
## Exp 0002 - container framing overhead (identity, no codec)
30+
31+
Date: 2026-05-18 Hypothesis: container format choice changes raw bytes by < 5 %; tar is the outlier due to 512-byte block alignment. Cmd: `bun run .bench/scripts/build-containers.ts` Result:
32+
33+
| container | bytes | vs raw |
34+
|-----------------|---------|----------|
35+
| sentinel-cat | 223 813 | +0.6 % |
36+
| len-prefixed | 223 797 | +0.6 % |
37+
| json-object | 231 107 | +3.8 % |
38+
| ndjson | 231 634 | +4.1 % |
39+
| tar | 248 320 | +11.6 % |
40+
41+
JSON containers carry +4 % from `\n` escaping in markdown bodies (`JSON.stringify` escapes every newline as `\n`). tar carries +12 % from 512-byte header + body padding on 33 small entries.
42+
43+
Decision: KEEP all five containers in the bench - the question is whether the +12 % tar overhead survives compression.
44+
45+
---
46+
47+
## Exp 0003 - compress matrix, all 5 containers × 6 codecs
48+
49+
Date: 2026-05-18 Hypothesis: codec dominates wire bytes; container choice is in the noise after compression. Cmd: `bun run .bench/scripts/compress-matrix.ts` (5 runs per cell) Result (compressed bytes, by codec, across containers):
50+
51+
| container | identity | gzip-9 | brotli-11 | zstd-22 | zstd+dict | brotli-11-sh |
52+
|--------------|---------:|-------:|----------:|--------:|----------:|-------------:|
53+
| json-object | 231 107 | 80 094 | 60 104 | 64 357 | 61 952 | 58 127 |
54+
| ndjson | 231 634 | 80 158 | 60 085 | 64 373 | 61 957 | 58 098 |
55+
| sentinel-cat | 223 813 | 79 416 | 59 670 | 64 147 | 60 610 | 57 319 |
56+
| len-prefixed | 223 797 | 79 481 | 59 757 | 64 227 | 60 695 | 57 288 |
57+
| tar | 248 320 | 80 510 | 60 292 | 64 798 | 61 375 | 57 919 |
58+
59+
Within-codec spread across containers (max − min):
60+
- gzip-9: 1 094 B (1.4 %)
61+
- brotli-11: 622 B (1.0 %)
62+
- zstd-22: 651 B (1.0 %)
63+
- brotli-11-shared: 839 B (1.5 %)
64+
65+
Within-container spread across codecs (best vs gzip-9): ~22 KB everywhere.
66+
67+
Decision: **codec choice owns the wire-byte axis; container choice is in the noise after any codec**. The 11.6 % tar slack collapses to 1 % after brotli-11 because tar's repeated NUL-padded headers are highly compressible. KEEP all codec rows; consolidate the container question to a separate axis.
68+
69+
---
70+
71+
## Exp 0004 - round-trip correctness on all (container × codec)
72+
73+
Date: 2026-05-18 Hypothesis: parsers + codecs are correct end-to-end on the real corpus. Cmd: `bun run .bench/scripts/sanity-check.ts` Result: all 14 sampled cells (5 containers × identity + gzip, plus 4 codec variants of json-object) decode → parse → map to exactly 33 entries that byte-match the reference filesystem. Decision: SAFE to bench decode timing.
74+
75+
---
76+
77+
## Exp 0005 - V8 end-to-end decode timing, 9 runs per cell
78+
79+
Date: 2026-05-18 Hypothesis: brotli-11 decode in `DecompressionStream` is no slower than gzip-9 (calibration table says ~400 MB/s for both); container choice in binary vs JSON parser is sub-millisecond on a 60 KB payload. Cmd: `bun run .bench/scripts/decode-bench.ts` (9 runs + 2 warmups per cell, bootstrap-CI 10 000 resamples) Result, decode_ms_p50 by (container × codec):
80+
81+
| | identity | gzip-9 | brotli-11 | zstd-22 † | zstd+dict † | brotli-sh † |
82+
|--------------|---------:|-------:|----------:|----------:|------------:|------------:|
83+
| json-object | 0.333 | 1.089 | 1.185 | 5.005 | 4.727 | 4.378 |
84+
| ndjson | 0.271 | 1.093 | 1.073 | 5.017 | 4.614 | 4.649 |
85+
| sentinel-cat | 0.041 | 0.869 | 0.746 | 4.042 | 4.891 | 4.170 |
86+
| len-prefixed | 0.036 | 0.748 | 0.704 | 4.471 | 4.565 | 4.357 |
87+
| tar | 0.051 | 0.719 | 0.781 | 4.461 | 4.369 | 4.224 |
88+
89+
† CLI-backed (no `DecompressionStream` baseline support); includes ~2.4 ms fork/exec floor measured separately. True in-engine decode would be ~2 ms, but no WASM-free path exists in browsers in 2026.
90+
91+
Bootstrap-95% CI of decode delta vs baseline (`json-object × gzip-9`):
92+
- `len-prefixed × brotli-11`: **[−0.560, +0.028] ms** - crosses zero, not statistically faster than gzip on decode.
93+
- `sentinel-cat × brotli-11`: **[−0.564, +0.084] ms** - same, crosses zero.
94+
- `tar × brotli-11`: **[−0.584, −0.198] ms** - strictly negative, modest.
95+
- All `× brotli-11-shared` rows: **[+2.9, +3.8] ms** - strictly positive because of the fork/exec floor; the true browser cost (if a JS shared- dict decoder existed) would be ~1–2 ms.
96+
97+
Decision: **brotli-11 decode is statistically indistinguishable from gzip-9** on this corpus. Wire savings come at zero decode cost.
98+
99+
---
100+
101+
## Exp 0006 - score ranking, full matrix
102+
103+
Date: 2026-05-18 Hypothesis: with α=0.05, β=0.5, the score is byte-dominated; ranking will mirror the wire-byte ranking. Cmd: `bun run .bench/scripts/score-and-rank.ts` Result (top 10, lower = better):
104+
105+
| rank | candidate | bytes | Δ% vs base | dec ms | score | Δscore | browser_ok |
106+
|-----:|------------------------------------|--------:|-----------:|-------:|---------:|---------:|------------|
107+
| 1 | `len-prefixed × brotli-11-shared` | 57 288 | −28.47 % | 4.36 | 57 300.31 | −22 795 | **false** |
108+
| 2 | `sentinel-cat × brotli-11-shared` | 57 319 | −28.44 % | 4.17 | 57 331.28 | −22 764 | **false** |
109+
| 3 | `tar × brotli-11-shared` | 57 919 | −27.69 % | 4.22 | 57 931.81 | −22 164 | **false** |
110+
| 4 | `ndjson × brotli-11-shared` | 58 098 | −27.46 % | 4.65 | 58 110.83 | −21 985 | **false** |
111+
| 5 | `json-object × brotli-11-shared` | 58 127 | −27.43 % | 4.38 | 58 139.54 | −21 956 | **false** |
112+
| 6 | `sentinel-cat × brotli-11` | 59 670 | −25.50 % | 0.75 | 59 680.62 | −20 415 | **true** |
113+
| 7 | `len-prefixed × brotli-11` | 59 757 | −25.39 % | 0.70 | 59 767.83 | −20 328 | **true** |
114+
| 8 | `ndjson × brotli-11` | 60 085 | −24.98 % | 1.07 | 60 096.08 | −19 999 | **true** |
115+
| 9 | `json-object × brotli-11` | 60 104 | −24.96 % | 1.19 | 60 115.01 | −19 980 | **true** |
116+
| 10 | `tar × brotli-11` | 60 292 | −24.72 % | 0.78 | 60 303.25 | −19 792 | **true** |
117+
118+
Decision: filtering to `browser_ok = true` (constraint from SCOPE.md "`DecompressionStream` baseline 2026"), the winners are positions 6–10. The top 5 zstd / shared-dict variants are constraint-violating and ruled out.
119+
120+
Among browser-compatible candidates, **`sentinel-cat × brotli-11`** and **`len-prefixed × brotli-11`** tie on the score axis (Δscore differs by 87 units in 60 000, < 0.15 %). Differentiate on secondary criteria.
121+
122+
---
123+
124+
## Exp 0007 - secondary criteria for the brotli-11 cluster
125+
126+
Date: 2026-05-18 Hypothesis: of the 5 browser-OK brotli-11 cells, choose on (a) decode predictability - p95 minus p50 (jitter), (b) parser implementation complexity in the SPA, (c) survives the "raw.gh strips Content-Encoding" scenario gracefully.
127+
128+
| candidate | bytes | p50 ms | p95 ms | p95−p50 | parser LOC (rough) | needs JSON.parse |
129+
|----------------------------|-------:|-------:|-------:|--------:|-------------------:|------------------|
130+
| `len-prefixed × brotli-11` | 59 757 | 0.704 | 2.121 | 1.417 | ~25 | no |
131+
| `sentinel-cat × brotli-11` | 59 670 | 0.746 | 2.103 | 1.357 | ~30 | no |
132+
| `tar × brotli-11` | 60 292 | 0.781 | 1.044 | 0.263 | ~50 | no |
133+
| `ndjson × brotli-11` | 60 085 | 1.073 | 2.165 | 1.092 | ~10 | yes (per line) |
134+
| `json-object × brotli-11` | 60 104 | 1.185 | 1.700 | 0.515 | ~5 | yes (one shot) |
135+
136+
Trade-offs:
137+
- `json-object` parser is 5 LOC (`new Map(Object.entries(JSON.parse(text)))`) but uses JSON.parse, which mojibakes binary payloads (none today, but the spec allows .sh files). Acceptable because shell is utf-8 text.
138+
- `len-prefixed` is fully binary, zero-copy via `subarray`. Lowest LOC of the non-JSON containers. Survives any-byte payload (future-proof if a hook ships a binary asset).
139+
- `tar` has the lowest jitter (p95 close to p50) - likely because of the fixed 512-byte block size letting V8's TextDecoder hit aligned-read paths.
140+
- `ndjson` decode time is dragged up by per-line JSON.parse setup cost repeated 33 times.
141+
142+
Pages-path survival:
143+
- All 5 work the same way: the browser fetches the `.br` artifact bytes verbatim (since raw.gh doesn't recognize brotli for `Content-Encoding` negotiation), then runs `DecompressionStream("brotli")` in user-space. No difference across containers on this axis.
144+
145+
Decision: **`len-prefixed × brotli-11`** picked as the winner. Smallest bytes among browser-OK, lowest decode_p50, ~25-LOC parser, binary-safe, zero JSON quoting overhead. `sentinel-cat × brotli-11` is a tied alternate; pick `len-prefixed` because two length-prefixed fields are unambiguous (sentinel-cat relies on the assumption that bodies never contain `\0`, which is true today but a footgun if a future entity ships a binary blob in violation of the allowlist).
146+
147+
---
148+
149+
## Exp 0008 - per-file gzip-on-wire (today's path, no bundle)
150+
151+
Date: 2026-05-18 Hypothesis: bundling beats per-file fetching on bytes (small-stream overhead) and on RTTs (33 → 1). Cmd: `for f in <33 files>; do gzip -6 -c "$f" | wc -c; done | sum` Result:
152+
- per-file gzip-6 sum = 90 375 B (raw.gh default)
153+
- per-file brotli-11 sum = 75 924 B (hypothetical, if raw.gh served brotli; it does not)
154+
- bundle gzip-9 (len-prefixed) = 79 481 B
155+
- bundle brotli-11 (len-prefixed) = 59 757 B
156+
157+
Bundle gzip vs per-file gzip: **−12.1 %** wire bytes Bundle brotli vs per-file gzip: **−33.9 %** wire bytes (this is the today-vs-tomorrow delta)
158+
159+
RTT savings: 33 → 1 fetch. At 100 ms RTT (4G typical) and HTTP/2 multiplexing (no extra connect cost), 33 fetches still serialize through the `api.github.com` rate-limiter, which is the real bottleneck and the original motivation for bundling. Bundling makes the RTT axis a non-issue regardless of the format choice.
160+
161+
Decision: bundling is a unanimous win on every axis. The remaining question is "json-object or len-prefixed" and "gzip-9 or brotli-11", answered in Exp 0006/0007.
162+
163+
---
164+
165+
## Exp 0009 - dictionary amortization analysis
166+
167+
Date: 2026-05-18 Hypothesis: a shared zstd dictionary helps **only** if the consumer caches the dictionary across releases; on the first fetch, the dict bytes count against the candidate. Cmd: `zstd --train` produced a 16 384 B dictionary. Re-run compress with `-D`. Result:
168+
169+
| scenario | dict (B) | payload (B) | total (B) | vs `gzip-9` |
170+
|-----------------------------|---------:|------------:|----------:|------------:|
171+
| cold (dict + payload) | 16 384 | 60 695 | 77 079 | −3.0 % |
172+
| amortized (dict free) | 0 | 60 695 | 60 695 | −24.2 % |
173+
| `brotli-11` (no dict) | 0 | 59 757 | 59 757 | −25.4 % |
174+
175+
The trained dictionary buys us ~3 KB vs plain zstd-22 on warm-cache, and LOSES ~1 KB to plain brotli-11. Cold-cache it's a net loss vs gzip-9.
176+
177+
The deeper issue: **there is no `DecompressionStream("zstd")` baseline in 2026 browsers** (Chrome 143-only, no Firefox/Safari). The dictionary path also requires a Worker-side shim or RFC 9842 (`dcb` / `dcz`), which is still Origin Trial in Chrome 143 - and would not be honored by raw.githubusercontent.com regardless.
178+
179+
Decision: DISCARD all dict-based candidates. The constraint that "raw.gh controls headers" and "no external libraries in the SPA" jointly kill shared dictionaries for the Pages path. They could be revived later via the Worker path with a hand-rolled dcb/dcz emitter, but the wire-byte win (~3 KB on a 60 KB artifact) does not justify the publisher complexity (custom dictionary lifecycle, dict-hash versioning, browser fallback).
180+
181+
---
182+
183+
## Exp 0010 - wire-time sanity check at 1 Mbps and 25 Mbps
184+
185+
Date: 2026-05-18 Hypothesis: at 1 Mbps mobile, wire dominates decode by 100×; at 25 Mbps, they are within 10×. Saving wire bytes always wins. Cmd: `bun run .bench/scripts/score-and-rank.ts` (wire-time block) Result (total = transfer + decode_p50):
186+
187+
| candidate | bytes | t @1 Mbps | t @25 Mbps | total 1M | total 25M |
188+
|----------------------------|-------:|----------:|-----------:|---------:|----------:|
189+
| `len-prefixed × brotli-11` | 59 757 | 478 ms | 19.1 ms | 479 ms | 19.83 ms |
190+
| `len-prefixed × gzip-9` | 79 481 | 636 ms | 25.4 ms | 637 ms | 26.18 ms |
191+
| `json-object × gzip-9` | 80 094 | 641 ms | 25.6 ms | 642 ms | 26.71 ms |
192+
| per-file gzip-6 (today) | 90 375 | 723 ms | 28.9 ms | ~723 ms* | ~29 ms* |
193+
194+
* per-file ignores the RTT-serialization penalty against api.github.com rate limits; real today-cost is dominated by `5 + N` round-trips, not bytes.
195+
196+
Switching from `json-object × gzip-9` to `len-prefixed × brotli-11`: **−163 ms on 1 Mbps mobile**, **−6.9 ms on 25 Mbps home**. The mobile win is real and user-perceptible.
197+
198+
Decision: confirmed. The byte axis matters; brotli-11 buys 25 % bytes at zero decode cost.
199+
200+
---
201+
202+
## Final verdict
203+
204+
**Winner: `len-prefixed × brotli-11`** (59 757 B, 0.70 ms decode, 25-LOC parser).
205+
206+
The runner-up is `sentinel-cat × brotli-11` (59 670 B, 0.75 ms decode); the two are statistically indistinguishable on bytes (87-byte delta, 0.15 %), and `len-prefixed` is picked on robustness grounds (binary-safe header without relying on the absence of NUL bytes in the payload).
207+
208+
All zstd and shared-dictionary candidates are DISCARDed because they violate the "no external libraries + `DecompressionStream` baseline" constraint from SCOPE.md §3.

0 commit comments

Comments
 (0)