Skip to content

Commit f1ae4d6

Browse files
ahrzbclaude-agent-ahrzb[bot]
authored andcommitted
docs: name the engine Confit in the live docs
Follow-up to the package extraction. The three reports and the limitations contract described the engine as "the SQL specializer" -- a job description, not a name -- while the package it now ships as is confit. docs/reports/architecture-of-the-sql-specializer.md -> confit-architecture.md prose: "the SQL specializer" / "the specializer" -> Confit docs/known-limitations.md: same, it is the user-facing contract Deliberately NOT renamed: * the `specializer` Rust module and every packages/confit/src/specializer path -- that is the code's own name for the partial-evaluation stage, and it is still accurate; * docs/superpowers/specs/** and the pins JSONs -- historical measurement records, verbatim by design. 2026-07-25-sql-specializer-design.md keeps its filename for the same reason. Gate: pytest 835 passed + 13 xfailed (the limitations doc has an executable twin, so a prose-only edit still gets gated), pre-commit clean.
1 parent a00b8d4 commit f1ae4d6

5 files changed

Lines changed: 10 additions & 10 deletions

backlog/tasks/task-62 - Technical-reports-architecture-methodology-performance-private-upstream-issues-review.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ Write up the specializer work as standalone technical reports under docs/reports
3232
Shipped as PR #49 (merged, master 16a8539).
3333

3434
**Public — `docs/reports/` (3 documents, ~50k chars):**
35-
- `architecture-of-the-sql-specializer.md` — the partial-evaluation engine model end to end: token pre-rewrites → binder → IR → verifier (its six rules quoted) → interpreter/cranelift backends; the null-lane design (no nullable SSA type, so 3VL bugs are unrepresentable rather than merely tested); strict block-param SSA; the shape contract as static proof; stage-B multiplicity (emit.to loops, multimaps, batch-side self-joins); the boundary layers; and the deliberate omissions with the engine-model reason for each.
35+
- `confit-architecture.md` — the partial-evaluation engine model end to end: token pre-rewrites → binder → IR → verifier (its six rules quoted) → interpreter/cranelift backends; the null-lane design (no nullable SSA type, so 3VL bugs are unrepresentable rather than merely tested); strict block-param SSA; the shape contract as static proof; stage-B multiplicity (emit.to loops, multimaps, batch-side self-joins); the boundary layers; and the deliberate omissions with the engine-model reason for each.
3636
- `pins-first-methodology.md` — the discipline: two-outcome contract, verbatim-pins fleets (including the wave-3 over-generalization incident that created the rule), the three-outcome corpus ladder (53→550 of 678, zero FAILs at every rung), the executable limitations twin, the standing differential fuzzer, and an honest accounting of what the discipline costs.
3737
- `performance-report.md` — boundary decomposition (~262 ns/row floor, ~37 ns per output column, ~1.7 µs/row compute vs the twin's 2.2 µs total), us-vs-DuckDB per call (2055× at n=1; crossover ~2–3k rows), the arrow boundary, and the columnar core's build-measure-close arc.
3838

docs/known-limitations.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Known limitations — deliberate, named, and loud
22

3-
This is the user-facing contract of the SQL specializer (`DuckDBInferFn`):
3+
This is the user-facing contract of Confit (`DuckDBInferFn`):
44
what it refuses to serve, **why**, and what you see when you hit a limit.
55
Its executable twin is [`packages/confit/tests/test_known_limitations.py`](../tests/test_known_limitations.py)
66
every limitation below is asserted there, so lifting one breaks a test and
@@ -21,7 +21,7 @@ recorded in a pins spec (`docs/superpowers/specs/`), not an accident.
2121

2222
## 1. The specialization bargain (inherent to the engine model)
2323

24-
The specializer's speed comes from doing ALL general work at build time:
24+
Confit's speed comes from doing ALL general work at build time:
2525
parse once, bind once, compile once, freeze the static tables into the
2626
code. Anything that would require re-doing general work per row is
2727
rejected, permanently by design:

docs/reports/architecture-of-the-sql-specializer.md renamed to docs/reports/confit-architecture.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
# The architecture of the SQL specializer: partial evaluation for row serving
1+
# The architecture of Confit: partial evaluation for row serving
22

3-
**Abstract.** The SQL specializer (`DuckDBInferFn`) is a partial evaluator: it takes fixed SQL and frozen static tables at build time and emits a specialized native function whose only remaining input is the request batch. Its contract is deliberately narrow — for any SQL it either serves bit-for-bit identical to DuckDB 1.5.5 or refuses loudly at build time with a named error; there is no third mode (docs/known-limitations.md). This report describes the pipeline from token stream to native code, the IR design decisions that make whole bug classes unrepresentable rather than merely tested for, the shape contract that turns row-multiplicity into a static proof, and the boundary work that the measurements say is where serving time actually goes. It also records what was deliberately not built, and why the engine model justifies each omission. All numbers are measured (release builds, p50) on this project; the corpus arc ran 53 → 395 → 505 → 511 → 529 → 546 → 550 of 678 DuckDB-mined statements bit-exact, with zero FAILs at every point.
3+
**Abstract.** Confit (`DuckDBInferFn`) is a partial evaluator: it takes fixed SQL and frozen static tables at build time and emits a specialized native function whose only remaining input is the request batch. Its contract is deliberately narrow — for any SQL it either serves bit-for-bit identical to DuckDB 1.5.5 or refuses loudly at build time with a named error; there is no third mode (docs/known-limitations.md). This report describes the pipeline from token stream to native code, the IR design decisions that make whole bug classes unrepresentable rather than merely tested for, the shape contract that turns row-multiplicity into a static proof, and the boundary work that the measurements say is where serving time actually goes. It also records what was deliberately not built, and why the engine model justifies each omission. All numbers are measured (release builds, p50) on this project; the corpus arc ran 53 → 395 → 505 → 511 → 529 → 546 → 550 of 678 DuckDB-mined statements bit-exact, with zero FAILs at every point.
44

55
## 1. The problem shape
66

@@ -15,7 +15,7 @@ Measured concretely (docs/proposals/2026-07-28-columnar-path.md, titanic scenari
1515
| 1024 | 7.75 ms | 3.42 ms | 2.3× |
1616
| 16384–262144 ||| DuckDB wins 3–5× |
1717

18-
DuckDB pays roughly 5.5–7 ms of per-query cost on every call regardless of batch size; the specializer pays it once at build. The crossover sits around 2–3k rows/call. Below it — the serving regime — the specializer wins by one to three orders of magnitude; above it, DuckDB's parallel columnar execution wins, and that is fine, because that is not serving (§8).
18+
DuckDB pays roughly 5.5–7 ms of per-query cost on every call regardless of batch size; Confit pays it once at build. The crossover sits around 2–3k rows/call. Below it — the serving regime — Confit wins by one to three orders of magnitude; above it, DuckDB's parallel columnar execution wins, and that is fine, because that is not serving (§8).
1919

2020
The design (docs/superpowers/specs/2026-07-25-sql-specializer-design.md) frames this as the first Futamura projection: `prepare(sql, static_tables) -> f`, where binding-time analysis collapses everything static — a hash join against a frozen table becomes a probe of a prepare-time map; with no pipeline breakers left, the query is a straight-line function over baked-in data.
2121

docs/reports/performance-report.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
# Performance report: the serving regime, measured
22

3-
**Scope.** This report collects the serving-path performance measurements of the SQL specializer (`DuckDBInferFn`) as of 2026-07-28: where a call's time actually goes, how the engine compares per call with DuckDB itself, what the Arrow boundary bought, and why the columnar execution core was built, measured, and then deliberately closed unmerged. Every number is a p50 from a release build on this project's benchmark harness (`benchmarks/bench_serving.py`, `scripts/bench_scaling.py` on branch `task-61-columnar-core`), taken after a three-way parity gate (specializer == DuckDB == handcrafted Python) confirmed the outputs agree — a scenario that disagrees aborts the bench before any timing. The headline, stated plainly: for calls of 1–1,000 rows the specializer is one to three orders of magnitude faster per call than invoking DuckDB, the crossover sits at a few thousand rows per call, and above that DuckDB wins by 3–5× — a boundary we have measured and chosen not to contest.
3+
**Scope.** This report collects the serving-path performance measurements of Confit (`DuckDBInferFn`) as of 2026-07-28: where a call's time actually goes, how the engine compares per call with DuckDB itself, what the Arrow boundary bought, and why the columnar execution core was built, measured, and then deliberately closed unmerged. Every number is a p50 from a release build on this project's benchmark harness (`benchmarks/bench_serving.py`, `scripts/bench_scaling.py` on branch `task-61-columnar-core`), taken after a three-way parity gate (Confit == DuckDB == handcrafted Python) confirmed the outputs agree — a scenario that disagrees aborts the bench before any timing. The headline, stated plainly: for calls of 1–1,000 rows Confit is one to three orders of magnitude faster per call than invoking DuckDB, the crossover sits at a few thousand rows per call, and above that DuckDB wins by 3–5× — a boundary we have measured and chosen not to contest.
44

55
## 1. The serving problem
66

7-
The product is per-call latency at small batches. An inference service receives one request — one row, or a few dozen, occasionally a thousand — and must answer now. Throughput over a quarter-million-row table is an analytics problem; DuckDB already solves it well. The specializer exists for the other regime: `prepare(sql, static_tables) -> f` once at fit time (seconds are fine), then `f(batch)` millions of times with a nanosecond budget (`docs/superpowers/specs/2026-07-25-sql-specializer-design.md`).
7+
The product is per-call latency at small batches. An inference service receives one request — one row, or a few dozen, occasionally a thousand — and must answer now. Throughput over a quarter-million-row table is an analytics problem; DuckDB already solves it well. Confit exists for the other regime: `prepare(sql, static_tables) -> f` once at fit time (seconds are fine), then `f(batch)` millions of times with a nanosecond budget (`docs/superpowers/specs/2026-07-25-sql-specializer-design.md`).
88

99
The performance numbers below are only meaningful against the correctness backdrop, so it is stated once: the engine's contract is *serve bit-exact with DuckDB 1.5.5 or refuse loudly at build time* (`docs/known-limitations.md`). The mined-corpus replay (`packages/confit/tests/test_corpus_replay.py`) enforces a three-outcome contract — match, clean-unsupported, or FAIL — with zero FAILs required. The match ladder over the waves ran 53 → 395 → 505 → 511 → 529 → 546 → 550 of 678 statements, zero FAILs throughout (wave census anchors in `docs/superpowers/specs/2026-07-26-wave5-structural-pins.md`, `2026-07-27-waveB-regexp-pins.md`, `2026-07-28-waveA-structural-tails.md`, `2026-07-28-stageB-multiplicity-pins.md`; current 550/678 recorded in `docs/known-limitations.md`). Nothing below was bought by relaxing that.
1010

@@ -80,7 +80,7 @@ And house_prices, the scenario where the columnar lane does best:
8080
| 16,384 | 4,379 | 5,792 | 5,592 | 1,815 |
8181
| 262,144 | 4,720 | 5,987 | 6,505 | 1,163 |
8282

83-
Readings, all visible in the raw JSON. Our per-row cost is essentially flat from n=1,024 upward (~3–5 µs/row depending on scenario) — the specializer does not get faster per row with scale, because it was never built to. DuckDB's per-row cost falls three orders of magnitude as its fixed cost amortises and its parallelism engages, crossing us at ~4k rows/call on titanic, fraud_txn and store_sales, and between 4k and 16k on house_prices (per call at 4,096: house 14.58 ms us vs 15.58 ms DuckDB — still ours; at 16,384: 71.8 ms vs 29.7 ms — DuckDB's). At the top end DuckDB wins by ~5× (titanic n=262,144: 700 vs 3,847 ns/row). The Arrow lane's fixed cost is stark at n=1 (189–445 µs across scenarios against a 4–10 µs row-path call) and fully amortised by ~1k, where it beats the twin on house_prices and titanic and trails it modestly on the string-heavy scenarios. One run-to-run honesty note: house_prices n=1,024 measured 2.80 ms per call in the TASK-60 bench and 3.79 ms in this sweep — different runs on different days with the columnar core active in the latter; treat gaps of that size between editions as within run variance, not as findings.
83+
Readings, all visible in the raw JSON. Our per-row cost is essentially flat from n=1,024 upward (~3–5 µs/row depending on scenario) — Confit does not get faster per row with scale, because it was never built to. DuckDB's per-row cost falls three orders of magnitude as its fixed cost amortises and its parallelism engages, crossing us at ~4k rows/call on titanic, fraud_txn and store_sales, and between 4k and 16k on house_prices (per call at 4,096: house 14.58 ms us vs 15.58 ms DuckDB — still ours; at 16,384: 71.8 ms vs 29.7 ms — DuckDB's). At the top end DuckDB wins by ~5× (titanic n=262,144: 700 vs 3,847 ns/row). The Arrow lane's fixed cost is stark at n=1 (189–445 µs across scenarios against a 4–10 µs row-path call) and fully amortised by ~1k, where it beats the twin on house_prices and titanic and trails it modestly on the string-heavy scenarios. One run-to-run honesty note: house_prices n=1,024 measured 2.80 ms per call in the TASK-60 bench and 3.79 ms in this sweep — different runs on different days with the columnar core active in the latter; treat gaps of that size between editions as within run variance, not as findings.
8484

8585
## 7. Methodology
8686

docs/reports/pins-first-methodology.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Pins-first: building a bit-exact engine twin without trusting yourself
22

3-
**Abstract.** The SQL specializer (`DuckDBInferFn`) is a partial evaluator that turns a SQL query plus frozen static tables into a specialized native serving function, bit-exact with DuckDB 1.5.5. Its central engineering problem is not performance — it is that a second implementation of somebody else's SQL dialect is, by default, a machine for producing plausible wrong answers. This report describes the discipline that kept it honest: a two-outcome contract (serve bit-for-bit or refuse loudly at build time), *pins* — measuring DuckDB with recorded, verbatim queries before writing any code — a three-outcome corpus replay mined from DuckDB's own test suite (550 of 678 statements bit-exact, zero failures throughout), an executable limitations document, and a standing differential fuzzer that has kept auditing the translation layer long after the initial battery passed. Every mechanism assumes the same thing: the author's intuition about SQL semantics is wrong until an executed query says otherwise. The costs of the discipline are real and are itemised at the end.
3+
**Abstract.** Confit (`DuckDBInferFn`) is a partial evaluator that turns a SQL query plus frozen static tables into a specialized native serving function, bit-exact with DuckDB 1.5.5. Its central engineering problem is not performance — it is that a second implementation of somebody else's SQL dialect is, by default, a machine for producing plausible wrong answers. This report describes the discipline that kept it honest: a two-outcome contract (serve bit-for-bit or refuse loudly at build time), *pins* — measuring DuckDB with recorded, verbatim queries before writing any code — a three-outcome corpus replay mined from DuckDB's own test suite (550 of 678 statements bit-exact, zero failures throughout), an executable limitations document, and a standing differential fuzzer that has kept auditing the translation layer long after the initial battery passed. Every mechanism assumes the same thing: the author's intuition about SQL semantics is wrong until an executed query says otherwise. The costs of the discipline are real and are itemised at the end.
44

55
## 1. The two-outcome contract
66

0 commit comments

Comments
 (0)