feat(phase-55): split setup vs query and honest percentiles (#33)

The 64.6s "TOC navigation" p50 was build_toc_segment (ingest-time
MockSummarizer rollup) plus two RocksDB lookups. Query navigation is now
timed separately: medium/warm single.toc p50 = 0.13ms (n=30).

Vector model load, index build, and query are three steps. p90 requires
n>=10; p99 requires n>=30; otherwise min/median/max. Warm = one setup +
N queries. Schema 2 baseline regenerated from a real medium/warm/30 run.

Co-authored-by: Richard Hightower <rick@spillwave.com>

Author: RichardHightower

.planning/ROADMAP.md

@@ -12,7 +12,7 @@
 - ✅ **v2.6 Cognitive Retrieval** — Phases 39-44 (shipped 2026-03-16)
 - ✅ **v2.7 Multi-Runtime Portability** — Phases 45-50 (shipped 2026-03-22)
 - **v3.0 Competitive Parity & Benchmarks** — Phases 51-53 + Phase 51.5 (in progress; Phase 51.5 merged 2026-04-28)
-- **v3.1 Make It True** — Phases 54-58 (in progress; Phase 54 Integration Truth executing)
+- **v3.1 Make It True** — Phases 54-58 (in progress; Phase 54 merged 2026-08-30, Phase 55 executing)
 
 ## Phases
 

.planning/STATE.md

@@ -3,14 +3,14 @@ gsd_state_version: 1.0
 milestone_name: Make It True
 status: in_progress
 stopped_at: null
-last_updated: "2026-08-30T08:00:00.000Z"
-last_activity: 2026-08-30 — Phase 54 Integration Truth implemented on feature/phase-54-integration-truth
+last_updated: "2026-08-30T17:30:00.000Z"
+last_activity: 2026-08-30 — Phase 55 Performance Truth implemented (medium/warm/30 artifact)
 progress:
   total_phases: 5
-  completed_phases: 0
+  completed_phases: 1
   total_plans: 14
-  completed_plans: 6
-  percent: 43
+  completed_plans: 8
+  percent: 57
 ---
 
 # Project State
@@ -20,39 +20,36 @@ progress:
 See: .planning/PROJECT.md (updated 2026-03-22)
 
 **Core value:** Agent can answer "what were we talking about last week?" without scanning everything
-**Current focus:** v3.1 Phase 54 — Integration Truth (wire orchestrator, fix silent no-ops)
+**Current focus:** v3.1 Phase 55 — Performance Truth (setup vs query split; honest percentiles)
 
 ## Current Position
 
-Phase: 54 of 58 (Integration Truth)
-Plan: 01-06 implemented on `feature/phase-54-integration-truth` (PR pending)
-Status: Phase 54 code complete; awaiting PR review
-Last activity: 2026-08-30 — RouteQuery spliced through MemoryOrchestrator; BM25 outbox indexes events; Hybrid fuses BM25+vector; recover_lock policy; honest `--background`
+Phase: 55 of 58 (Performance Truth)
+Plan: 01-02 implemented on `feature/phase-55-performance-truth` (PR pending)
+Status: Phase 54 merged; Phase 55 code + medium/warm/30 artifact ready
+Last activity: 2026-08-30 — `single.toc` query p50 = 0.13ms; 64.6s was `toc_build`
 
-Progress: [████░░░░░░] ~43% (6/14 plans; Phase 54 of 54-58)
+Progress: [██████░░░░] ~57% (8/14 plans; Phase 55 of 54-58)
 
 ## Out-of-band Work
 
 ### Open PRs
 
-| PR | What | Notes |
-|---|---|---|
-| #31 | v3.1 design spec (docs only) | Keep separate from this implementation PR |
+None.
 
 ### Recently Merged
 
 | PR | What | Merged |
 |---|---|---|
+| #32 | Phase 54 Integration Truth | 2026-08-30 |
+| #31 | v3.1 Make It True design spec | 2026-08-30 |
+| #30 | Phase 53 Benchmark Suite | 2026-08-30 |
 | #25 | Phase 53.5: cross-project federated query | 2026-05-14 |
 | #29 | Phase 52: Simple CLI API | 2026-05-14 |
 | #28 | Phase 51: Retrieval Orchestrator | 2026-04-28 |
-| #27 | Phase 51.5: API summarizer wiring | 2026-04-27 |
 
 ## Decisions
 
 - v3.1 scope: Make It True — no new capabilities; close claim/reality gap (Phases 54-58)
-- Orchestrator is wired on the daemon/service side behind RouteQuery (gRPC callers benefit)
-- Canonical fusion API: `fuse` / `fuse_weighted` in memory-orchestrator (only site matching rrf|reciprocal)
-- Lock policy: recover_lock, never panic
-- `--background` exits non-zero; default start is foreground
-- Execution-evidence + crate-reachability + human_verification-as-blocker rules in `.planning/config.json`
+- Phase 55: split setup vs query in `perf_bench`; p90/p99 withheld below 10/30 samples
+- Warm = one setup + N query samples; cold = new store per iteration

.planning/phases/55-performance-truth/55-01-PLAN.md

@@ -0,0 +1,17 @@
+---
+phase: 55-performance-truth
+plan: 01
+type: execute
+wave: 1
+depends_on: []
+files_modified:
+  - crates/e2e-tests/src/bin/perf_bench.rs
+  - docs/benchmarks.md
+  - crates/e2e-tests/benchmarks/latest.json
+  - crates/e2e-tests/benchmarks/baseline.json
+autonomous: true
+---
+
+<objective>
+Split setup vs query in perf_bench so `*.toc` and `*.vector` measure navigation/search, not rollup/model load. Attribute ingest-time cost honestly.
+</objective>

.planning/phases/55-performance-truth/55-02-PLAN.md

@@ -0,0 +1,16 @@
+---
+phase: 55-performance-truth
+plan: 02
+type: execute
+wave: 1
+depends_on: []
+files_modified:
+  - crates/e2e-tests/src/bin/perf_bench.rs
+  - docs/benchmarks.md
+  - crates/e2e-tests/benchmarks/baseline.json
+autonomous: true
+---
+
+<objective>
+Honest percentiles: default 30 query iterations; p90/p99 omitted unless sample count supports them; cold vs warm structurally different; regenerated baseline + caveats.
+</objective>

.planning/phases/55-performance-truth/55-CONTEXT.md

@@ -0,0 +1,34 @@
+# Phase 55: Performance Truth - Context
+
+**Gathered:** 2026-08-30
+**Status:** In execution
+**Source:** docs/plans/v3.1-make-it-true-plan.md
+
+## Phase Boundary
+
+Make recorded perf numbers support — rather than contradict — the core value
+claim. Measurement methodology must survive scrutiny. No new retrieval
+capabilities.
+
+## Root cause (55-01) — confirmed in code
+
+`crates/e2e-tests/src/bin/perf_bench.rs` times **setup + query** under query
+step names:
+
+- `*.toc` wraps `build_toc_segment` (MockSummarizer rollup of the whole
+  corpus) plus two `get_toc_node` lookups. That is the 64.6s "TOC navigation"
+  number. Real navigation is the lookups.
+- `*.vector` wraps Candle embed + HNSW index build plus one search. That is
+  the 7.2s "vector" number.
+- `*.bm25` wraps Tantivy index build plus one search (~245ms).
+- `*.route_query` already times only the RPC (~2.3ms) — the honest query path.
+
+## Decisions
+
+- Split every step into `*_build`/`*_index` (setup, ingest-time) vs query.
+- Default query iterations = 30. p90 only if n≥10; p99 only if n≥30; otherwise
+  min/median/max and say so.
+- Warm: setup once, warmup one query, then N query samples.
+- Cold: new harness per iteration; still split setup vs query timers.
+- `vector_model_load` is a one-shot setup metric, never folded into query.
+- Re-baseline `baseline.json` schema version 2; rewrite `docs/benchmarks.md`.

.planning/phases/55-performance-truth/55-VERIFICATION.md

@@ -0,0 +1,27 @@
+---
+phase: 55-performance-truth
+verified: 2026-08-30
+status: passed
+---
+
+# Phase 55: Performance Truth Verification
+
+**Phase Goal:** recorded perf numbers support the core value claim; methodology survives scrutiny.
+
+## Execution evidence
+
+| # | Truth | Status | Evidence |
+|---|-------|--------|----------|
+| 1 | 64.6s "TOC navigation" was rollup | RUN | `single.toc_build` p50 = 76714 ms on 240 events; old `single.toc` was this timer |
+| 2 | Query `*.toc` is navigation | RUN | `single.toc` p50 = **0.13 ms**, n=30, medium/warm |
+| 3 | Vector model load not in `*.vector` | RUN | `vector_model_load` 156 ms; `vector_index` 12.6 s; `vector` 4.15 s (query embed) |
+| 4 | p90/p99 withheld below 10/30 samples | UNIT + RUN | unit tests; setup steps in latest.json omit p90/p99 (samples=1) |
+| 5 | Warm vs cold are different loops | CODE | Warm: one setup + N queries; cold: new store per iteration |
+| 6 | docs name corpus, samples, caveats | DOCS | `docs/benchmarks.md` committed-result table |
+| 7 | Re-baselined latest.json / baseline.json | RUN | schema 2, 2026-08-30T17:30:20Z, linux/x86_64, 240 events, 30 samples |
+
+## Human verification (blockers)
+
+- [x] Committed `latest.json` from a real medium/warm/30 run
+- [x] `single.toc` warm p50 < 500ms (0.13 ms)
+- [x] Vector split: model load vs index vs query