You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs(specs): the analytics surfaces consolidate into REQ-INSIGHTS-HTML-EXPORT (issue #181)
Commit 4 of the consolidation, plus the one code change the spec pass
exposed: removing the terminal surfaces left REQ-TOPOLOGY-GRAPH (h)'s
next/overdue facts with NO surface (the TUI and text renderers carried
them alone; the artifact's tips never did) -- the exact design-to-data
gap the clause exists to prevent -- so the scene's trigger tips now
render next/overdue against the page's own generation instant, in the
same PR that removed their last other home, with pins.
- REQ-INSIGHTS-HTML-EXPORT: the bare command, the i key, and every
normative clause REQ-GRAPH-HTML-EXPORT carried, absorbed verbatim.
- REQ-GRAPH-HTML-EXPORT: SUPERSEDED; the ID stays a permanent address.
- REQ-COST-ANALYTICS: surfaces become the page, dispatch_costs and
insights whatif; rules (a)-(g) untouched; the no-TTY row leaves with
the command; the (no flow) row rewords to the fold grain.
- REQ-TOPOLOGY-GRAPH: the surface is the artifact's topology pane;
honesty rules stand, (h) retargeted to tips and badges.
- REQ-ADMIN-VIA-PI-EXTENSION: the command list; tools UNCHANGED,
checked (dispatch_costs stays; topology stays non-model-callable).
- DES-ADMIN-VIA-PI-EXTENSION: four views, one snapshot poll, the
i-key done-action route recorded with its rationale;
DES-QUEUE-BULLMQ-OVER-CUSTOM's parenthetical renamed.
- OQ-024 rescoped to the insights export, posture unchanged.
- INT-* UNCHANGED, checked (grep-verified: no interface names the
removed commands).
Suite green in the CI posture: 2181 tests, 0 skipped, live Valkey.
Signed-off-by: Rob Boerman <robboerman@live.nl>
Copy file name to clipboardExpand all lines: specs/design.md
+19-30Lines changed: 19 additions & 30 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -126,7 +126,7 @@ money with no upstream turn limit (`REQ-RUNNER-TURN-BUDGET`).
126
126
retry policy — each of which is an independent requirement here, not a bonus. Building four mechanisms
127
127
to avoid one dependency is precisely how a solo-maintainer project drowns in maintenance. Its Bull
128
128
Board dashboard was a fifth draw; `DES-ADMIN-VIA-PI-EXTENSION` drops the **served** web surface
129
-
entirely (the graph HTML artifact is a static file the operator's own browser opens — nothing serves
129
+
entirely (the insights HTML artifact is a static file the operator's own browser opens — nothing serves
130
130
it, nothing listens), so the case now stands on the four queue mechanisms alone. Redis persistence (AOF) is what makes
131
131
`REQ-QUEUE-BURST-NO-DROP` survive a reboot; an in-memory queue would lose the wait-list on the first
132
132
restart.
@@ -859,8 +859,8 @@ money with no upstream turn limit (`REQ-RUNNER-TURN-BUDGET`).
859
859
-**Decision**: The admin surface is a **pi extension** shipped in an `admin/` workspace, loaded into the
860
860
operator's own interactive pi session (via `-e`, `~/.pi/agent/extensions`, or a trust-gated
861
861
`.pi/extensions`). It provides operator-only slash commands
862
-
(`/dispatch status|pause|resume|runs|logs|budget|triggers|costs|graph|settings|set|unset`) and one
863
-
self-refreshing TUI overlay component with **six in-component views**: **LIST** — a framed,
862
+
(`/dispatch status|pause|resume|runs|logs|budget|triggers|insights|settings|set|unset`) and one
863
+
self-refreshing TUI overlay component with **four in-component views**: **LIST** — a framed,
864
864
**theme-colored** panel (color via pi's injected `Theme`, applied post-layout so pi's ANSI-aware
865
865
`visibleWidth` still frames it) carrying a status header, day/week/month **SPEND meters** (colored by the
866
866
same `windowState` the worker enforces) plus a daily **token** counter, a unified **TRIGGERS** pane whose
@@ -883,33 +883,21 @@ money with no upstream turn limit (`REQ-RUNNER-TURN-BUDGET`).
883
883
job's `.log`**inside the overlay** through an injected `deps.tailLog` seam whose `fs` read lives in
884
884
`index.ts` (the log CONTENT stays `clip`-stripped and uncolored — only the chrome is themed), opening
885
885
**pinned to the bottom in follow mode**: scrolling up pauses following, reaching the bottom re-arms it,
886
-
and the footer names the state (`follow`/`paused`) so stale lines cannot pass as live; and **COSTS**
887
-
(`c`, `REQ-COST-ANALYTICS`) — verdict-first cost analytics over one retention-bounded fold
888
-
(`DES-COST-FOLD-BY-SCAN`): per-plan SAVING/LOSING verdicts with the API-rate comparison, a daily spend
889
-
sparkline, by-flow/by-model tables whose every money cell funnels through the typed-cost formatter,
890
-
plan blocks showing amortized $/run and peak-window **facts** (never an invented burn-down), a
891
-
provenance footer naming the pi-ai pin and the unmetered/not-repriceable counts, and a keyboard
892
-
**what-if** (`w` cycles a shortlist of models seen in history plus declared counterfactual targets;
893
-
`/` type-to-filters the full priced catalog through the line-input primitive) whose estimates are
894
-
always marked, name their rates version, and carry the cross-provider tokenizer caveat when the
895
-
target's provider differs. The costs data path is lazy and throttled: fetched on view entry and window
896
-
change, refreshed on the poll tick only when stale — the fold is cheap, but a per-second full-directory
897
-
scan is the kind of quiet load a dashboard must not add. And **GRAPH** (`g`, `REQ-TOPOLOGY-GRAPH`,
898
-
issue #54) — the trigger/flow topology from the same assembled model as `/dispatch graph`
899
-
(`DES-GRAPH-EDGE-DERIVATION` is the one place its edge rules live): a folder-grouped tree with
900
-
foldable group headers, trigger rows carrying joined run stats and dangling badges, skill rows with
901
-
reachability badges and their outgoing edges as evidence-labelled annotation rows, the honesty
902
-
counters, and the caps line on every render. Enter on a trigger row reuses the existing
903
-
TRIGGER_DETAIL drill by raw file index. The graph data path is the **strictest** of the three view
904
-
policies: fetched on entry and on `r` only, NEVER on the poll tick, because the enumeration spawns
905
-
git per folder — heavier than even the costs scan, and topology changes when the operator edits
906
-
things, not per second. The unframed degrade reuses `renderGraph` whole. `graph html`
907
-
(`REQ-GRAPH-HTML-EXPORT`) additionally renders the same model as a **self-contained HTML artifact**
908
-
(inline SVG/CSS/JS, zero external requests, works over `file://`) written **atomically to the stable
909
-
path**`<graphDir>/graph.html` (tmp+rename — stable so re-running updates an already-open tab
910
-
through the page's own Reload/auto-reload controls, atomic so that tab's reload never reads half a
911
-
file), then prints the `file://` URL **first** and best-effort opens the platform browser through
912
-
the worker's shared opener — skipped and said over SSH or without a display, `--no-open` always. The
886
+
and the footer names the state (`follow`/`paused`) so stale lines cannot pass as live. Analytics
887
+
live on the **insights artifact** (`REQ-INSIGHTS-HTML-EXPORT`, issue #181), the ONE surface for
888
+
"what is this deployment wired to do and what does it cost": bare `/dispatch insights` (and the
889
+
overlay's `i` key, which resolves the overlay with a done-action so `index.ts` writes and opens
890
+
the page between overlays — the addTrigger route, no dep seam, no TUI suspend bracket) writes the
891
+
self-contained page **atomically to the stable path**`<graphDir>/insights.html` (tmp+rename —
892
+
stable so re-running updates an already-open tab through the page's own Reload/auto-reload
893
+
controls, atomic so that tab's reload never reads half a file), prints the `file://` URL **first**
894
+
and best-effort opens the platform browser through the worker's shared opener — skipped and said
895
+
over SSH or without a display, `--no-open` always; `insights whatif` keeps the re-pricing
896
+
estimator as a command (its reply goes through the admin channel like every read). The dashboard
897
+
itself carries **no analytics fetch paths at all** anymore: the two per-view refresh policies the
898
+
removed COSTS/GRAPH views needed (the stale-gated tick piggyback; entry-plus-`r`-only around the
899
+
git-spawning enumeration) left with them, and the overlay is back to one snapshot poll plus the
900
+
tail read. The
913
901
extension still **binds no network port at all**: a file with no server is not a surface — nothing
914
902
listens, nothing off-machine gained reachability — so this stays strictly narrower than the
915
903
superseded `127.0.0.1` panel. The artifact carries **run-record fields and operator-authored
@@ -1957,6 +1945,7 @@ a tunnel.
1957
1945
1958
1946
| Date | Change |
1959
1947
|---|---|
1948
+
| 2026-08-12 | Issue #181 (insights becomes the ONE analytics surface). **DES-ADMIN-VIA-PI-EXTENSION AMENDED**: six in-component views become FOUR — the COSTS and GRAPH views leave the overlay for the insights artifact, taking their two per-view refresh policies with them (the stale-gated tick piggyback and the entry-plus-`r`-only posture existed for those fetch paths; the overlay is back to one snapshot poll plus the tail read); the slash-command list drops `costs` and `graph` for `insights`; the graph-html paragraph re-homes onto the bare `insights` command; the LIST footer's `c costs`/`g graph` pair becomes `i insights` (51+18=69 of 76 columns, the arithmetic comment re-run), and the `i` key resolves the overlay with a done-action so index.ts writes and opens the page between overlays — the addTrigger route, deliberately not a dep seam and not a TUI suspend bracket, and deliberately BEFORE the dialog guard (the action needs no dialogs, and an older pi without them must still reach the one analytics surface). `DES-QUEUE-BULLMQ-OVER-CUSTOM`'s parenthetical names the insights artifact now. The dashboard's fs ban is UNCHANGED, checked, and dashboard.ts drops its pricing/costs/graph-model imports entirely. **DES-COST-FOLD-BY-SCAN UNCHANGED, checked** (the fold and its joins are what the page is made of; nothing about them moved). **DES-GRAPH-EDGE-DERIVATION UNCHANGED, checked** (the edge rules' one home; the model gained no vocabulary). |
1960
1949
| 2026-08-12 | Issue #175 (the insights artifact, the fourth slice). **DES-ADMIN-VIA-PI-EXTENSION AMENDED**: `insights html` joins the slash-command list (bare `insights` answers usage — the artifact IS the feature), completion covers `insights html <window>`, and the artifact lands beside `graph.html` under the one `graphDir` (a second directory would churn resolvePaths and the wizard for zero capability). The no-port property (:913) and the served-page rejection (the #54 row) are not reopened: this is a second `file://` artifact, same socket-to-file substitution. The factoring facts are load-bearing and recorded here: `graph-html.mjs` may be a source of exports for a sibling pure emitter, it may never load one itself — its purity pin is substring-level and directional, which is exactly what makes the reuse safe — so `buildGraphScene` (the normalize+layout+SVG-emission half of `buildGraphHtml`, a behavior-preserving extraction) is what `insights-html.mjs` composes, and the money strings come from the REAL `panel.mjs` formatter (zero own module loads there), not a hand-copied twin. Rejected: a shared third emitter module (impossible under the substring ban without weakening it); duplicating the layout/escaping (an escapeHtml drift between two artifacts is an XSS waiting); folding costs into `graph.html` in place (an operator sharing topology should not be forced to share spend); a charting library (the file:// posture forbids external requests, and hand-rolled rectangles need no supply chain). **DES-GRAPH-EDGE-DERIVATION UNCHANGED, checked. DES-COST-FOLD-BY-SCAN UNCHANGED, checked** (the artifact consumes the fold; the fold learned nothing new). |
1961
1950
| 2026-08-12 | Issue #175 (spend and schedule on the graph, the third insights slice). **DES-GRAPH-EDGE-DERIVATION AMENDED**, one clause: a trigger node may carry `cost` (the typed spend `foldTriggerCosts` mapped onto its node id) — spend is a node fact beside runs/lastOutcome, NOT a new edge kind and NOT a flag, so the closed vocabularies and their pins stand byte-identical; the assemblers wire it with one extra file read (subscriptions) and two pure folds over the scan they already paid for, which is why the GRAPH view's entry-plus-`r`-only refresh policy is untouched (the real-poll pin proves it). **DES-ADMIN-VIA-PI-EXTENSION AMENDED**: gtrigger rows phrase `next` as a countdown against the model's own `generatedAt`, never a live clock — a stale model shows its stale countdown honestly, and render() stays a pure read of state. graph.html's normalizeModel allowlist gains `meta.chainRefusals`, `meta.injectedUnreachable` and observed-edge `lastEndedAt`; node `cost` is deliberately NOT allowlisted there (the page cannot use the `from` clause, and a hand-copied money formatter pinned by parity test is a cost the insights artifact avoids by taking the real formatter). **DES-COST-FOLD-BY-SCAN UNCHANGED, checked.** |
1962
1951
| 2026-08-12 | Issue #175 (per-trigger and per-repo spend, the second insights slice). **DES-COST-FOLD-BY-SCAN AMENDED**: the fold gains `byTrigger`/`byRepo` rollups and the `foldTriggerCosts` node-id-keyed spend map, with the join passed IN (`attributeRunsToTriggers`, new in the read-model beside `joinRunsToTriggers`, whose index+type doctrine and cron jobId grammar it reuses verbatim — `triggerMatchLabel` is now exported from graph-model so the label vocabulary has one home); "the fold re-deriving the join" joins the Rejected list. `repoOfTarget` moves the target-stripping grammar into costs.mjs and `forgeRepoTargets` now calls it, so the graph's repo list and the cost fold's repo table can never disagree on what a repo is. **DES-ADMIN-VIA-PI-EXTENSION AMENDED**: the COSTS view's `f` cycles four tables; the trigger join adds one FILE read (readTriggers) to fetchCosts — no spawn, so the 10s stale-gated poll piggyback policy stands and the GRAPH view's entry-plus-`r` posture is untouched. **DES-GRAPH-EDGE-DERIVATION UNCHANGED, checked.** |
the insights page) best-effort spawns the platform opener (`open`/`xdg-open`/`cmd start`)
766
767
through the worker's shared `open-browser` module. The argv table is pinned per platform by unit
767
768
test and the module is the same one `setup github` has shipped since issue #81 — but no CI runs a
768
769
desktop session on all three platforms, so "the browser actually appears" is verified by reading
@@ -776,14 +777,16 @@ adversarial passes did.
776
777
already have.
777
778
-**What would close it**: a per-platform desktop CI job, which this project will not buy for a
778
779
convenience spawn.
779
-
-**Raised by**: issue #54, while landing `REQ-GRAPH-HTML-EXPORT`.
780
+
-**Raised by**: issue #54, while landing `REQ-GRAPH-HTML-EXPORT`; rescoped to the insights export
781
+
by issue #181, which superseded that entry into `REQ-INSIGHTS-HTML-EXPORT`.
780
782
781
783
---
782
784
783
785
## Revision History
784
786
785
787
| Date | Change |
786
788
|---|---|
789
+
| 2026-08-12 | Issue #181. **OQ-024 RESCOPED** from the graph export to the insights export: `graph html` is gone (REQ-GRAPH-HTML-EXPORT superseded into REQ-INSIGHTS-HTML-EXPORT) and the bare `/dispatch insights` is now the one command carrying the browser spawn; the question, the WATCH posture, the bounds (`--no-open`, the SSH/display skip, the printed URL as the contract) and what-would-close-it are all unchanged — only the surface name moved. |
787
790
| 2026-08-11 | Issue #54 (`REQ-GRAPH-HTML-EXPORT`). Added **OQ-024** (`WATCH`, the `OQ-016` shape): the graph export's browser spawn is verified by reading and by the pinned per-platform argv table, not by a desktop CI on three platforms this project will not buy for a convenience spawn; the printed `file://` URL is the contract and the failure mode is one manual click. **OQ-008 and OQ-009 pointers already in place from the earlier #54 slices, UNCHANGED, checked**: the export re-reads the triggers file per run like every other graph consumer, and draws no forge-parent or cross-folder chain edge because the model it renders cannot contain one. |
788
791
| 2026-08-09 | Follow-up audit after issue #60. Added **`OQ-023`**: a prepare-stage refusal posts no comment on the issue, so a requester sees a label applied and then nothing. True of `sha-gone` since it shipped and now true of six reasons, because #60 added five. Recorded rather than fixed, and recorded as NOTICED rather than designed: the ordering that makes these refusals free is what puts them past the point the processor's `comment` seam is wired for, so the silence is a consequence of `CONST-BUDGET-BEFORE-TOKENS` rather than a judgement about what is worth saying. What bounds it is that every one is in the run record with its own reason token and that `doctor` reports the `skills-dir-*` causes before anything fires. What would close it is threading the `comment` seam into the policy branch, which carries one real question: a repo over a cap would then comment on every delivery, so it wants the dedup the spend refusals already have. This entry exists because the #60 plan said it was worth an OQ and then did not write one. |
789
792
| 2026-08-09 | Issue #60 (Gap 2). Added **`OQ-022`**: injected skills are deliberately not AI-reachable, because `DES-AI-TRIGGER-FLOW-GATE` reads the target repo's committed `.pi/skills` at a pinned sha and an injected skill is not there. Recorded as ACCEPTED rather than as a risk row, because it is the fail-CLOSED direction and the residual is an operator SURPRISE rather than a hole: an injected `SKILL.md` carrying `ai-trigger: allow` is never read, so the opt-in is written and nothing honours it, which is why `doctor` now warns. Records what would NOT be an acceptable close -- reading the frontmatter out of the injected tree, since that opt-in is meaningful in a repo only because a merge gated it, and in a runtime-editable directory it would put the authorization inside the content being authorized. `OQ-012` **UNCHANGED, checked**: an operator-built image is unaffected, since the injected root is a worker-side copy into `/job` and needs nothing of the image. `OQ-004` **UNCHANGED in kind**: the injected tier adds instructions, not egress. |
0 commit comments