Skip to content

Commit 41ac08d

Browse files
committed
feat(admin): spend per trigger and per repo, and the COSTS table cycle (issue #175)
The second insights slice: the joins the graph already owned become the cost breakdowns the fold was missing. - attributeRunsToTriggers (read-model, beside joinRunsToTriggers): the per-jobId join for the cost fold -- cron via the raw repeat jobId grammar cronRunStats already uses, forge via the index+type agreement doctrine. A forge record whose pair disagrees with the current file (or predates the field) gets an explicit unattributed entry, never silence, so the fold cannot misfile it under manual. - foldCosts gains triggerJoin -> byTrigger (typed costs, outcome split, failedCost, honesty buckets chained/manual/unattributed pinned to the tail) and byRepo (repoOfTarget, the one stripping grammar, now also called by forgeRepoTargets). byTrigger is null without a join: not computed and nothing attributed are different sentences. - foldTriggerCosts: the per-trigger spend map keyed by the graph node id trigger:<index>, for the topology surfaces the next slices add. - The COSTS view's f cycles flow/model/trigger/repo; footer hint renamed [f] table so width 80 still fits whole; w stays flow-table-only. /dispatch costs renders the two new tables; the trigger join is one file read, so the 10s stale-gated poll piggyback policy stands untouched. Specs in the same PR: REQ-COST-ANALYTICS and DES-COST-FOLD-BY-SCAN and DES-ADMIN-VIA-PI-EXTENSION amended; REQ-TOPOLOGY-GRAPH UNCHANGED, checked (its join doctrine gained a second consumer, not a second definition); DES-GRAPH-EDGE-DERIVATION UNCHANGED, checked. Suite green in the CI posture: 2194 tests, 0 skipped, live Valkey. Signed-off-by: Rob Boerman <robboerman@live.nl>
1 parent 043a23b commit 41ac08d

13 files changed

Lines changed: 520 additions & 20 deletions

admin/src/costs.mjs

Lines changed: 129 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -235,7 +235,7 @@ function repriceRows(rows, counterfactual, pricing, { extraExcluded = 0 } = {})
235235
* read-model the costs view renders. Everything below is derived per call and thrown away -- nothing
236236
* classified here is ever written anywhere.
237237
*/
238-
export function foldCosts({ records, subscriptions, pricing, nowMs, piAiPin = null, sinceMs = null }) {
238+
export function foldCosts({ records, subscriptions, pricing, nowMs, piAiPin = null, sinceMs = null, triggerJoin = null }) {
239239
const subs = Array.isArray(subscriptions) ? subscriptions : [];
240240
const runs = [];
241241
for (const record of Array.isArray(records) ? records : []) {
@@ -267,6 +267,10 @@ export function foldCosts({ records, subscriptions, pricing, nowMs, piAiPin = nu
267267
daily: buildDaily(runs, firstRunMs ?? fromMs, toMs),
268268
byFlow: buildByFlow(runs, subs, pricing),
269269
byModel: buildByModel(runs),
270+
// null, not [], without a join: "not computed" and "nothing attributed" are different sentences,
271+
// and a caller that wired no triggers must not render an empty table that looks exhaustive.
272+
byTrigger: triggerJoin ? buildByTrigger(runs, triggerJoin) : null,
273+
byRepo: buildByRepo(runs),
270274
plans: buildPlans(runs, subs, pricing, days),
271275
provenance: buildProvenance(runs, piAiPin),
272276
};
@@ -331,6 +335,130 @@ function buildByFlow(runs, subs, pricing) {
331335
return byFlow.sort((a, b) => b.cost.usd - a.cost.usd || a.flow.localeCompare(b.flow));
332336
}
333337

338+
// The honesty buckets a run lands in when no trigger claims it, pinned to the table's tail in this
339+
// order however the dollars compare: a bucket named "(unattributed)" sorting above real triggers on
340+
// cost would read as the biggest spender when it is really the biggest unknown.
341+
const TRIGGER_TAIL = ["chained", "manual", "unattributed"];
342+
const TRIGGER_TAIL_LABELS = { chained: "(chained runs)", manual: "(manual/local)", unattributed: "(unattributed)" };
343+
344+
/**
345+
* Per-trigger rollup over the attribution the read-model passed IN (`attributeRunsToTriggers` --
346+
* the index+type agreement doctrine and the raw cron jobId grammar live there and are not re-derived
347+
* by this fold). Every run lands somewhere: a joined run under its trigger's graph node id
348+
* (`trigger:<index>`), a forge run the join refused under "unattributed", a run with a parentJobId
349+
* under "chained" (not rolled up to the ancestor trigger: walking parent chains across the retention
350+
* boundary would attribute partially, and a partial rollup wearing a trigger's name is a lie), and
351+
* everything else under "manual". `key` is the machine key, `label` display-only -- the byFlow
352+
* lesson. The outcome split rides along because "what did failed runs cost" is a per-trigger
353+
* question: a trigger whose spend is mostly failures is a different problem than an expensive one.
354+
*/
355+
function buildByTrigger(runs, triggerJoin) {
356+
const byJobId = triggerJoin?.byJobId && typeof triggerJoin.byJobId === "object" ? triggerJoin.byJobId : {};
357+
const groups = new Map();
358+
for (const r of runs) {
359+
const record = r.record;
360+
const entry = typeof record.jobId === "string" ? byJobId[record.jobId] : undefined;
361+
let bucket;
362+
if (entry && entry.key !== "unattributed") {
363+
bucket = { key: entry.key, index: entry.index ?? null, type: entry.type ?? null, label: entry.label ?? entry.key };
364+
} else if (entry) {
365+
bucket = { key: "unattributed", index: null, type: null, label: TRIGGER_TAIL_LABELS.unattributed };
366+
} else if (typeof record.parentJobId === "string" && record.parentJobId !== "") {
367+
bucket = { key: "chained", index: null, type: null, label: TRIGGER_TAIL_LABELS.chained };
368+
} else {
369+
bucket = { key: "manual", index: null, type: null, label: TRIGGER_TAIL_LABELS.manual };
370+
}
371+
if (!groups.has(bucket.key)) groups.set(bucket.key, { ...bucket, members: [] });
372+
groups.get(bucket.key).members.push(r);
373+
}
374+
const rows = [];
375+
for (const g of groups.values()) {
376+
const outcomes = { completed: 0, policy: 0, failed: 0 };
377+
for (const m of g.members) {
378+
const o = m.record.outcome;
379+
if (o === "completed" || o === "policy" || o === "failed") outcomes[o] += 1;
380+
}
381+
const failedMembers = g.members.filter((m) => m.record.outcome === "failed");
382+
rows.push({
383+
key: g.key,
384+
index: g.index,
385+
type: g.type,
386+
label: g.label,
387+
runs: g.members.length,
388+
tokens: g.members.reduce((sum, m) => sum + (m.record.tokens?.total ?? 0), 0),
389+
cost: combineContributions(g.members.map((m) => m.contribution)),
390+
outcomes,
391+
failedCost: failedMembers.length > 0 ? combineContributions(failedMembers.map((m) => m.contribution)) : null,
392+
});
393+
}
394+
return rows.sort((a, b) => {
395+
const tailA = TRIGGER_TAIL.indexOf(a.key);
396+
const tailB = TRIGGER_TAIL.indexOf(b.key);
397+
if (tailA !== -1 || tailB !== -1) return tailA === -1 ? -1 : tailB === -1 ? 1 : tailA - tailB;
398+
return b.cost.usd - a.cost.usd || a.label.localeCompare(b.label);
399+
});
400+
}
401+
402+
/** A target's repo shape: the forge issue/MR tail stripped (`repo#12` -> `repo`, `proj!3` -> `proj`);
403+
* targets without a numeric tail (local:<basename>) ride through whole; null stays null. The one
404+
* grammar forgeRepoTargets (read-model) also strips by -- shared so the two can never disagree. */
405+
export function repoOfTarget(target) {
406+
if (typeof target !== "string" || target === "") return null;
407+
const repo = target.replace(/[#!]\d+$/, "");
408+
return repo === "" ? null : repo;
409+
}
410+
411+
/** Per-repo/target rollup. `key` null (with the "(no target)" display label) for records carrying no
412+
* target at all; `kind` is the records' uniform kind or null when a repo saw mixed kinds. */
413+
function buildByRepo(runs) {
414+
const groups = new Map();
415+
for (const r of runs) {
416+
const repo = repoOfTarget(r.record.target);
417+
const mapKey = repo ?? "\u0000none";
418+
if (!groups.has(mapKey)) groups.set(mapKey, { key: repo, label: repo ?? "(no target)", kinds: new Set(), members: [] });
419+
const g = groups.get(mapKey);
420+
if (typeof r.record.kind === "string") g.kinds.add(r.record.kind);
421+
g.members.push(r);
422+
}
423+
const rows = [];
424+
for (const g of groups.values()) {
425+
rows.push({
426+
key: g.key,
427+
label: g.label,
428+
kind: g.kinds.size === 1 ? [...g.kinds][0] : null,
429+
runs: g.members.length,
430+
tokens: g.members.reduce((sum, m) => sum + (m.record.tokens?.total ?? 0), 0),
431+
cost: combineContributions(g.members.map((m) => m.contribution)),
432+
});
433+
}
434+
return rows.sort((a, b) => b.cost.usd - a.cost.usd || a.label.localeCompare(b.label));
435+
}
436+
437+
/**
438+
* The per-trigger spend map for the topology surfaces, keyed by GRAPH NODE ID (`trigger:<index>` --
439+
* graph-model mints exactly this, so a badge lands on its node with no second join vocabulary).
440+
* Only real triggers appear: the chained/manual/unattributed honesty buckets have no node to badge
441+
* and live in byTrigger instead.
442+
*/
443+
export function foldTriggerCosts({ records, subscriptions, pricing, triggerJoin }) {
444+
const subs = Array.isArray(subscriptions) ? subscriptions : [];
445+
const byJobId = triggerJoin?.byJobId && typeof triggerJoin.byJobId === "object" ? triggerJoin.byJobId : {};
446+
const groups = new Map();
447+
for (const record of Array.isArray(records) ? records : []) {
448+
if (!record || typeof record !== "object") continue;
449+
if (recordMs(record) === null) continue; // the fold's own bucketing rule: unplaceable records are dropped
450+
const entry = typeof record.jobId === "string" ? byJobId[record.jobId] : undefined;
451+
if (!entry || entry.key === "unattributed" || !entry.key.startsWith("trigger:")) continue;
452+
if (!groups.has(entry.key)) groups.set(entry.key, []);
453+
groups.get(entry.key).push(runContribution(record, subs, pricing));
454+
}
455+
const out = {};
456+
for (const [key, contribs] of groups) {
457+
out[key] = { cost: combineContributions(contribs), runs: contribs.length };
458+
}
459+
return out;
460+
}
461+
334462
/** Per-(provider,model) rollup from the attribution rows, sorted by cost descending. A run appears in
335463
* a model's `runs` count once, however many of its calls landed there. */
336464
function buildByModel(runs) {

admin/src/dashboard.ts

Lines changed: 63 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ import { windowEndAt } from "@edgehero/pi-dispatch/pause-windows";
3030
// injected seams (`fetchCosts`/`listPricedModels`/`whatIf`), never the façade, so tests stay fully
3131
// canned and the one worker/pricing coupling sits beside the queue and redis this module already owns.
3232
import * as pricing from "@edgehero/pi-dispatch/pricing";
33-
import { listRuns, readSettingsView, mapSchedulers, readTriggers, readPauseWindows, readStagedPackages, readSubscriptions, scanRunRecords, GRAPH_LIMITS, cronRunStats, joinRunsToTriggers, observedChainEdges, collectGraphInputs, forgeRepoTargets } from "./read-model.mjs";
33+
import { listRuns, readSettingsView, mapSchedulers, readTriggers, readPauseWindows, readStagedPackages, readSubscriptions, scanRunRecords, GRAPH_LIMITS, cronRunStats, joinRunsToTriggers, attributeRunsToTriggers, observedChainEdges, collectGraphInputs, forgeRepoTargets } from "./read-model.mjs";
3434
import { renderStatus, renderBudget, renderTriggers, renderSettingsView, renderGraph } from "./render.mjs";
3535
import { buildGraphModel } from "./graph-model.mjs";
3636
import { matchesKey } from "./keys.mjs";
@@ -62,6 +62,9 @@ const DRILL_WIDTH = 70;
6262
// full-directory read is the kind of quiet load a dashboard must not add, so while the view is open a
6363
// poll tick refreshes the fold only once the last fetch is older than this.
6464
const COSTS_STALE_MS = 10_000;
65+
// The rollup tables `f` cycles through (issue #175 added trigger and repo -- the joins the graph
66+
// already owned, finally answering "which trigger burns the most" and "what does repo X cost").
67+
const COSTS_TABLES = ["flow", "model", "trigger", "repo"];
6568
// GRAPH: the cursor-following row window, on the RUNS_VIEWPORT/TAIL_VIEWPORT precedent -- a fixed
6669
// bound with no height dependency, so an unknown terminal height changes nothing. The graph REFRESHES
6770
// only on entry and on `r`, never on the poll tick: fetchGraph spawns git per enumerated folder, a
@@ -138,7 +141,12 @@ export function createDashboardDeps(paths: any) {
138141
if (!Array.isArray(records)) return { unreachable: (records as any)?.unreachable ?? "scan failed" };
139142
const subsView: any = readSubscriptions({ subscriptionsPath: paths.subscriptionsPath });
140143
const subscriptions = Array.isArray(subsView?.subscriptions) ? subsView.subscriptions : [];
141-
const fold = foldCosts({ records, subscriptions, pricing, nowMs, piAiPin: pricing.piAiVersion(), sinceMs });
144+
// The trigger join behind byTrigger: one FILE read (readTriggers, the fetchSnapshot kind) plus a
145+
// pure fold -- no git spawn anywhere on this path, so the 10s stale-gated poll piggyback stays
146+
// exactly as cheap as it was. The graph's entry+`r`-only policy is about spawns, not reads.
147+
const triggersView: any = readTriggers({ triggersPath: paths.triggersPath });
148+
const triggerJoin = attributeRunsToTriggers({ records, triggers: Array.isArray(triggersView?.triggers) ? triggersView.triggers : [] });
149+
const fold = foldCosts({ records, subscriptions, pricing, nowMs, piAiPin: pricing.piAiVersion(), sinceMs, triggerJoin });
142150
return { fold, records, subscriptions };
143151
},
144152
/**
@@ -694,7 +702,7 @@ export function makeDashboard({
694702
return;
695703
}
696704
if (data === "f" || data === "F") {
697-
costs.table = costs.table === "flow" ? "model" : "flow";
705+
costs.table = COSTS_TABLES[(COSTS_TABLES.indexOf(costs.table) + 1) % COSTS_TABLES.length];
698706
costsSel = 0;
699707
tui?.requestRender?.();
700708
return;
@@ -1802,10 +1810,12 @@ function costsWindowLabel(windowKey: string): string {
18021810
return `${MONTHS[now.getUTCMonth()]} ${now.getUTCFullYear()} (mtd)`;
18031811
}
18041812

1805-
/** The active COSTS table's rows -- the array `costsSel` and ↑↓ span. */
1813+
/** The active COSTS table's rows -- the array `costsSel` and ↑↓ span. byTrigger may be null (the
1814+
* seam wired no triggers): the empty array degrades the table to its stated empty line. */
18061815
function costsTableRows(costs: any): any[] {
18071816
const fold = costs?.data?.fold;
1808-
const rows = costs?.table === "model" ? fold?.byModel : fold?.byFlow;
1817+
const rows =
1818+
costs?.table === "model" ? fold?.byModel : costs?.table === "trigger" ? fold?.byTrigger : costs?.table === "repo" ? fold?.byRepo : fold?.byFlow;
18091819
return Array.isArray(rows) ? rows : [];
18101820
}
18111821

@@ -2008,6 +2018,51 @@ function costsTableLines(fold: any, table: string, costsSel: number, inner: numb
20082018
});
20092019
return out;
20102020
}
2021+
if (table === "trigger") {
2022+
// The per-trigger rollup (issue #175): the FAIL column and the failed-spend suffix exist because
2023+
// a trigger whose spend is mostly failures is a different problem than an expensive one. The
2024+
// suffix rides inside the flexible COST column only when nonzero, so the row count (and with it
2025+
// the collapse budget) never depends on outcomes.
2026+
const wName = 24;
2027+
const wFail = 4;
2028+
const wCost = Math.max(8, inner - 2 - wName - wRuns - wFail - wTok - 4 * gap.length);
2029+
out.push(fitLine(" " + [head("TRIGGER", wName), head("RUNS", wRuns, "right"), head("FAIL", wFail, "right"), head("TOKENS", wTok, "right"), head("COST", wCost)].join(gap), inner, styler));
2030+
const rows = Array.isArray(fold.byTrigger) ? fold.byTrigger : [];
2031+
if (rows.length === 0) return [...out, fitLine(styler.fg("dim", " (no runs in this window)"), inner, styler)];
2032+
rows.forEach((r: any, i: number) => {
2033+
const failed = r.outcomes?.failed ?? 0;
2034+
const costCell =
2035+
r.failedCost && failed > 0
2036+
? styler.fmtCost(r.cost) + styler.fg("dim", ` (failed `) + styler.fmtCost(r.failedCost) + styler.fg("dim", `)`)
2037+
: styler.fmtCost(r.cost, wCost);
2038+
const cells = [
2039+
styler.cell(r.label ?? r.key ?? "-", wName),
2040+
styler.cell(String(r.runs ?? 0), wRuns, { align: "right" }),
2041+
styler.cell(failed > 0 ? String(failed) : "·", wFail, { align: "right" }),
2042+
styler.cell(fmtTokens(r.tokens ?? 0), wTok, { align: "right" }),
2043+
costCell,
2044+
];
2045+
out.push(fitLine(cursor(i === costsSel) + " " + cells.join(gap), inner, styler));
2046+
});
2047+
return out;
2048+
}
2049+
if (table === "repo") {
2050+
const wName = 30;
2051+
const wCost = Math.max(8, inner - 2 - wName - wRuns - wTok - 3 * gap.length);
2052+
out.push(fitLine(" " + [head("REPO/TARGET", wName), head("RUNS", wRuns, "right"), head("TOKENS", wTok, "right"), head("COST", wCost)].join(gap), inner, styler));
2053+
const rows = Array.isArray(fold.byRepo) ? fold.byRepo : [];
2054+
if (rows.length === 0) return [...out, fitLine(styler.fg("dim", " (no runs in this window)"), inner, styler)];
2055+
rows.forEach((r: any, i: number) => {
2056+
const cells = [
2057+
styler.cell(r.label ?? "-", wName),
2058+
styler.cell(String(r.runs ?? 0), wRuns, { align: "right" }),
2059+
styler.cell(fmtTokens(r.tokens ?? 0), wTok, { align: "right" }),
2060+
styler.fmtCost(r.cost, wCost),
2061+
];
2062+
out.push(fitLine(cursor(i === costsSel) + " " + cells.join(gap), inner, styler));
2063+
});
2064+
return out;
2065+
}
20112066
const wFlow = 18;
20122067
const wCost = 14;
20132068
const wApi = Math.max(8, inner - 2 - wFlow - wRuns - wTok - wCost - 4 * gap.length);
@@ -2107,10 +2162,11 @@ function provenanceLine(fold: any, inner: number, styler: any): string {
21072162
return fitLine(styler.fg("dim", text), inner, styler);
21082163
}
21092164

2110-
/** The COSTS footer hints -- one line, every layer's keys named. */
2165+
/** The COSTS footer hints -- one line, every layer's keys named. `[f] table` names the cycle rather
2166+
* than enumerating four table names into a footer that must fit width 80 whole. */
21112167
function costsHints(inner: number, styler: any): string {
21122168
const k = (key: string, label: string) => styler.fg("accent", key) + " " + styler.fg("dim", label);
2113-
return fitLine([k("[↑↓]", "row"), k("[f]", "flow/model"), k("[t]", "7d/30d/mtd"), k("[w]", "what-if"), k("[esc]", "back")].join(" "), inner, styler);
2169+
return fitLine([k("[↑↓]", "row"), k("[f]", "table"), k("[t]", "7d/30d/mtd"), k("[w]", "what-if"), k("[esc]", "back")].join(" "), inner, styler);
21142170
}
21152171

21162172
/** ms until the next UTC midnight / Monday 00:00 UTC / month-1 00:00 UTC. */

admin/src/graph-model.mjs

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -460,8 +460,10 @@ export function buildGraphModel({ triggers, schedulers, folderSkills, injectedSk
460460
return model;
461461
}
462462

463-
/** The display label for a trigger's match side, mirroring render.mjs's triggerLine vocabulary. */
464-
function triggerMatchLabel(t) {
463+
/** The display label for a trigger's match side, mirroring render.mjs's triggerLine vocabulary.
464+
* Exported for the read-model's cost attribution (issue #175): one vocabulary for what a trigger is
465+
* called, however many tables call it. */
466+
export function triggerMatchLabel(t) {
465467
switch (t?.type) {
466468
case "cron":
467469
return `${t.id ?? "-"} ${t.pattern ?? "-"}`;

admin/src/index.ts

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,7 @@ import {
7878
GRAPH_LIMITS,
7979
cronRunStats,
8080
joinRunsToTriggers,
81+
attributeRunsToTriggers,
8182
observedChainEdges,
8283
collectGraphInputs,
8384
forgeRepoTargets,
@@ -1030,13 +1031,18 @@ function assembleCosts(paths: any, window: string, flow?: string): any {
10301031
if (!Array.isArray(records)) return records; // { unreachable }
10311032
const subs: any = readSubscriptions({ subscriptionsPath: paths.subscriptionsPath });
10321033
const scoped = typeof flow === "string" && flow !== "" ? records.filter((r: any) => (r?.flow ?? null) === flow) : records;
1034+
// The trigger join over the SCOPED records (a flow-filtered fold attributes only what it folds),
1035+
// the same file-read-plus-pure-fold path the dashboard seam takes.
1036+
const triggersView: any = readTriggers({ triggersPath: paths.triggersPath });
1037+
const triggerJoin = attributeRunsToTriggers({ records: scoped, triggers: Array.isArray(triggersView?.triggers) ? triggersView.triggers : [] });
10331038
const fold = foldCosts({
10341039
records: scoped,
10351040
subscriptions: Array.isArray(subs?.subscriptions) ? subs.subscriptions : [],
10361041
pricing: PRICING,
10371042
nowMs,
10381043
piAiPin: piAiVersion(),
10391044
sinceMs, // the same instant the scan cut at -- proration denominates on the requested window
1045+
triggerJoin,
10401046
});
10411047
return { fold };
10421048
}

0 commit comments

Comments
 (0)