feat(estimate): add sentinel estimate and per-run model instrumentation

README.md

@@ -49,7 +49,7 @@ Full documentation is available online at **[docs.nanocollective.org](https://do
 - **[Configuration](docs/configuration/index.md)** — The `sentinel.yaml` reference
 - **[Workflow](docs/workflow/index.md)** — The scheduled audit, run modes, and the execution model
 - **[Findings & Issues](docs/findings/index.md)** — The severity model, issue filing, dedup, and suppression
-- **[CLI](docs/cli/index.md)** — `init` and `run` command reference
+- **[CLI](docs/cli/index.md)** — `init`, `run`, and `estimate` command reference
 - **[Community](docs/community.md)** — Contributing, Discord, and how to help
 
 ## What Sentinel is not (in v1)

docs/cli/index.md

@@ -6,7 +6,7 @@ sidebar_order: 7
 
 # CLI
 
-The `@nanocollective/sentinel` package is both the scaffolder and the runtime. It exposes two commands: `init` scaffolds a config repo, and `run` performs an audit. The scheduled [workflow](../workflow/index.md) invokes `run` under the hood, and you can invoke either directly.
+The `@nanocollective/sentinel` package is both the scaffolder and the runtime. It exposes three commands: `init` scaffolds a config repo, `run` performs an audit, and `estimate` sizes an audit before you run it. The scheduled [workflow](../workflow/index.md) invokes `run` under the hood, and you can invoke any of them directly.
 
 ```bash
 npx @nanocollective/sentinel <command> [options]
@@ -57,3 +57,43 @@ npx @nanocollective/sentinel run \
 A local `run` **writes findings to a Markdown file and never files issues** — issue filing needs a GitHub token, which is only present in the Actions path. This makes local `run` the [calibration path](../rule-packs/authoring.md#calibrate-before-you-file) for pack authors: iterate on a pack against a real repo, read the Markdown, adjust, repeat, all without touching anyone's issue tracker.
 
 The same validator, dedup logic, and findings model apply in both contexts, so what you see locally is what the workflow will produce.
+
+## `estimate`
+
+Sizes an audit **before** it runs: how many repositories and rule packs are in scope, how many files they put in front of the model, and roughly how many model requests, tokens, and minutes that costs. It runs no model and files no issues — `--clone` is the one flag that writes anything, checking out missing repos. Useful when you are about to point Sentinel at a dozen more repositories, or adding a pack to every target and want to know what that does to the nightly window.
+
+```bash
+npx @nanocollective/sentinel estimate
+```
+
+```markdown
+# Sentinel audit estimate
+
+- **Repositories:** 18
+- **Rule packs:** 5
+- **Files:** 1,204
+- **Estimated AI requests:** ~420
+- **Estimated tokens:** ~3.8M
+- **Estimated runtime:** ~14 minute(s)
+
+Calibrated from the last 6 run record(s).
+```
+
+| Flag | Description |
+| --- | --- |
+| `--config <path>` | Path to `sentinel.yaml`. Defaults to `./sentinel.yaml`. |
+| `--packs-dir <path>` | Rule packs directory. Defaults to `rule-packs/` beside the config. |
+| `--workspace <path>` | Where the target repos are checked out. Defaults to `.`. |
+| `--records-dir <path>` | Run records to calibrate from. Defaults to `runs`. |
+| `--clone` | Check out any target repo not already present in the workspace. |
+| `--output <path>` | Write the Markdown estimate here. Defaults to stdout. |
+
+### How the figures are produced
+
+The token figure is **measured, not guessed**: `estimate` assembles the same prompts the audit would send — the pack body, the reporting contract, and the source files scoped by each pack's `applies_to.paths` — and counts them. What varies between installs is the per-request cost, so the request, token, and runtime figures are calibrated from the run records the last ten runs committed. Every run is instrumented for this: it records how long each pack pass took, how many model requests it made (auto-fix retries included), and the tokens it sent and received across every attempt — an auto-fix retry resends the prompt and generates a second response, and both are counted.
+
+Runtime is **not** a flat per-request average. A request costs a fixed amount regardless of size plus an amount that tracks prompt size, and both terms are fitted from the records, so sizing a config far larger than anything you have run is not priced as though the prompts stayed the same. When the records cannot separate the two — a single run, or every run the same size — the measured average is split using the proportion the built-in defaults imply.
+
+Until a run has been recorded, the figures fall back to built-in defaults, and the output says so. Treat a first, uncalibrated estimate as an order of magnitude rather than a number to schedule against.
+
+Repositories already checked out under `--workspace` are measured from their real files. Any that are not are counted with zero files and called out in the output, so a partial estimate never reads as the whole picture — pass `--clone` to check the rest out first. A repo that *is* checked out but whose files no pack matches gets a separate warning: nothing needs cloning, but a pack is pointed at a repository it cannot see.

docs/configuration/index.md

@@ -86,6 +86,8 @@ Systematic noise on one repository is handled with an opt-in `sentinel.yaml` pla
 
 Cost scales with repositories × rule packs × schedule frequency. Twenty repos with four packs each, run daily, is eighty model calls a day. Local models keep that cost at zero; cloud models do not. Keep the default configuration modest, lean on local models for the routine passes, and reserve a cloud fallback for the cases that genuinely need it.
 
+To put numbers on a specific config before committing to it, run [`sentinel estimate`](../cli/index.md#estimate) — it reports the requests, tokens, and runtime the config implies, calibrated from your own recorded runs.
+
 ## Observability
 
 Each run commits a run record to the config repo and writes a step summary. A lightweight static dashboard is generated into the config repo's GitHub Pages from those committed records — no database. See [Workflow → observability](../workflow/index.md#observability-and-run-history).

docs/index.md

@@ -41,7 +41,7 @@ npx @nanocollective/sentinel init
 - [Configuration](configuration/index.md) — The `sentinel.yaml` reference
 - [Workflow](workflow/index.md) — The scheduled audit, run modes, and the execution model
 - [Findings & Issues](findings/index.md) — The severity model, issue filing, dedup, and suppression
-- [CLI](cli/index.md) — `init` and `run` command reference
+- [CLI](cli/index.md) — `init`, `run`, and `estimate` command reference
 - [Community](community.md) — Get involved
 
 > Sentinel is in active development toward its v1. These docs describe the v1 design settled in the [Sentinel whitepaper](/collective/whitepapers/sentinel). Where a feature is planned rather than shipped, the docs say so.

docs/workflow/index.md

@@ -64,7 +64,7 @@ Sentinel is not a model and does not train one. It uses whichever Nanocoder-conf
 There is no database. Run history is:
 
 - **Step summary** — the immediate run's result, in the Actions run.
-- **Run record** — a committed record per run, in the config repo. This is the durable store.
+- **Run record** — a committed record per run, in the config repo. This is the durable store. Alongside the findings it carries what the run cost the model: requests made, wall-clock time, and tokens sent and received. [`sentinel estimate`](../cli/index.md#estimate) reads those back to calibrate its figures.
 - **Dashboard** — a lightweight static site generated into the config repo's GitHub Pages from the committed run records. The read-side surface for trends: per-pack hit rates, findings over time, and aggregate model cost per run.
 
 ## Triggers in v1

source/cli.ts

@@ -3,8 +3,9 @@
 /**
  * Sentinel CLI entry point.
  *
- *   sentinel init   scaffold a configuration repo
- *   sentinel run    perform an audit pass (not yet implemented)
+ *   sentinel init      scaffold a configuration repo
+ *   sentinel run       perform an audit pass
+ *   sentinel estimate  size an audit before running it
  *
  * The init command's logic lives in ./init; this file is the interactive glue
  * (prompting, printing) and is excluded from coverage.
@@ -30,6 +31,7 @@ import {buildRunRecord, recordFilename} from './observe/record.js';
 import type {RunMode, RunRecord} from './observe/types.js';
 import {nanocoderRunner} from './orchestrator/nanocoder-runner.js';
 import {prepareRepo} from './run/clone.js';
+import {estimateRun, renderEstimate} from './run/estimate.js';
 import {renderPreview} from './run/preview.js';
 import {ghRepoLister} from './run/repo-lister.js';
 import {renderReport} from './run/report.js';
@@ -39,8 +41,9 @@ import {fsPackLoader, fsRepoFiles} from './run/sources.js';
 const USAGE = `sentinel <command>
 
 Commands:
-  init    Scaffold a Sentinel configuration into the current repository
-  run     Perform an audit pass against a rule pack and a repository
+  init      Scaffold a Sentinel configuration into the current repository
+  run       Perform an audit pass against a rule pack and a repository
+  estimate  Size an audit — requests, tokens, runtime — without running it
 
 Run 'sentinel <command> --help' for command-specific options.`;
 
@@ -186,20 +189,26 @@ function writeRunRecord(record: RunRecord, recordsDir: string): void {
 	console.log(`Wrote run record to ${path}`);
 }
 
-function writeDashboard(recordsDir: string, dashboardDir: string): void {
+function readRunRecords(recordsDir: string): RunRecord[] {
 	const records: RunRecord[] = [];
-	if (existsSync(recordsDir)) {
-		for (const name of readdirSync(recordsDir)) {
-			if (!name.endsWith('.json')) {
-				continue;
-			}
-			try {
-				records.push(JSON.parse(readFileSync(join(recordsDir, name), 'utf8')));
-			} catch {
-				// Skip a malformed record rather than fail the whole dashboard.
-			}
+	if (!existsSync(recordsDir)) {
+		return records;
+	}
+	for (const name of readdirSync(recordsDir)) {
+		if (!name.endsWith('.json')) {
+			continue;
+		}
+		try {
+			records.push(JSON.parse(readFileSync(join(recordsDir, name), 'utf8')));
+		} catch {
+			// Skip a malformed record rather than fail the whole read.
 		}
 	}
+	return records;
+}
+
+function writeDashboard(recordsDir: string, dashboardDir: string): void {
+	const records = readRunRecords(recordsDir);
 	mkdirSync(dashboardDir, {recursive: true});
 	const path = join(dashboardDir, 'index.html');
 	writeFileSync(path, renderDashboard(records));
@@ -326,6 +335,71 @@ async function runRun(argv: string[]): Promise<number> {
 	return 0;
 }
 
+const ESTIMATE_USAGE = `sentinel estimate [options]
+
+Size an audit before running it: repositories, rule packs, files, model
+requests, tokens, and wall-clock runtime. Runs no model and files no issues;
+--clone is the one flag that writes anything, checking out missing repos.
+
+Figures are calibrated from the committed run records when any exist, so they
+sharpen against your own hardware and model. Repos already checked out under
+the workspace are measured from their real files; pass --clone to check out the
+rest.
+
+Options:
+  --config <path>       Path to sentinel.yaml (default ./sentinel.yaml)
+  --packs-dir <path>    Rule packs directory (default ./rule-packs)
+  --workspace <path>    Where target repos are checked out (default .)
+  --records-dir <path>  Run records to calibrate from (default runs)
+  --clone               Check out any target repo not already present
+  --output <path>       Write the Markdown estimate here (default stdout)`;
+
+async function runEstimate(argv: string[]): Promise<number> {
+	if (argv.includes('--help') || argv.includes('-h')) {
+		console.log(ESTIMATE_USAGE);
+		return 0;
+	}
+	const flags = flagMap(argv);
+
+	const configPath = flagStr(flags, 'config') ?? 'sentinel.yaml';
+	const parsed = parseConfig(readFileSync(configPath, 'utf8'));
+	if (!parsed.valid || !parsed.config) {
+		for (const error of parsed.errors) {
+			console.error(`config error — ${error.field}: ${error.message}`);
+		}
+		return 1;
+	}
+
+	const estimate = await estimateRun(
+		parsed.config,
+		{
+			files: fsRepoFiles,
+			packs: fsPackLoader,
+			repoLister: ghRepoLister,
+			cloneRepo: flags.get('clone') === true ? prepareRepo : undefined,
+			records: readRunRecords(flagStr(flags, 'records-dir') ?? 'runs'),
+		},
+		{
+			workspaceDir: flagStr(flags, 'workspace') ?? '.',
+			packsDir:
+				flagStr(flags, 'packs-dir') ?? join(dirname(configPath), 'rule-packs'),
+		},
+	);
+
+	const output = flagStr(flags, 'output');
+	writeReport(renderEstimate(estimate), output);
+	// The estimate already renders these as caveats, so repeat them on stderr
+	// only when the report went to a file and nobody would otherwise see them.
+	if (output) {
+		for (const error of estimate.targetErrors) {
+			console.error(`target: ${error}`);
+		}
+	}
+	// Deliberately 0 even when targets failed: estimate is advisory, and a
+	// partial estimate is still useful — the caveats say what is missing.
+	return 0;
+}
+
 async function main(argv: string[]): Promise<number> {
 	const [command, ...rest] = argv;
 
@@ -334,6 +408,8 @@ async function main(argv: string[]): Promise<number> {
 			return runInit(rest);
 		case 'run':
 			return runRun(rest);
+		case 'estimate':
+			return runEstimate(rest);
 		case undefined:
 		case '--help':
 		case '-h':

source/index.ts

@@ -103,6 +103,7 @@ export type {
 	RepoRunRecord,
 	RunMode,
 	RunRecord,
+	RunUsage,
 	SeverityCounts,
 } from './observe/types.js';
 export {runAudit} from './orchestrator/audit.js';
@@ -148,6 +149,17 @@ export {
 	type PrepareResult,
 	prepareRepo,
 } from './run/clone.js';
+export {
+	type AuditEstimate,
+	type Calibration,
+	calibrate,
+	type EstimateDeps,
+	type EstimateOptions,
+	estimateRun,
+	estimateTokens,
+	type RepoEstimate,
+	renderEstimate,
+} from './run/estimate.js';
 export {
 	type ExpandResult,
 	expandTargets,
@@ -175,13 +187,19 @@ export {
 	runFromConfig,
 	runLocal,
 } from './run/run.js';
-export {isEnabledPackPath, unionPatterns} from './run/select.js';
+export {
+	isEnabledPackPath,
+	type SelectedPacks,
+	selectPacks,
+	unionPatterns,
+} from './run/select.js';
 export {fsPackLoader, fsRepoFiles} from './run/sources.js';
 export type {
 	LoadedPacks,
 	PackLoadError,
 	PackLoader,
 	PackOutcome,
+	PackUsage,
 	RepoFiles,
 	RepoOutcome,
 	RunOutcome,

source/observe/record.spec.ts

@@ -20,14 +20,19 @@ function finding(severity: Finding['severity']): Finding {
 	};
 }
 
-function pack(findings: Finding[]): PackOutcome {
+function pack(
+	findings: Finding[],
+	overrides: Partial<PackOutcome> = {},
+): PackOutcome {
 	return {
 		pack: 'p',
 		version: '1.0.0',
 		findings,
 		attempts: 1,
 		ok: true,
 		errors: [],
+		usage: {durationMs: 1000, promptTokens: 400, outputTokens: 50},
+		...overrides,
 	};
 }
 
@@ -107,6 +112,41 @@ test('carries target errors', t => {
 	t.deepEqual(record.targetErrors, ['boom']);
 });
 
+test('totals the measured model usage across every pack pass', t => {
+	const r = report({
+		outcome: {
+			repos: [
+				repo('org/a', [
+					pack([], {
+						attempts: 2,
+						usage: {durationMs: 3000, promptTokens: 800, outputTokens: 90},
+					}),
+					pack([]),
+				]),
+				repo('org/b', [pack([])]),
+			],
+		},
+	});
+	const record = buildRunRecord(r, TS, 'audit-only');
+	// 2 attempts on the first pass, 1 on each of the others.
+	t.deepEqual(record.totals.usage, {
+		requests: 4,
+		durationMs: 5000,
+		promptTokens: 1600,
+		outputTokens: 190,
+	});
+});
+
+test('records zeroed usage when nothing was audited', t => {
+	const record = buildRunRecord(report(), TS, 'dry-run');
+	t.deepEqual(record.totals.usage, {
+		requests: 0,
+		durationMs: 0,
+		promptTokens: 0,
+		outputTokens: 0,
+	});
+});
+
 test('recordFilename is filesystem-safe', t => {
 	t.is(recordFilename(TS), '2026-07-21T06-00-00-000Z.json');
 	t.false(recordFilename(TS).includes(':'));

source/observe/record.ts

@@ -11,6 +11,7 @@ import type {
 	RepoRunRecord,
 	RunMode,
 	RunRecord,
+	RunUsage,
 	SeverityCounts,
 } from './types.js';
 
@@ -57,6 +58,21 @@ export function buildRunRecord(
 		totalFindings += repo.findings;
 	}
 
+	const usage: RunUsage = {
+		requests: 0,
+		durationMs: 0,
+		promptTokens: 0,
+		outputTokens: 0,
+	};
+	for (const repo of report.outcome.repos) {
+		for (const pack of repo.packs) {
+			usage.requests += pack.attempts;
+			usage.durationMs += pack.usage.durationMs;
+			usage.promptTokens += pack.usage.promptTokens;
+			usage.outputTokens += pack.usage.outputTokens;
+		}
+	}
+
 	const record: RunRecord = {
 		timestamp,
 		mode,
@@ -65,6 +81,7 @@ export function buildRunRecord(
 			repos: repos.length,
 			findings: totalFindings,
 			bySeverity: totalsBySeverity,
+			usage,
 		},
 		targetErrors: report.targetErrors,
 	};