Skip to content

Commit 00c918c

Browse files
committed
feat(check): grade architectural change from the CLI, not only over MCP
The delta that diff_snapshot serves to an agent could not gate anything. internal/diff had a complete engine — comparability, structural-cause attribution, incidental-drift separation, renderers — reachable only from an MCP tool call, so closing the loop depended on the agent remembering to ask. Enforcement was a nudge in a tool response. Add pkg/check: a pure Evaluate(*diff.SnapshotDiff, Policy) Verdict, and an `enola check` subcommand that exits 0 clean / 1 regression / 2 could-not-run / 3 declined. 3 is deliberately not 1: when two snapshots were built over different inputs the delta describes how they were produced, not what was edited, and reporting that as a failing change would be a lie. Comparability gains a typed Kind set. The existing Comparable bool spans everything from "different repositories" to "the baseline is four days old"; a reader can weigh that from the prose, a gate cannot, and consuming the bool would have made every stale baseline a hard refusal — contrary to the deliberate design of staleBaselineDays. Stale now warns, says what it means for the delta, and still grades. Warnings keeps its type and JSON shape, so no existing consumer changes. Policy keys on the explainer, not on confidence, because the documented "1.0 is a structural fact" invariant is not enforced: god-class clamps a fan-in ratio to 1.0 and layers emits an informational pattern finding that can reach it. Gating on the number would fail builds for a new statistical outlier and for a re-detected pattern. Default is cycles only; confidence is a floor within it, which still screens out the 0.4 coupling-density finding the cycles explainer also emits. Also: - baseline pin|show|clear, with pin snapshotting first so a days-old snapshot is not frozen as "the state before my change" - path arguments resolve as directory=repo, file=config everywhere; a directory previously fell through to config.Load, warned, and silently analysed the working directory instead - unrecognized arguments are rejected rather than absorbed as config paths, which had made every typo a silent wrong action - inverted-pair detection requires a strictly negative gap; GeneratedAt is second-resolution, so pin-then-check inside one second is simultaneous, not inverted, and used to exit non-zero on a clean tree - the verdict reports what moved, not just how much: per-kind tallies, named facts with file:line, edges broken down by relation with the mechanical declares link sorted last - baseline selector resolution shared between the CLI and the MCP tools - pre-commit hook and CI workflow examples
1 parent a7d777a commit 00c918c

18 files changed

Lines changed: 2472 additions & 57 deletions

File tree

.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,11 @@
88
*.so
99
*.dylib
1010

11+
# The binary produced by `go build -o enola ./cmd/enola` (the command CONTRIBUTING.md
12+
# gives). Leading slash on purpose: an unanchored `enola` would also match the
13+
# cmd/enola/ DIRECTORY, silently hiding every new source file added there.
14+
/enola
15+
1116
# Test binary, built with `go test -c`
1217
*.test
1318

ARCHITECTURE.md

Lines changed: 40 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -223,6 +223,8 @@ The shared module-graph construction and statistical-outlier helpers used by sev
223223

224224
`--help`, `--list` and `--status` are served from two public packages, for the same reason as `pkg/explain`: a wrapper binary must be able to print *its* version of each without restating the shared text. Both extension points are plain data, so nothing a wrapper adds is known here.
225225

226+
**Subcommands vs. flags.** `check`, `baseline` and `upgrade` are dispatched in [`cmd/enola/main.go`](cmd/enola/main.go) *before* the flag loop, and each owns a `flag.FlagSet`. The loop itself is an exact-match switch over `os.Args` and cannot parse `--flag=value`, so anything taking its own flags has to be a subcommand. The loop's `default:` case **validates** rather than absorbs: a directory is a repository, an existing file is a config, and anything else is rejected. It used to pass every unrecognized token through as a config path, and because an unreadable config is only a *warning* inside `config.Load` — by design, so a missing `mcp-arch.yaml` falls back to defaults — every typo inherited that leniency and became a silent wrong action (`enola chekc .` started an MCP server; `enola --generate /does/not/exist.yaml` snapshotted the working directory). The default-path fallback is unchanged; only an *explicitly named* path that does not exist is now fatal.
227+
226228
**[`pkg/cli`](pkg/cli)** renders what a binary prints about itself.
227229

228230
- `OSSTools()` is the `--list` catalogue: name plus a one-line summary. It is hand-written on purpose — the descriptions registered with `mcp.AddTool` are multi-paragraph agent prompts, unusable in a terminal. `TestE2E_ToolCatalogueMatchesRegisteredTools` ([`internal/server/e2e_test.go`](internal/server/e2e_test.go)) asserts set equality against the tools the running server actually registers, so the two cannot drift. `RenderToolList(ToolListSpec)` renders it; a wrapper passes its own tools in `Extra` (or an unlock note via `ExtraLocked`/`LockedNote`), and with a zero spec the output never mentions that a wrapper exists.
@@ -557,6 +559,8 @@ Per-service edge-coverage report, so you can tell a genuinely isolated service f
557559

558560
Pins the current snapshot as a diff baseline by copying the snapshot artifacts (`facts.jsonl`, `insights.json`, `snapshot.meta.json`) into `.enola/baseline/`. Call it once at the start of a task, after the first `generate_snapshot`. The pinned baseline **survives subsequent `generate_snapshot` runs**, so it stays valid across several rounds of edits — unlike the auto-rotated `.enola/previous/`, which only ever holds the immediately-preceding run. Takes no parameters.
559561

562+
> The CLI equivalent, `enola baseline pin [repo|config]`, **snapshots first and then pins** — there is no separate `--generate` step. The MCP tool cannot do that (the agent has just generated, and re-generating inside the tool would be surprising), but from a shell "pin a baseline of this repo" is one intent, and pinning whatever happened to be on disk risks freezing a days-old snapshot as "the state before my change" — precisely the staleness the diff then warns about.
563+
560564
### `diff_snapshot` — "what did my change actually do?"
561565

562566
Computes the architectural **delta** between a baseline snapshot and the current one. This is the verification counterpart to `impact_analysis`: where impact analysis *plans* a change, diff_snapshot *confirms* it, replacing "re-read the files to check what got built" with a deterministic answer.
@@ -583,7 +587,42 @@ The typical loop is `generate_snapshot → set_baseline → edit → generate_sn
583587
| `output_mode` | `summary` (default — headline regressions/improvements + structural tally) → `compact` (adds finding descriptions, evidence, and the changed edges/facts) → `full` (complete JSON). |
584588
| `max_tokens` | Optional hard cap on output size. |
585589

586-
The engine lives in [`internal/diff`](internal/diff/diff.go) (pure `Compute` + deterministic renderers) and is re-exported for out-of-module use via [`pkg/diff`](pkg/diff/diff.go); baseline persistence and the on-disk loader live in [`internal/engine/baseline.go`](internal/engine/baseline.go). `diff_snapshot` also runs the **comparability guard** (`diff.CompareMeta`): it reads the receipt fields on both snapshots' metadata and warns, above the delta, when they were not generated over equivalent inputs.
590+
The engine lives in [`internal/diff`](internal/diff/diff.go) (pure `Compute` + deterministic renderers) and is re-exported for out-of-module use via [`pkg/diff`](pkg/diff/diff.go); baseline persistence, the on-disk loader and the shared selector resolution (`ResolveBaselineDir`) live in [`internal/engine/baseline.go`](internal/engine/baseline.go). `diff_snapshot` also runs the **comparability guard** (`diff.CompareMeta`): it reads the receipt fields on both snapshots' metadata and warns, above the delta, when they were not generated over equivalent inputs.
591+
592+
#### Comparability is a spectrum, not a boolean
593+
594+
`Comparability` carries `Comparable bool` (invariant: `Comparable == (len(Warnings) == 0)`), free-text `Warnings`, and a **set of `Kinds`** categorizing them. The kinds exist because the boolean spans everything from *"these are different repositories, the delta is meaningless"* to *"the baseline is four days old, the delta is real but also contains the repo's own drift"*. A human reader can weigh that from the prose; a **gate cannot** — and consuming `Comparable` would turn every stale baseline into a hard refusal, contradicting the deliberate design of `staleBaselineDays`.
595+
596+
| Kind | Raised when | Treated as |
597+
|------|-------------|-----------|
598+
| `different_repo` | the two snapshots are of different repositories | blocking |
599+
| `version_mismatch` | different enola versions (extractor changes read as churn) | blocking |
600+
| `extractor_set` | a language present on one side only | blocking |
601+
| `ignore_globs` | the set of files parsed changed | blocking |
602+
| `unclassified` | contributed via `AddWarning` by a caller that knows something this package cannot (notably `engine.Drift`) | blocking — a gate must **fail closed** on a caveat it cannot categorize |
603+
| `inverted_pair` | the baseline is *newer* than the current snapshot | usage error (concrete remedy: re-generate) |
604+
| `stale_baseline` | the baseline is ≥ 3 days older | **advisory** — warn and still grade |
605+
| `pre_receipt` | the baseline predates snapshot receipts | advisory |
606+
607+
`Kinds` is a **set**, not a per-message list, so `Warnings` keeps its type and JSON shape for every existing consumer (the dashboard, `output_mode='full'`, and out-of-module readers via `pkg/diff`). Callers that know their category should use `AddWarningKind` rather than `AddWarning`.
608+
609+
> Note on timestamps: `GeneratedAt` is RFC3339, i.e. **second** resolution, so a baseline pinned and then diffed inside the same second yields a zero gap. Zero is *simultaneous*, not inverted — `inverted_pair` requires a strictly negative gap. Treating zero as inverted made a no-op check on an untouched repository report "the current snapshot does not contain your change".
610+
611+
---
612+
613+
### The gate (`pkg/check`) — `diff_snapshot` as an exit code
614+
615+
[`pkg/check`](pkg/check/check.go) is a thin, **pure** policy layer over `internal/diff`: `Compute` decides *what changed*, `Evaluate(*diff.SnapshotDiff, Policy) Verdict` decides whether that is allowed to break a build. Same delta plus same policy always yields the same verdict, so a gate is as reproducible as the snapshot underneath it. It backs the `enola check` CLI, and exists as a public package so a wrapper can build a graded verdict on top of it rather than re-deriving one.
616+
617+
`Status.ExitCode()` is the contract with CI: `0` clean · `1` regression · `2` usage error · `3` incomparable. Precedence is **blocking → usage error → regression**; blocking comes first because when the snapshots were built over different inputs, the inverted-pair remedy ("re-generate") would send the caller down the wrong path. Nothing is hidden by the ordering — every warning is reported regardless of which decided the status.
618+
619+
**Why the policy keys on the explainer rather than on confidence.** The obvious design is "fail at confidence `1.0`, because [Insights](#insights-explainers) says `1.0` is a structural fact and anything below is a flagged heuristic". That does not survive contact with the explainers: `godclass` computes confidence from a fan-in ratio and **clamps it to `1.0`**, so a statistical outlier at twice the threshold presents as a certainty; and `layers` emits an informational `Architecture pattern: <name>` finding whose confidence is the share of the codebase matching the pattern, which can also reach `1.0`. A gate keyed on the number alone would fail builds for a new statistical outlier and for a re-detected pattern after a reorganization.
620+
621+
So the **explainer is the primary filter** (`DefaultFailExplainers = ["cycles"]`) and confidence is a floor applied within it (`DefaultMinConfidence = 1.0`). The floor still does real work: `cycles` emits both a true cycle at `1.0` and a "highly coupled module cluster" at `0.4` whose own description calls it "a coupling-density signal, not a defect to break".
622+
623+
> The confidence-invariant violation above is a **real inconsistency between the docs and the explainers**, worked around here rather than fixed. Capping `god-class` below `1.0` and reclassifying the `layers` pattern finding as informational would let the gate key on confidence directly — but it changes insight output, so it needs golden regeneration.
624+
625+
**New coupling is reported, never failed.** `diff.Edge` is name-level, so `EdgesAdded` is populated by virtually any change — adding a function that calls another adds edges. A gate firing on that would be switched off within a day. Only module-level and cross-repo coupling deltas are worth escalating, and that needs an edge filter that does not exist yet.
587626

588627
---
589628

README.md

Lines changed: 81 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -342,6 +342,15 @@ If the diff shows a regression, hand it straight back:
342342
343343
Two prompts, no file re-reading, no review meeting. Repeat until the diff is boring.
344344

345+
**The same loop without an agent.** Everything above is also a shell command, so the check can run in a git hook or CI instead of depending on the agent remembering to ask:
346+
347+
```bash
348+
enola baseline pin # before editing
349+
enola check # after - exits 1 on a structural regression
350+
```
351+
352+
See [The gate - `enola check`](#the-gate---enola-check).
353+
345354
#### 5. Go multi-repo
346355

347356
Generate the first repo, then add the rest with append mode - enola links them into one cross-repo graph:
@@ -608,17 +617,87 @@ For interactive per-module blast-radius queries with configurable depth, see the
608617

609618
Run `enola --help` for the full text. With no flags, enola starts the MCP server on stdio.
610619

620+
Every path argument follows the same rule: **a directory is a repository, a file is a config.** Anything that is neither is rejected rather than silently ignored.
621+
622+
| Command | What it does |
623+
|------|--------------|
624+
| `baseline pin\|show\|clear [repo\|config]` | Manage the diff baseline - the "before" a change is graded against. `pin` snapshots the repository and freezes it (no separate `--generate` needed); `show` reports what the current baseline describes; `clear` removes it. Stored per repository, in that repo's `.enola/baseline`, so several repos each keep their own. |
625+
| `check [flags] [repo\|config]` | **Grade what a change did to the architecture**, and exit with a code CI can act on. Read-only: writes nothing and leaves the baseline in place, so it can be run repeatedly. See [The gate](#the-gate---enola-check). |
626+
| `upgrade` | Download and install the latest release over the running binary. |
627+
611628
| Flag | What it does |
612629
|------|--------------|
613-
| `--generate [config_path]` | Generate a snapshot and exit - no MCP server. Artifacts go to `output.dir` (default `.enola/`). With `repos:` in the config, indexes the whole cluster in one run. |
630+
| `--generate [repo_path\|config_path]` | Generate a snapshot and exit - no MCP server. Artifacts go to `output.dir` (default `.enola/`). With `repos:` in the config, indexes the whole cluster in one run. |
614631
| `--explain [repo_path\|config_path]` | Print the statistics report above and exit. Read-only: nothing is written to `.enola/`. A directory is a repository; a file is a config, so a `repos:` config reports over the whole cluster. |
615632
| `--list` | List the MCP tools this build serves, with one-line summaries. |
616633
| `--status` | List every enola server running right now - PID, repos, uptime, calls, dashboard URL - plus per-tool call counts and an estimate of the reconstruction those calls saved, in time and tokens. |
617634
| `--status --all` | The same usage, broken down per repository. |
618635
| `--no-dashboard` | Start the MCP server without the localhost dashboard. |
619636
| `--version` | Print the build version. |
620637
| `--help`, `-h` | Show usage. |
621-
| `upgrade` | Download and install the latest release over the running binary. |
638+
639+
### The gate - `enola check`
640+
641+
`diff_snapshot` answers "what did my change actually do?" for an agent. `enola check` asks the same question from a shell, and turns the answer into an **exit code** - so the same delta can gate a commit or a CI job with no agent in the loop.
642+
643+
```bash
644+
enola baseline pin /path/to/repo # 1. freeze how it looks now, BEFORE editing
645+
# …make your changes…
646+
enola check /path/to/repo # 2. grade what they did
647+
```
648+
649+
| Exit | Meaning |
650+
|------|---------|
651+
| `0` | **clean** - no structural regression |
652+
| `1` | **regression** - the policy was violated |
653+
| `2` | **error** - the gate could not run (no baseline pinned, bad argument, inverted snapshot pair) |
654+
| `3` | **declined** - the baseline is not comparable, so it refused to grade |
655+
656+
`3` is deliberately not `1`. When the two snapshots were built over different inputs - a different enola version, a different extractor set, changed ignore globs - the delta describes *how they were produced*, not what you edited. Reporting that as a failing change would be a lie, so the gate says it declined and why.
657+
658+
**A stale baseline warns; it never blocks.** Past three days it tells you exactly how stale and what that means (the delta now also contains whatever the repo itself changed in between) - then grades anyway, because a long-lived baseline is a legitimate way to measure a multi-day refactor and only you know which you meant.
659+
660+
**What fails by default is narrow: a newly introduced dependency cycle, and nothing else.** Everything below that is reported, not failed - so a red gate is always real. Widen it per repo:
661+
662+
```bash
663+
enola check --fail-on=cycles,layers --min-confidence=0.8 # also fail on new layer violations
664+
enola check --warn-only # report everything, never fail
665+
enola check --json # machine-readable verdict
666+
enola check --detail # full delta under the verdict
667+
enola check --baseline=previous # compare against the preceding snapshot
668+
enola check --focus=internal/auth # narrow the delta to what you touched
669+
enola check --write # also persist the snapshot (default: read-only)
670+
```
671+
672+
The output names what moved rather than counting it - the added symbols with their `file:line`, the new coupling with its relation kinds, and any finding whose content shifted:
673+
674+
```
675+
FAIL — 1 structural regression introduced.
676+
677+
Regressions (fail):
678+
- [cycles] 1.00 — Cyclic dependency detected (2 modules)
679+
module "pkga" is part of the cycle
680+
681+
What changed
682+
symbols +2
683+
dependencies +1
684+
edges +4 (imports +1, calls +1, declares +2)
685+
686+
Added (3):
687+
symbol pkga.AlphaViaB pkga/a.go:7
688+
symbol pkgb.Helper pkgb/b.go:7
689+
dependency pkga -> example.com/gate/pkgb pkga/a.go:3
690+
691+
New coupling (4):
692+
pkga --imports--> pkgb
693+
pkga.AlphaViaB --calls--> pkgb.Helper
694+
pkga.AlphaViaB --declares--> pkga
695+
pkgb.Helper --declares--> pkgb
696+
```
697+
698+
Lists cap at 12 entries with a `--detail` pointer, and `declares` edges - the mechanical one-per-new-symbol link to their module - always sort last, since they say nothing about what got coupled.
699+
700+
Ready-made wiring: [`examples/hooks/pre-commit`](examples/hooks/pre-commit) (blocks only on exit `1`; a missing or incomparable baseline skips the gate rather than blocking someone over setup they haven't done) and [`examples/ci/architecture-gate.yml`](examples/ci/architecture-gate.yml) (a GitHub Action that pins a baseline from the PR's merge base).
622701

623702
### What it saved you - `--status`
624703

0 commit comments

Comments
 (0)