Skip to content

Commit 80d2edb

Browse files
committed
explainers: make findings attributable, and keep confidence 1.0 structural
- Layer violations cite the dependency fact and the raw import target, not only the importing file. The diff matches fact names and edge endpoints, so file-only evidence never attributed and every new violation landed in the incidental bucket, which the gate does not grade. - Unused-route findings key on the route fact name and carry the file, so they attribute and can be scoped by repository. - common.ModuleDir resolves a repo-prefixed file back to its module, used by layers and the shared module-graph builder. Without it a composed graph resolves to phantom nodes and cycles go undetected. No-op for single-repo. - god-class and the layers pattern clamp at common.MaxHeuristicConfidence. Both compute a saturating score, and 1.0 is what the receipt, the dashboard and query_insights(min_confidence=) read as a structural fact. A cycle is the only claim computed with certainty and the only one that reaches it. - layers and complexity-outliers exclude test code, the last two explainers that did not. layers gates on the importing file, since resolution walks up and would attribute a nested mock to the production layer, and drops test modules from the pattern population. complexity-outliers drops test symbols from the distribution too: a symbol's cyclomatic value is its own, so narrowing the population falsifies nothing retained. Benchmarks re-measured and republished.
1 parent cac0d1a commit 80d2edb

15 files changed

Lines changed: 613 additions & 81 deletions

File tree

ARCHITECTURE.md

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -174,7 +174,7 @@ Three plugin roles drive the middle of the pipeline — **extractors** (source
174174
Explainers turn raw facts into architectural observations. Each insight carries a **confidence** score: `1.0` means it's a structural fact, below `1.0` means it's a heuristic. Every insight is also tagged with the explainer that produced it (`Insight.Source`), and the whole set is retrievable through the **`query_insights`** tool — filter by `explainer`, `repo`, or `min_confidence` — so an agent fetches a finding directly instead of re-deriving it from raw facts or scraping it out of `explore depth=2` / `.enola/insights.json`.
175175

176176
- **Cycles** ([`internal/explainers/cycles`](internal/explainers/cycles/cycles.go)) — finds cyclic module dependencies using **Tarjan's strongly-connected-components algorithm**. A cycle either exists in the import graph or it doesn't, so these land at confidence `1.0`, with every module in the cycle listed as evidence.
177-
- **Layers** ([`internal/explainers/layers`](internal/explainers/layers/layers.go)) — recognizes common architectural shapes by matching module paths against known patterns: **hexagonal** (application / port / adapter / domain / …), **Next.js** (pages / components / hooks / lib / api / …), and **Go-standard** (cmd / internal / pkg / api). Confidence is computed from how much of the codebase matches. It also flags **layer violations** — an inner layer importing an outer one — as lower-confidence heuristic warnings.
177+
- **Layers** ([`internal/explainers/layers`](internal/explainers/layers/layers.go)) — recognizes common architectural shapes by matching module paths against eight known taxonomies: **hexagonal** (application / port / adapter / domain / …), **Go-standard** (cmd / internal / pkg / api), **Next.js** (pages / components / hooks / lib / api / …), **rails-mvc**, **django**, **spring-layered**, **android-clean** and **ios-clean**. Most are gated on a detected framework or language, so only one is ever reported: the most specific match wins, and ties break on confidence. Confidence is computed from how much of the codebase matches — capped below `1.0`, because a directory-name match is a well-supported guess and never a proof. Test modules are excluded from that measurement (a build file's `module_role` outranks what a path looks like), so a test source set cannot vote on the architecture. It also flags **layer violations** — an inner layer importing an outer one — as lower-confidence heuristic warnings.
178178
- **Cross-repo** ([`internal/explainers/crossrepo`](internal/explainers/crossrepo/crossrepo.go)) — summarizes the cross-repo edges found by the linker. Returns nothing for a single-repo snapshot.
179179
- **Coverage** ([`internal/explainers/coverage`](internal/explainers/coverage/coverage.go)) — turns the per-service `edge_coverage` counts the linker records into **coverage-gap** insights: a service with no resolved outbound edges but unresolved outbound call sites is flagged as a blind spot ("appears isolated but…"), distinct from one that is genuinely a leaf. Distinguishes absence of edges from a gap in coverage. Returns nothing for a single-repo snapshot. Surfaced programmatically by the `coverage_report` tool.
180180
- **Unused-routes** ([`internal/explainers/unusedroutes`](internal/explainers/unusedroutes/unusedroutes.go)) — the **server-side inverse** of the cross-repo HTTP linker: it rolls up the `route` facts that *no loaded client calls* (tagged `unmatched_by_clients` during linking — see [Finding unused endpoints](#finding-unused-endpoints)) into one candidate-cleanup insight per service. Deliberately conservative: it only considers repos that actually serve a cross-repo client (an HTTP *provider* — never a frontend's own page routes), skips low-signal generic paths (`/health`, single-segment), and biases toward false negatives. Each insight carries the mandatory caveat that candidates are unused *by the loaded clients only* — consumers outside the snapshot (admin scripts, cron, webhooks, third-party clients, deep links) don't appear, so verify before deleting. Confidence `0.6` (a candidate to review, not a verdict). Returns nothing for a single-repo snapshot.
@@ -642,11 +642,9 @@ Both absolute paths stay in the warning text, and it names which signal decided,
642642

643643
`Status.ExitCode()` is the contract with CI: `0` clean · `1` regression · `2` usage error · `3` incomparable. Precedence is **blocking → usage error → regression**; blocking comes first because when the snapshots were built over different inputs, the inverted-pair remedy ("re-generate") would send the caller down the wrong path. Nothing is hidden by the ordering — every warning is reported regardless of which decided the status.
644644

645-
**Why the policy keys on the explainer rather than on confidence.** The obvious design is "fail at confidence `1.0`, because [Insights](#insights-explainers) says `1.0` is a structural fact and anything below is a flagged heuristic". That does not survive contact with the explainers: `godclass` computes confidence from a fan-in ratio and **clamps it to `1.0`**, so a statistical outlier at twice the threshold presents as a certainty; and `layers` emits an informational `Architecture pattern: <name>` finding whose confidence is the share of the codebase matching the pattern, which can also reach `1.0`. A gate keyed on the number alone would fail builds for a new statistical outlier and for a re-detected pattern after a reorganization.
645+
**Why the policy keys on the explainer rather than on confidence.** Confidence alone is not enough to name what should break a build, even though [Insights](#insights-explainers) guarantees that `1.0` is a structural fact: it says how strong a claim is, not what kind of claim it is. A statistical outlier and a proven cycle are different objects, and only the second is a defect by construction. So the **explainer is the primary filter** (`DefaultFailExplainers = ["cycles"]`) and confidence is a floor applied within it (`DefaultMinConfidence = 1.0`). The floor does real work: `cycles` emits both a true cycle at `1.0` and a "highly coupled module cluster" at `0.4` whose own description calls it "a coupling-density signal, not a defect to break".
646646

647-
So the **explainer is the primary filter** (`DefaultFailExplainers = ["cycles"]`) and confidence is a floor applied within it (`DefaultMinConfidence = 1.0`). The floor still does real work: `cycles` emits both a true cycle at `1.0` and a "highly coupled module cluster" at `0.4` whose own description calls it "a coupling-density signal, not a defect to break".
648-
649-
> The confidence-invariant violation above is a **real inconsistency between the docs and the explainers**, worked around here rather than fixed. Capping `god-class` below `1.0` and reclassifying the `layers` pattern finding as informational would let the gate key on confidence directly — but it changes insight output, so it needs golden regeneration.
647+
> **The two-part filter is what keeps `1.0` meaningful in both directions.** Every explainer that computes rather than proves its confidence clamps at `common.MaxHeuristicConfidence`, strictly below `1.0``god-class`, whose score is a fan-in ratio against a statistical threshold, and `layers`, whose pattern confidence is a coverage share. Both saturate on real repositories, and letting them reach `1.0` would have published a statistical outlier as a structural fact to everything downstream that reads the number: the receipt's heuristic-insight count, the dashboard's structural/candidate split, and `query_insights(min_confidence=…)`. A cycle is the only claim enola computes with certainty, and it is the only one that reaches `1.0`.
650648
651649
**New coupling is reported, never failed.** `diff.Edge` is name-level, so `EdgesAdded` is populated by virtually any change — adding a function that calls another adds edges. A gate firing on that would be switched off within a day. Only module-level and cross-repo coupling deltas are worth escalating, and that needs an edge filter that does not exist yet.
652650

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ New coupling (5):
5757
invoice.Retry --declares--> invoice
5858
```
5959

60-
Two packages here, for a readable example. On a 68,000-fact repository carrying 335 pre-existing findings it behaves identically: it reported the one thing the change introduced, and none of the 335.
60+
Two packages here, for a readable example. On a 68,000-fact repository carrying 268 pre-existing findings it behaves identically: it reported the one thing the change introduced, and none of the 268.
6161

6262
## Set it up
6363

docs/BENCHMARKS.md

Lines changed: 37 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -28,47 +28,47 @@ would have been.
2828

2929
## The corpus
3030

31-
38 repositories, 239,349 source files parsed, 4,211,113 facts carrying 13 distinct
31+
38 repositories, 239,349 source files parsed, 4,211,133 facts carrying 13 distinct
3232
language tags (C, C++, Go, Java, Kotlin, PHP, Python, Ruby, Rust, Swift, TypeScript,
3333
gRPC, OpenAPI). Public open-source only — every row is a repository you can clone and
3434
re-run.
3535

3636
| Repository | Language | Files parsed | Facts | Modules | Routes | Cold | Warm |
3737
|---|---|---|---|---|---|---|---|
38-
| linux | C | 55,399 | 1,892,343 | 4,293 | 0 | 157.6s | 33.9s |
39-
| gitlab | Ruby | 47,711 | 435,020 | 10,484 | 1,848 | 29.1s | 19.4s |
40-
| rust | Rust | 36,090 | 394,970 | 3,759 | 0 | 20.4s | 9.1s |
41-
| shopware | PHP | 13,731 | 218,814 | 4,188 | 724 | 11.1s | 4.9s |
38+
| linux | C | 55,399 | 1,892,343 | 4,293 | 0 | 157.4s | 33.6s |
39+
| gitlab | Ruby | 47,711 | 435,020 | 10,484 | 1,848 | 29.3s | 19.6s |
40+
| rust | Rust | 36,090 | 394,970 | 3,759 | 0 | 20.9s | 8.9s |
41+
| shopware | PHP | 13,731 | 218,814 | 4,188 | 724 | 11.3s | 5.0s |
4242
| grafana | Go | 10,310 | 163,620 | 2,295 | 1,714 | 7.5s | 5.0s |
43-
| thingsboard | Java | 6,371 | 158,987 | 1,164 | 1,403 | 4.8s | 3.1s |
44-
| nextcloud-server | PHP | 6,037 | 101,462 | 1,538 | 12 | 5.9s | 2.0s |
45-
| discourse | Ruby | 10,353 | 93,295 | 1,483 | 1,217 | 6.9s | 4.1s |
46-
| dubbo | Java | 4,351 | 82,550 | 954 | 19 | 4.7s | 1.9s |
47-
| supabase | TypeScript | 6,918 | 70,203 | 1,679 | 83 | 5.6s | 2.4s |
48-
| airflow | Python | 4,061 | 67,992 | 1,261 | 463 | 8.7s | 5.3s |
49-
| deno | Rust | 4,353 | 62,118 | 2,056 | 4 | 7.8s | 2.0s |
43+
| thingsboard | Java | 6,371 | 158,987 | 1,164 | 1,403 | 4.8s | 3.2s |
44+
| nextcloud-server | PHP | 6,037 | 101,462 | 1,538 | 12 | 5.8s | 2.1s |
45+
| discourse | Ruby | 10,353 | 93,295 | 1,483 | 1,217 | 7.1s | 4.2s |
46+
| dubbo | Java | 4,351 | 82,550 | 954 | 19 | 2.3s | 1.8s |
47+
| supabase | TypeScript | 6,918 | 70,203 | 1,679 | 83 | 5.8s | 2.5s |
48+
| airflow | Python | 4,061 | 67,992 | 1,261 | 463 | 8.7s | 5.4s |
49+
| deno | Rust | 4,353 | 62,118 | 2,056 | 4 | 5.8s | 2.0s |
5050
| superset | Python | 3,834 | 51,645 | 984 | 302 | 4.8s | 2.7s |
5151
| cal.com | TypeScript | 4,596 | 48,637 | 1,453 | 270 | 3.0s | 1.5s |
52-
| dbt-core | Rust | 1,371 | 46,935 | 321 | 202 | 2.9s | 1.0s |
53-
| wordpress | PHP | 2,840 | 45,681 | 441 | 6,671 | 5.5s | 1.2s |
52+
| dbt-core | Rust | 1,371 | 46,935 | 321 | 202 | 3.0s | 1.1s |
53+
| wordpress | PHP | 2,840 | 45,681 | 441 | 6,671 | 5.6s | 1.1s |
5454
| gmsh | C++ | 1,679 | 41,348 | 86 | 0 | 2.9s | 0.8s |
5555
| chatwoot | Ruby | 3,643 | 36,046 | 863 | 744 | 2.1s | 1.3s |
56-
| gitea | Go | 2,210 | 34,088 | 395 | 758 | 1.5s | 1.1s |
56+
| gitea | Go | 2,210 | 34,088 | 395 | 758 | 1.5s | 1.2s |
5757
| saleor | Python | 2,473 | 30,052 | 202 | 4 | 3.1s | 2.1s |
5858
| flarum | PHP | 2,702 | 28,300 | 764 | 11 | 1.3s | 0.7s |
5959
| cognee | Python | 1,489 | 16,909 | 394 | 214 | 1.7s | 0.8s |
6060
| cognee-rs | Rust | 846 | 15,827 | 124 | 77 | 1.1s | 0.4s |
6161
| tokio | Rust | 780 | 14,450 | 99 | 0 | 0.8s | 0.3s |
6262
| crates-io | Rust | 878 | 12,451 | 214 | 96 | 0.8s | 0.4s |
63-
| solidus | Ruby | 1,766 | 10,009 | 383 | 171 | 0.9s | 0.7s |
63+
| solidus | Ruby | 1,766 | 10,009 | 383 | 171 | 0.9s | 0.6s |
6464
| excalidraw | TypeScript | 526 | 8,738 | 85 | 3 | 0.9s | 0.4s |
65-
| isowords | Swift | 382 | 5,615 | 160 | 2 | 0.4s | 0.2s |
65+
| isowords | Swift | 382 | 5,615 | 160 | 2 | 0.5s | 0.2s |
6666
| nowinandroid | Kotlin | 312 | 5,110 | 123 | 4 | 0.4s | 0.3s |
6767
| nextcloud-collectives | PHP | 383 | 4,966 | 70 | 1 | 0.5s | 0.2s |
6868
| getdp | C++ | 169 | 4,562 | 9 | 0 | 0.8s | 0.2s |
69-
| enola | Go | 158 | 3,506 | 56 | 6 | 0.8s | 0.3s |
69+
| enola | Go | 158 | 3,526 | 56 | 6 | 0.3s | 0.2s |
7070
| elk | Vue | 380 | 2,556 | 79 | 56 | 0.2s | 0.1s |
71-
| nextcloud-contacts | PHP | 170 | 1,706 | 51 | 1 | 0.3s | 0.2s |
71+
| nextcloud-contacts | PHP | 170 | 1,706 | 51 | 1 | 0.3s | 0.1s |
7272
| grpc-web-example | Go + gRPC + TS | 12 | 293 | 8 | 5 | 0.1s | 0.1s |
7373
| sveltekit-realworld | Svelte | 42 | 195 | 14 | 13 | 0.1s | 0.1s |
7474
| cachet | PHP | 21 | 101 | 11 | 0 | 0.1s | 0.1s |
@@ -88,7 +88,7 @@ three. Running cold then warm is the point: it tests that a cached run and a
8888
from-scratch run agree, not merely that the same code path repeats itself.
8989

9090
> **38 of 38 repositories produced a byte-identical `snapshot_id` and a
91-
> byte-identical `facts.jsonl` across all three runs — 114 runs, 4,211,113 facts,
91+
> byte-identical `facts.jsonl` across all three runs — 114 runs, 4,211,133 facts,
9292
> zero drift.** `insights.json` is byte-stable on all 38 as well.
9393
9494
This is the property everything else rests on. `snapshot_id` is
@@ -111,15 +111,15 @@ delete. The cycle case adds two new modules that import each other.
111111

112112
| Repository | Language | Pre-existing findings | No change | Benign addition | Injected cycle | Reverted |
113113
|---|---|---|---|---|---|---|
114-
| superset | Python | 347 | PASS · +0 facts | PASS · +2 facts | **FAIL · 1 regression** | PASS · +0 |
114+
| superset | Python | 133 | PASS · +0 facts | PASS · +2 facts | **FAIL · 1 regression** | PASS · +0 |
115115
| gitea | Go | 159 | PASS · +0 facts | PASS · +2 facts | **FAIL · 1 regression** | PASS · +0 |
116-
| enola | Go | 144 | PASS · +0 facts | PASS · +2 facts | **FAIL · 1 regression** | PASS · +0 |
117-
| cognee | Python | 123 | PASS · +0 facts | PASS · +2 facts | **FAIL · 1 regression** | PASS · +0 |
116+
| enola | Go | 145 | PASS · +0 facts | PASS · +2 facts | **FAIL · 1 regression** | PASS · +0 |
117+
| cognee | Python | 121 | PASS · +0 facts | PASS · +2 facts | **FAIL · 1 regression** | PASS · +0 |
118118
| excalidraw | TypeScript | 111 | PASS · +0 facts | PASS · +2 facts | **FAIL · 1 regression** | PASS · +0 |
119119
| chatwoot | Ruby | 104 | PASS · +0 facts | PASS · +3 facts | **FAIL · 1 regression** | PASS · +0 |
120120
| solidus | Ruby | 72 | PASS · +0 facts | PASS · +3 facts | **FAIL · 1 regression** | PASS · +0 |
121121
| crates-io | Rust | 65 | PASS · +0 facts | PASS · +2 facts | **FAIL · 1 regression** | PASS · +0 |
122-
| nowinandroid | Kotlin | 40 | PASS · +0 facts | PASS · +2 facts | **FAIL · 1 regression** | PASS · +0 |
122+
| nowinandroid | Kotlin | 37 | PASS · +0 facts | PASS · +2 facts | **FAIL · 1 regression** | PASS · +0 |
123123
| elk | TypeScript | 26 | PASS · +0 facts | PASS · +2 facts | **FAIL · 1 regression** | PASS · +0 |
124124
| sveltekit-realworld | TypeScript | 1 | PASS · +0 facts | PASS · +2 facts | **FAIL · 1 regression** | PASS · +0 |
125125
| cachet | PHP | 0 | PASS · +0 facts | PASS · +3 facts | **FAIL · 1 regression** | PASS · +0 |
@@ -131,8 +131,8 @@ Read the columns as four separate claims, all of which hold on all twelve:
131131
columns mean something.
132132
- **Benign addition → PASS**, with the delta naming exactly the 2–3 facts added.
133133
A new leaf module is not a regression, and reporting it as one is how gates die.
134-
- **Injected cycle → FAIL, exactly 1 regression** — out of **1,192 pre-existing
135-
findings across these repositories**, up to 347 in a single one. None of them was
134+
- **Injected cycle → FAIL, exactly 1 regression** — out of **974 pre-existing
135+
findings across these repositories**, up to 159 in a single one. None of them was
136136
repeated. The ratchet holds.
137137
- **Reverted → PASS again**, +0 facts. The verdict is a function of the tree, not
138138
of history.
@@ -150,19 +150,19 @@ which is exactly the property that makes the no-change column worth reading.
150150

151151
### Why only cycles fail
152152

153-
Across the corpus enola produced **24,648 findings**. Broken down:
153+
Across the corpus enola produced **24,012 findings**. Broken down:
154154

155155
| Explainer | Findings | Class |
156156
|---|---|---|
157-
| hotspots | 20,891 | statistical outlier |
158-
| layers | 953 | heuristic |
157+
| hotspots | 20,625 | statistical outlier |
158+
| layers | 585 | heuristic |
159159
| **cycles** | **887** | **structural fact — the only one that fails a build** |
160160
| god-class | 714 | statistical outlier |
161-
| complexity-outliers | 491 | statistical outlier |
161+
| complexity-outliers | 489 | statistical outlier |
162162
| exported-surface | 442 | candidate |
163163
| dependency-depth | 270 | statistical outlier |
164164

165-
3.6% of findings are eligible to fail a build. The other 96.4% are reported and let
165+
3.7% of findings are eligible to fail a build. The other 96.3% are reported and let
166166
you through. That ratio is the design, not an accident: a gate that fails on one
167167
thing, and says which, is a gate people leave switched on.
168168

@@ -249,15 +249,15 @@ so the demonstration proves its own limit in the same run.
249249

250250
| | |
251251
|---|---|
252-
| Largest repository indexed | **Linux kernel** — 55,399 files, **1,892,343 facts**, 157.6s cold / 33.9s warm |
253-
| Largest Ruby | GitLab — 47,711 files, 435,020 facts, 29.1s / 19.4s |
254-
| Largest Rust | rust-lang/rust — 36,090 files, 394,970 facts, 20.4s / 9.1s |
252+
| Largest repository indexed | **Linux kernel** — 55,399 files, **1,892,343 facts**, 157.4s cold / 33.6s warm |
253+
| Largest Ruby | GitLab — 47,711 files, 435,020 facts, 29.3s / 19.6s |
254+
| Largest Rust | rust-lang/rust — 36,090 files, 394,970 facts, 20.9s / 8.9s |
255255
| Largest Go | Grafana — 10,310 files, 163,620 facts, 7.5s / 5.0s |
256256
| Throughput | 7,500–35,000 facts/sec depending on language |
257257
| Parse errors, all 38 repositories | **0** |
258258
| Memory, Linux kernel | heap peaked well inside a laptop's budget; no repository required tuning |
259259

260-
Warm runs are 1.33×–5.22× faster than cold (over the 21 repositories whose cold run
260+
Warm runs are 1.28×–4.97× faster than cold (over the 28 repositories whose cold run
261261
exceeds 0.5s; below that the timing is noise), from the per-file content-hash cache
262262
in `snapshot.meta.json`. Speed is not a claim enola competes on; these numbers are here
263263
to establish that the graph the other four sections rely on can actually be built on
@@ -277,8 +277,8 @@ file-get-contents 9 · hono 7 · gorilla/mux 5 · retrofit 4 · django 4 · expr
277277
urlsession 2
278278
```
279279

280-
Fact kinds: 3,065,486 symbols · 979,468 dependencies · 61,328 file refs ·
281-
42,546 modules · 41,753 test refs · 17,098 routes · 3,434 storage.
280+
Fact kinds: 3,065,499 symbols · 979,474 dependencies · 61,328 file refs ·
281+
42,546 modules · 41,754 test refs · 17,098 routes · 3,434 storage.
282282

283283
Two rows are worth reading carefully rather than as a score. **django 4** counts
284284
routes, not coverage: Saleor is GraphQL-first, so its REST surface really is that

0 commit comments

Comments
 (0)