fix(resolver): stop dropping call edges for inherited-method calls through a known receiver type

Root-caused via B15's investigation into why calm scored below CodeGraph
on file-recall: resolve_sites_to_edges keyed by_name_class lookups on
(callee, EXACT declaring class) with no superclass walk. When a receiver's
static type was known (target_class populated from a tracked binding --
Java's formal_parameter, Go's parameter_declaration, etc.) but the called
method was only inherited from an ancestor class, the exact-key lookup
missed and the edge was dropped entirely -- unlike an unknown-type
receiver, which correctly falls back to an ambiguous fan-out edge.
Knowing more about the receiver's type made calm strictly worse at
finding the edge.

Fixed by falling back to the unscoped by_name lookup, but only when the
receiver's class is itself a symbol this project actually declares
(ctx.by_name.contains_key(cls)) -- so a receiver typed as an unmodeled
external/stdlib type (Rust's HashMap::new()) keeps the original "no
candidates" behavior instead of wrongly fanning out project-wide. Caught
test_type_path_call_resolves_scoped_not_fanned_out regressing on the
first, unguarded attempt at this fix.

Also fixes a benchmark oracle bug found in the same investigation:
ground_truth.py's Java method regex (made modifier-optional 2026-08-18
to catch package-private JUnit methods) accidentally became a
near-universal matcher for control-flow lines ending in a brace
(`if (x.isNew()) {`), causing _looks_like_a_definition to wrongly
exclude real call sites from oracle ground truth.

New regression test: test_java_formal_parameter_resolves_inherited_superclass_method.
Full cargo test -p calm-core --lib (1234) + calm-server --lib (395): all green.
Re-ran B15 with both fixes: calm moved from 68/72 (94.4%, behind CodeGraph's
95.8%) to 72/73 (98.6%, ahead of CodeGraph's 94.5%, within 1.4pt of
Context+'s recall-maximizing ceiling) -- see benchmarks/b15_cross_lang_competitor_ab/README.md's
"Root-cause investigation" section for the full writeup.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Author: Your Name

benchmarks/b12_tier1_tier2_tool_correctness/ground_truth.py

@@ -19,7 +19,7 @@
 from dataclasses import dataclass
 from pathlib import Path
 
-_NOT_A_NAME = {"if", "for", "while", "switch", "catch", "return", "function"}
+_NOT_A_NAME = {"if", "for", "while", "switch", "catch", "return", "function", "synchronized", "try"}
 
 
 @dataclass(frozen=True)
@@ -131,8 +131,29 @@ def _looks_like_a_definition(line_text: str, lang: str) -> bool:
     ~10 different test functions, not one real call anywhere -- and
     `jinja_loader` "call sites" were actually substring hits inside the
     unrelated, longer identifier `create_global_jinja_loader(`. Both are
-    ground-truth bugs this filter (plus the word-boundary fix below) closes."""
-    return any(re.search(pat, line_text) for pat, _kind in PATTERNS[lang])
+    ground-truth bugs this filter (plus the word-boundary fix below) closes.
+
+    2026-08-18 fix (B15 investigation): the 2026-08-18 fix that made Java's
+    method-modifier group optional (see PATTERNS["java"]'s own comment)
+    turned that pattern into an accidental near-universal matcher for ANY
+    control-flow line ending in a brace -- `if (pet.isNew()) {`, `while
+    (x) {`, `for (...) {`, `catch (E e) {`, `synchronized (lock) {` all
+    satisfy "optional modifiers, some word-ish text, NAME(...) {" with
+    NAME captured as the keyword itself ("if"/"while"/...). Every one of
+    those was then wrongly treated as "looks like a redefinition" and
+    excluded from call-site ground truth -- verified live on spring-
+    petclinic: `isNew()`'s real call sites inside `if (pet.isNew() && ...)
+    {`-shaped conditions in Owner.java/PetController.java/PetValidator.java
+    were silently dropped from the oracle, undercounting EVERY tool's
+    (not just CALM's) B15 recall on that row. Reuses the same `_NOT_A_NAME`
+    keyword set `extract_definitions` already filters the captured name
+    against, applied here to the definition-pattern's own captured group
+    instead of just checking "does any pattern match at all"."""
+    for pat, _kind in PATTERNS[lang]:
+        m = re.search(pat, line_text)
+        if m and m.group(1) not in _NOT_A_NAME:
+            return True
+    return False
 
 
 # 2026-08-02 fix: single-line comment marker per language, used by

benchmarks/b15_cross_lang_competitor_ab/README.md

@@ -94,7 +94,7 @@ a hit) — but it's a real, disclosable precision problem, worth weighing agains
 claim on Context+'s own README, and worth reading the raw `contextplus_files` column for, not just
 the recall fraction.
 
-## Results (2026-08-18, `calm` @ `822e238`, N=8 symbols/corpus, single pass)
+## Results (2026-08-18, `calm` @ `0d9aa3a` + the inheritance-fallback fix below, N=8 symbols/corpus, single pass)
 
 File-recall on "who calls this symbol", per language (hit/total oracle files):
 
@@ -105,26 +105,96 @@ File-recall on "who calls this symbol", per language (hit/total oracle files):
 | go | 11/11 (100%) | 10/11 (91%) | **1/11 (9%) — see disclosure below, not a capability claim** | 11/11 (100%) |
 | javascript | 7/8 (88%) | 8/8 (100%) | **0/8 (0%) — see disclosure below, not a capability claim** | 8/8 (100%) |
 | typescript | 11/11 (100%) | 9/11 (82%) | **1/11 (9%) — see disclosure below, not a capability claim** | 11/11 (100%) |
-| java | 21/24 (88%) | 24/24 (100%) | 23/24 (96%) | 24/24 (100%) |
-| **aggregate (java only for Ctxo — see below)** | **68/72 (94.4%)** | **69/72 (95.8%)** | **23/24 (96%)** | **72/72 (100%)** |
+| java | 25/25 (100%) | 24/25 (96%) | 23/25 (92%) | 25/25 (100%) |
+| **aggregate (java only for Ctxo — see below)** | **72/73 (98.6%)** | **69/73 (94.5%)** | **23/25 (92%)** | **73/73 (100%)** |
+
+**This table supersedes the first published run** (calm 68/72 = 94.4%, below CodeGraph's 69/72 =
+95.8%) — see "Root-cause investigation" below for why, and note the java column's *sample itself*
+changed between runs: fixing the oracle bug changed which 8 symbols `sample_symbols` deterministically
+picks (the same symbol names — `getName`/`isNew` — that motivated the investigation are no longer in
+this particular sample, but the fix they drove is independently verified live in the investigation
+section, not just inferred from this table moving).
 
 **Read the "Ctxo go/js/ts: a real, unresolved integration-reliability finding" section below before
 citing the go/js/ts Ctxo numbers for anything** — they are published for transparency (raw data in
 `results.json`), but this benchmark's own investigation could not confirm they measure Ctxo's real
 call-graph quality, so the aggregate row above excludes them and counts only Ctxo's java result
 (the one arm verified end-to-end with real, non-empty query results).
 
-**Reading the rest of the table**: CodeGraph and Context+ both land at or near 100% on every
-language they run on — CodeGraph misses 3 files total (go/1, typescript/2, both same-file or
-private-symbol edge cases, not investigated further here), Context+ misses none in this sample
-(see the CSS false-positive section below for why "0 misses" doesn't mean "flawless"). `calm`'s 4
-misses were spot-checked, not just counted: the javascript one (`User`, `examples/view-locals/
-user.js`) is a real, disclosable gap — the symbol is invoked exclusively via `new User(...)`, and
-CALM's JS/TS call extractor doesn't currently treat a `new`-expression as a call site for the
-constructed class name. The java ones are inheritance/cross-file dispatch cases (`getName`/`isNew`
-defined in a base class, invoked through a subclass instance) — a much harder class of problem for
-any syntactic (non-type-checking) resolver, consistent with this suite's own prior findings on
-Rust `Self::`/method-name collisions.
+**Reading the rest of the table**: `calm`'s only remaining miss across all 6 languages (48 sampled
+symbols) is the javascript `User` case — spot-checked, not just counted: the symbol is invoked
+exclusively via `new User(...)`, and CALM's JS/TS call extractor doesn't currently treat a
+`new`-expression as a call site for the constructed class name (a real, disclosable, separate gap —
+not investigated further here). CodeGraph misses 4 files total (go/1, typescript/2, java/1 — not
+investigated further here); Context+ misses none in this sample (see the CSS false-positive section
+below for why "0 misses" doesn't mean "flawless"). Ctxo's java column moved 23/24→23/25 between runs
+purely because the sample changed size (24→25 oracle files) under the fixed oracle, not because
+Ctxo's own behavior changed.
+
+## Root-cause investigation (2026-08-18): why did `calm` initially score below CodeGraph?
+
+The first published run above showed `calm` (94.4%) narrowly behind CodeGraph (95.8%) and well
+behind Context+ (100%). Investigated end-to-end — not just re-read the numbers — by reproducing
+`calm`'s exact misses against a live corpus, inspecting its `call_edges` table directly, and building
+a minimal 2-class Java repro. Three distinct things were found, not one:
+
+1. **A benchmark-design property, not a bug**: this task measures *recall only*, never precision.
+   Context+ and CodeGraph both lean toward "return more, don't worry about false positives" —
+   Context+'s `get_blast_radius` was caught doing plain substring text matching (see the CSS
+   false-positive section below), which trivially maximizes recall at zero precision cost under this
+   scoring. A recall-only metric structurally favors that strategy over a resolver that tries to stay
+   precise. Not fixed here (would need a precision column, e.g. `len(tool_files - oracle_files)`, to
+   fairly separate "found the real callers" from "returned everything and hoped").
+2. **A real, freshly-introduced oracle bug** (same file this benchmark's own ground truth lives in):
+   the 2026-08-18 fix that made Java's method-definition regex's modifier group optional (to catch
+   package-private JUnit test methods, see `ground_truth.py`'s `PATTERNS["java"]` comment) turned that
+   pattern into an accidental near-universal matcher for control-flow lines ending in a brace —
+   `if (pet.isNew() && ...) {` matches "modifiers?, some text, NAME(...) {" with NAME captured as the
+   keyword "if" itself. `_looks_like_a_definition` then wrongly excluded every such line from call-site
+   ground truth. Verified live: `isNew`'s real oracle should have been 3 files
+   (`Owner.java`/`PetController.java`/`PetValidator.java` — every OTHER tool in this benchmark found
+   all 3) but had collapsed to 1. Fixed in `ground_truth.py` by reusing the same `_NOT_A_NAME` keyword
+   set `extract_definitions` already filters against, applied to the definition pattern's own captured
+   group instead of just "did any pattern match at all".
+3. **A real, previously-undocumented bug in `calm`'s own core resolver** — the one that actually moved
+   the table above. Root-caused to `crates/calm-core/src/indexer/pipeline.rs::resolve_sites_to_edges`:
+   when a call site's receiver has a STATICALLY KNOWN type (`target_class` populated from a tracked
+   binding — Java's `formal_parameter`, Go's `parameter_declaration`, etc.), the code looked up
+   `ctx.by_name_class.get(&(callee, class))` — keyed on the method's *exact declaring class*, with no
+   superclass walk — and on a miss returned **zero candidates**, dropping the call edge entirely,
+   with none of the `ambiguous`-fan-out fallback an UNKNOWN-type receiver already gets. So a call to an
+   *inherited* method (`getName()`/`isNew()` declared on `NamedEntity`/`BaseEntity`, invoked through a
+   `Pet`-typed parameter) silently vanished — while the identical call through a same-type LOCAL
+   variable (untracked, so genuinely "unknown" to the resolver) correctly fell back to `ambiguous`.
+   **Knowing more about the receiver's type made `calm` strictly worse at finding the edge** — the
+   opposite of the intended tiered-confidence design. Verified with a minimal, isolated 2-class Java
+   repro (`Base`/`Sub`/3 receiver shapes) before touching any real code, confirmed as the root cause of
+   3 of `calm`'s 4 original misses via direct `call_edges` inspection, and fixed by falling back to the
+   unscoped `by_name` lookup — but ONLY when `cls` is itself a symbol this project actually declares
+   (`ctx.by_name.contains_key(cls)`), so a receiver typed as an unmodeled external/stdlib type (Rust's
+   `HashMap::new()` with no `HashMap` anywhere in the project) keeps the original "no candidates"
+   behavior rather than wrongly fanning out project-wide — caught live by
+   `test_type_path_call_resolves_scoped_not_fanned_out` regressing when the fallback was first tried
+   unguarded. New regression test:
+   `test_java_formal_parameter_resolves_inherited_superclass_method` (`pipeline.rs`). Full
+   `cargo test -p calm-core --lib` (1234 tests) and `cargo test -p calm-server --lib` (395 tests): all
+   green, 0 regressions. Re-verified live post-fix against the real spring-petclinic corpus: `getName`
+   now correctly includes `Owner.java`/`PetController.java`, `isNew` now correctly includes all 3 real
+   files.
+
+**Net effect**: after both fixes, `calm` moved from 94.4% (behind CodeGraph) to 98.6% (ahead of
+CodeGraph, within 1.4 points of Context+'s recall-maximizing ceiling) — with its one remaining miss
+being the already-known, separately-tracked `new`-expression gap, not the inheritance bug. This is
+also a broader finding than B15 itself: `by_name_class`'s inheritance blindness is generic (any
+language populating `target_class` — Go, Rust, TS, C#, PHP, C/C++ — could hit the same shape), not
+Java-specific; only Java's was live-confirmed here.
+
+**The benchmark-design finding (point 1) also answers a broader question worth stating plainly**:
+this task doesn't exercise `calm`'s actual differentiators at all. "File-recall on who calls X" is
+the one axis where a tool that returns more, unfiltered, structurally cannot lose — it says nothing
+about the pre-edit safety gate, cross-session memory, or token-efficiency tasks B11 already measures
+(and where CodeGraph/Context+ don't compete at all). A benchmark built to showcase `calm`'s strengths
+specifically would weight those differently, not just recall on one call-graph query shape.
 
 ## Ctxo go/js/ts: a real, unresolved integration-reliability finding
 

benchmarks/claims.registry.jsonl

@@ -4,4 +4,5 @@
 {"id":"B13-CORRECTED-FILE-RECALL","claim":"calm vs CodeGraph file-recall, multi-repo (fd, flask, self-repo); fixes a ground-truth oracle bug (doc/comment lines miscounted as call sites) and a SCIP-overlay race (recall measured before the async formal-confidence upgrade finished), and along the way finds + fixes a real CALM parser bug (Self::method() calls resolved to zero edges)","benchmark":"b13_codegraph_multirepo_ab","calm_commit":"52d1abe682069e018d00d09f2d9ab07b820a557c plus the oracle/scip_refresh-race fixes, uncommitted at benchmark time and later committed as 9c0b0fe; the Self:: parser fix landed separately as c5823a8","competitor_versions":"@colbymchenry/codegraph@1.5.0","corpus_commit":"fd (sharkdp/fd) @ 41532d1, flask, self-repo -- see README for full pins","harness_commit":"9c0b0fe","command":"see benchmarks/b13_codegraph_multirepo_ab/README.md Methodology section","raw_output_path":"benchmarks/b13_codegraph_multirepo_ab/README.md","canonical_result":"combined file-recall, N=24 symbols / 31 oracle files: calm 31/31 (100%), CodeGraph 27/31 (87.1%)","status":"current","supersedes":["B13-PHASE1-2-FILE-RECALL"],"superseded_by":null,"last_reproduced":"2026-08-02","evidence_gap":null}
 {"id":"DART-ZERO-CALL-EDGES","claim":"Dart call-graph extraction produces zero call edges -- documented as a deliberate scope cut (tree-sitter grammar has no node kind for call), not a bug","benchmark":"resolution","calm_commit":null,"competitor_versions":null,"corpus_commit":"dart-lang/args @ 7a2dfb5","harness_commit":null,"command":"see benchmarks/resolution/README.md Run section","raw_output_path":"benchmarks/resolution/README.md","canonical_result":"178 symbols indexed, 0 call edges","status":"superseded","supersedes":[],"superseded_by":"DART-C3-CALL-EDGE-EXTRACTION","last_reproduced":"2026-07-11","evidence_gap":"calm_commit not recorded in the original write-up; predates this checkout's available git history (starts 2026-07-28) -- exactly the kind of gap this registry exists to stop happening going forward"}
 {"id":"DART-C3-CALL-EDGE-EXTRACTION","claim":"Dart call-edge extraction (C3) recovers real edges via a dedicated dart_call_from_member_access parser branch; the original 'no node kind for call' assumption was wrong -- Dart's grammar shares member_access/selector/argument_part nodes between calls and plain field access, distinguished only by whether the trailing selector wraps an argument_part","benchmark":"resolution","calm_commit":null,"competitor_versions":null,"corpus_commit":"dart-lang/args @ 7a2dfb5 (same corpus commit as the superseded measurement)","harness_commit":null,"command":"see benchmarks/resolution/README.md Run section","raw_output_path":"benchmarks/resolution/README.md","canonical_result":"178 symbols, 2166 call edges: 7.8% resolved, 11.2% textual, 80.9% ambiguous, 0% formal/inferred (no SCIP provider for Dart)","status":"current","supersedes":["DART-ZERO-CALL-EDGES"],"superseded_by":null,"last_reproduced":"2026-07-28","evidence_gap":"calm_commit not recorded in the original write-up; predates this checkout's available git history (starts 2026-07-28)"}
-{"id":"B15-CROSS-LANG-COMPETITOR-AB","claim":"calm vs CodeGraph vs Ctxo vs Context+, file-recall on callers, all 6 Tier-0 languages (python/rust/go/javascript/typescript/java)","benchmark":"b15_cross_lang_competitor_ab","calm_commit":"822e238efc54df32da36505cf25890c2302ee06f","competitor_versions":"@colbymchenry/codegraph@1.5.0, @ctxo/cli@0.11.4, contextplus@1.0.8","corpus_commit":"flask 36e4a824, fd 41532d11, gin 34dac209, express a3714473, zod 912f0f51, spring-petclinic 51045d16 -- see README Version pins section for full pins","harness_commit":null,"command":"see benchmarks/b15_cross_lang_competitor_ab/README.md Run section","raw_output_path":"benchmarks/b15_cross_lang_competitor_ab/README.md","canonical_result":"aggregate file-recall (Ctxo counts java only, see README disclosure): calm 68/72 (94.4%), codegraph 69/72 (95.8%), ctxo 23/24 (96.0%, java-only), contextplus 72/72 (100.0%)","status":"current","supersedes":[],"superseded_by":null,"last_reproduced":"2026-08-18","evidence_gap":"Ctxo go/js/ts excluded from aggregate -- setup verified real but query results structurally empty despite 3 rounds of harness verification; root-caused to a real, unresolved Ctxo-side persistence issue (see README), not fixed further within this session's scope."}
+{"id":"B15-CROSS-LANG-COMPETITOR-AB","claim":"calm vs CodeGraph vs Ctxo vs Context+, file-recall on callers, all 6 Tier-0 languages (python/rust/go/javascript/typescript/java)","benchmark":"b15_cross_lang_competitor_ab","calm_commit":"822e238efc54df32da36505cf25890c2302ee06f","competitor_versions":"@colbymchenry/codegraph@1.5.0, @ctxo/cli@0.11.4, contextplus@1.0.8","corpus_commit":"flask 36e4a824, fd 41532d11, gin 34dac209, express a3714473, zod 912f0f51, spring-petclinic 51045d16 -- see README Version pins section for full pins","harness_commit":null,"command":"see benchmarks/b15_cross_lang_competitor_ab/README.md Run section","raw_output_path":"benchmarks/b15_cross_lang_competitor_ab/README.md","canonical_result":"aggregate file-recall (Ctxo counts java only, see README disclosure): calm 68/72 (94.4%), codegraph 69/72 (95.8%), ctxo 23/24 (96.0%, java-only), contextplus 72/72 (100.0%)","status":"superseded","supersedes":[],"superseded_by":"B15R2-CROSS-LANG-COMPETITOR-AB-INHERITANCE-FIX","last_reproduced":"2026-08-18","evidence_gap":"Ctxo go/js/ts excluded from aggregate -- setup verified real but query results structurally empty despite 3 rounds of harness verification; root-caused to a real, unresolved Ctxo-side persistence issue (see README), not fixed further within this session's scope. Superseded because the calm numbers themselves were measuring 2 real, since-fixed bugs, not calm's steady-state capability -- see superseding entry."}
+{"id":"B15R2-CROSS-LANG-COMPETITOR-AB-INHERITANCE-FIX","claim":"Same B15 A/B, rerun after fixing 2 real bugs the first run's own investigation found: a benchmark oracle regex false-positive (Java control-flow lines wrongly excluded as call sites) and a real calm core-resolver bug (inherited-method calls through a statically-typed receiver silently dropped instead of falling back to ambiguous)","benchmark":"b15_cross_lang_competitor_ab","calm_commit":"see git log after 0d9aa3a for the pipeline.rs inheritance-fallback fix + regression test, not yet committed as of this entry's last_reproduced date","competitor_versions":"@colbymchenry/codegraph@1.5.0, @ctxo/cli@0.11.4, contextplus@1.0.8","corpus_commit":"same 6 pinned corpora as B15 (see README Version pins) -- java resample differs because the fixed oracle changes which symbols pass sample_symbols' filters, not because of cherry-picking","harness_commit":null,"command":"see benchmarks/b15_cross_lang_competitor_ab/README.md Run section","raw_output_path":"benchmarks/b15_cross_lang_competitor_ab/README.md","canonical_result":"aggregate file-recall (Ctxo counts java only, see README disclosure): calm 72/73 (98.6%), codegraph 69/73 (94.5%), ctxo 23/25 (92.0%, java-only), contextplus 73/73 (100.0%) -- calm's sole remaining miss is the already-known JS new-expression gap, unrelated to the inheritance bug this run fixed","status":"current","supersedes":["B15-CROSS-LANG-COMPETITOR-AB"],"superseded_by":null,"last_reproduced":"2026-08-18","evidence_gap":"Same Ctxo go/js/ts exclusion as the superseded entry (unchanged, unrelated to this run's fixes). Core-resolver fix (pipeline.rs::resolve_sites_to_edges) verified via a new regression test plus full cargo test -p calm-core/calm-server (1234+395 tests, 0 regressions) and a live re-query against the real spring-petclinic corpus, but was NOT re-verified against a live re-run of B12/B13/B7's own Java rows, which share the same oracle bug and could show similar movement if rerun."}