Summary
`scripts/check-b2-thresholds.sh` (added 2026-08-04) has failed every nightly run since 2026-08-05 — never passed since it was created. Root-caused this session; two real resolver bugs fixed and tested in 0.7.0, a third documented here as a scoped follow-up, plus a separate benchmark-methodology gap.
Fixed in 0.7.0 (`resolve_sites_to_edges`, `crates/calm-core/src/indexer/pipeline.rs`)
Mechanism 1 — unscoped by-name fallback for unknown-receiver method calls. A `.`-receiver call whose type this indexer never tracks (`some_hashmap.get(k)?`) fell through to the unscoped `ctx.by_name` fallback and confidently resolved to whatever unrelated same-named function happens to be the ONLY one of that name in the whole repo. Verified live: `crates/calm-core/src/analysis/coverage.rs`'s real `row.get(..)` calls (a `rusqlite::Row::get`, receiver="row", target_class NULL) were among 1114 false `textual` edges all pointing at the unrelated `crates/calm-core/src/txn.rs::get` — the only "get" in the entire Rust symbol table.
Mechanism 2 — same-file method-name collision. The existing `same_file` preference ("Rust's own scoping, not a heuristic") is true for bare FREE-FUNCTION calls but not for METHOD calls — a file can define its own `impl SomeEnum { fn as_str() }` and separately call `some_string.as_str()` (the stdlib's) in the same file; these are unrelated. Verified live: `crates/calm-server/src/tools/edit.rs`'s own `.as_str()` on a `String` field (edit.rs also defines `GateRequirement::as_str`) and `crates/calm-core/src/txn.rs`'s `.as_str()` on a `TxState` variant (txn.rs also defines `TxState::as_str`) both fanned out this way.
Both fixed via one signal (`weak_receiver`): a `.`-receiver call whose `target_class` never resolved downgrades to `Ambiguous` instead of trusting whatever narrowing branch (same_file/same_dir/the final catch-all) happened to produce a single survivor — except `self`/`this` receivers, which are excluded (their real type IS the enclosing impl by construction, so same_file is still strong evidence there; caught live by `test_caller_count_excludes_ambiguous_fan_out_edges`'s `self.as_str()` regressing before the exclusion was added). `Type::path()` calls and tier-2-typed receivers (target_class already known) are unaffected either way.
Two new/existing regression tests cover this: `test_unresolved_receiver_method_call_does_not_fan_out_to_unrelated_same_named_function` (new) plus the existing suite (77/77 pipeline tests, 1233/1233 full calm-core lib tests, all green).
Real before/after via `benchmarks/b2_call_graph_quality/run_benchmark.py` against this repo:
- `ambiguous` tier count: 3057 → 4817 (the false positives correctly reclassified, not silently dropped)
- `textual`-tier count: 2328 → 704
- `resolved`-tier count: 324 → 211
- `formal`-tier precision: unaffected (0.998, as expected — this fix only touches the syntactic by-name/by-name-class paths, never SCIP-upgraded edges)
NOT fixed — documented follow-up: fully-qualified external-crate-root paths
`std::fs::write(...)` (main.rs, ~30 call sites) still misattributes to the same `txn.rs`-style local hub via the SAME by-name fallback, but with `receiver: NULL` — this is a qualified FREE-FUNCTION-style path call (`std::fs::write`, not a `.`-receiver), so mechanism 1/2's fix (scoped to `receiver.is_some()`) correctly doesn't touch it, by design (bare/qualified free-function calls were deliberately left alone since Rust's real name-resolution rules make the by-name fallback more justified for them in general). The bug here is narrower: a path explicitly rooted at a well-known external crate (`std::`/`core::`/`alloc::`) should never fall through to the local by-name table at all — the qualification itself proves it's not a local symbol. Needs its own scoped change to `module_hint_of`/the qualified-path handling in `parser.rs` (different subsystem from `resolve_sites_to_edges`), not attempted here to avoid rushing a second, less-well-understood change into the same session.
Separate finding: benchmark oracle coverage gap, not a CALM bug
`rust-analyzer scip .` (default features, matching what the benchmark script runs) has zero occurrences for 13 real source files, including `crates/calm-core/src/bundle.rs`, all of `crates/calm-core/src/lsp/*.rs`, and both `http.rs` files — almost certainly because these live behind non-default Cargo features (`index-bundles`, `lsp-overlay`, etc.) that a default-features `rust-analyzer scip` run never compiles, while CALM's own tree-sitter-based indexer parses them unconditionally regardless of feature flags. Any CALM edge touching these files can never match the oracle, correct or not — inflating the apparent false-positive count independent of resolver quality. `benchmarks/b2_call_graph_quality/run_benchmark.py` should probably run `rust-analyzer scip` with matching feature flags (or the benchmark's known-limitations doc should note this gap) — not attempted here, out of scope for a resolver-correctness session.
Why check-b2-thresholds.sh's floors still won't pass yet
Even after both mechanism-1/2 fixes, `inferred`/`resolved`/`textual`-tier precision are still measured at 0.0 in this run — the remaining ~967 non-ambiguous, non-formal edges are dominated by the qualified-path gap above and the oracle coverage gap, not (as far as this session's investigation found) further resolver false positives. Confirming that fully needs the module_hint_of follow-up and/or the oracle feature-flag fix landed first; the floors themselves (set from a one-off manual measurement in 2026-08-04, never reproduced by an actual CI run since) may also be worth revisiting once both are in.
Ask
- Land the `std::`/`core::`/`alloc::`-rooted qualified-path fix as its own scoped PR.
- Fix or document the benchmark's oracle feature-flag gap.
- Re-measure and reset `check-b2-thresholds.sh`'s floors against a real, reproduced CI run once both land.
Summary
`scripts/check-b2-thresholds.sh` (added 2026-08-04) has failed every nightly run since 2026-08-05 — never passed since it was created. Root-caused this session; two real resolver bugs fixed and tested in 0.7.0, a third documented here as a scoped follow-up, plus a separate benchmark-methodology gap.
Fixed in 0.7.0 (`resolve_sites_to_edges`, `crates/calm-core/src/indexer/pipeline.rs`)
Mechanism 1 — unscoped by-name fallback for unknown-receiver method calls. A `.`-receiver call whose type this indexer never tracks (`some_hashmap.get(k)?`) fell through to the unscoped `ctx.by_name` fallback and confidently resolved to whatever unrelated same-named function happens to be the ONLY one of that name in the whole repo. Verified live: `crates/calm-core/src/analysis/coverage.rs`'s real `row.get(..)` calls (a `rusqlite::Row::get`, receiver="row", target_class NULL) were among 1114 false `textual` edges all pointing at the unrelated `crates/calm-core/src/txn.rs::get` — the only "get" in the entire Rust symbol table.
Mechanism 2 — same-file method-name collision. The existing `same_file` preference ("Rust's own scoping, not a heuristic") is true for bare FREE-FUNCTION calls but not for METHOD calls — a file can define its own `impl SomeEnum { fn as_str() }` and separately call `some_string.as_str()` (the stdlib's) in the same file; these are unrelated. Verified live: `crates/calm-server/src/tools/edit.rs`'s own `.as_str()` on a `String` field (edit.rs also defines `GateRequirement::as_str`) and `crates/calm-core/src/txn.rs`'s `.as_str()` on a `TxState` variant (txn.rs also defines `TxState::as_str`) both fanned out this way.
Both fixed via one signal (`weak_receiver`): a `.`-receiver call whose `target_class` never resolved downgrades to `Ambiguous` instead of trusting whatever narrowing branch (same_file/same_dir/the final catch-all) happened to produce a single survivor — except `self`/`this` receivers, which are excluded (their real type IS the enclosing impl by construction, so same_file is still strong evidence there; caught live by `test_caller_count_excludes_ambiguous_fan_out_edges`'s `self.as_str()` regressing before the exclusion was added). `Type::path()` calls and tier-2-typed receivers (target_class already known) are unaffected either way.
Two new/existing regression tests cover this: `test_unresolved_receiver_method_call_does_not_fan_out_to_unrelated_same_named_function` (new) plus the existing suite (77/77 pipeline tests, 1233/1233 full calm-core lib tests, all green).
Real before/after via `benchmarks/b2_call_graph_quality/run_benchmark.py` against this repo:
NOT fixed — documented follow-up: fully-qualified external-crate-root paths
`std::fs::write(...)` (main.rs, ~30 call sites) still misattributes to the same `txn.rs`-style local hub via the SAME by-name fallback, but with `receiver: NULL` — this is a qualified FREE-FUNCTION-style path call (`std::fs::write`, not a `.`-receiver), so mechanism 1/2's fix (scoped to `receiver.is_some()`) correctly doesn't touch it, by design (bare/qualified free-function calls were deliberately left alone since Rust's real name-resolution rules make the by-name fallback more justified for them in general). The bug here is narrower: a path explicitly rooted at a well-known external crate (`std::`/`core::`/`alloc::`) should never fall through to the local by-name table at all — the qualification itself proves it's not a local symbol. Needs its own scoped change to `module_hint_of`/the qualified-path handling in `parser.rs` (different subsystem from `resolve_sites_to_edges`), not attempted here to avoid rushing a second, less-well-understood change into the same session.
Separate finding: benchmark oracle coverage gap, not a CALM bug
`rust-analyzer scip .` (default features, matching what the benchmark script runs) has zero occurrences for 13 real source files, including `crates/calm-core/src/bundle.rs`, all of `crates/calm-core/src/lsp/*.rs`, and both `http.rs` files — almost certainly because these live behind non-default Cargo features (`index-bundles`, `lsp-overlay`, etc.) that a default-features `rust-analyzer scip` run never compiles, while CALM's own tree-sitter-based indexer parses them unconditionally regardless of feature flags. Any CALM edge touching these files can never match the oracle, correct or not — inflating the apparent false-positive count independent of resolver quality. `benchmarks/b2_call_graph_quality/run_benchmark.py` should probably run `rust-analyzer scip` with matching feature flags (or the benchmark's known-limitations doc should note this gap) — not attempted here, out of scope for a resolver-correctness session.
Why check-b2-thresholds.sh's floors still won't pass yet
Even after both mechanism-1/2 fixes, `inferred`/`resolved`/`textual`-tier precision are still measured at 0.0 in this run — the remaining ~967 non-ambiguous, non-formal edges are dominated by the qualified-path gap above and the oracle coverage gap, not (as far as this session's investigation found) further resolver false positives. Confirming that fully needs the module_hint_of follow-up and/or the oracle feature-flag fix landed first; the floors themselves (set from a one-off manual measurement in 2026-08-04, never reproduced by an actual CI run since) may also be worth revisiting once both are in.
Ask