fix(consistency): close the function-to-file relation blind spot from ADR-0018

ADR-0018 confirmed, with isolated probes, that a claim like
`apply_tax` calls `billing/tax_rates.py` never parsed as a relation at
all -- the target pattern (`[\w.]+`) excluded `/`, so the claim silently
decomposed into two independent, both-true existence checks and the false
relation went unchecked.

Fix: extended the relation target pattern in claims.py to accept file
paths (`[\w./-]+`), and added check_symbol_calls_file() in check.py,
dispatched from check_symbol_relation() whenever the target looks like a
path. A file-targeted relation claim asserts a module-level dependency,
not a function call, so it's checked against the file-level IMPORTS graph
(the same edges GraphQuery.file_dependencies already uses) instead of a
symbol-level CALLS edge between function nodes -- reusing existing graph
infrastructure, no ingester changes.

Verified against the exact probe ADR-0018 used to demonstrate the gap:
`apply_tax` calls `billing/tax_rates.py` on the real billing fixture now
reports CONTRADICTED ("the file defining 'apply_tax' does not import
'billing/tax_rates.py'") instead of silently vanishing. Along the way,
found (but did not fix, as it's a separate pre-existing gap) that the
ingester only registers an IMPORTS edge to a package's __init__.py for
`from package import submodule`-style imports, not to the submodule file
itself.

5 new regression tests (test_claims.py, test_consistency_check.py's new
TestSymbolCallsFileRelation). ADR-0018 updated to reflect the fix and its
residual, honestly-stated limitations (loose relation phrasing still
unrecognized; accuracy bounded by ingester import resolution).

512 tests passing (507 + 5 new).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Author: Juanpacol

README.md

@@ -271,7 +271,7 @@ producing a genuine mix of true and hallucinated claims:
 | Claim class | Result |
 |---|---|
 | Invented function names (14 across 5 trials) | **100% caught** |
-| Function-to-file relation hallucinations (~6) | **0% caught** — structurally invisible to the relation extractor |
+| Function-to-file relation hallucinations (~6) | **0% caught at the time** — structurally invisible to the relation extractor |
 | Backtick-quoted local variable names (8) | False positives — real names, just not graph-indexed |
 
 A real bug was also found and fixed in decision resurfacing: with only one
@@ -280,10 +280,18 @@ always normalized to 100% confidence regardless of actual relevance — a
 genuinely unrelated proposal "resembled" an unrelated rejected decision just
 as strongly as an actual paraphrase of it. Confirmed with an isolated probe,
 fixed by normalizing against the checked text's own best-possible score
-instead of the in-corpus max, and locked in with a regression test. See
-ADR-0018 for the full write-up, including what's still an open, unfixed
-blind spot (the relation-extraction gap) versus what got fixed this pass
-(the resurfacing bug).
+instead of the in-corpus max, and locked in with a regression test.
+
+**The function-to-file relation blind spot was closed the same day**: the
+relation target pattern now accepts file paths, and a new check
+(`check_symbol_calls_file`) verifies the claim against the file-level
+`IMPORTS` graph instead of a symbol-level `CALLS` edge. The exact probe
+this measurement used to demonstrate the gap —
+`` `apply_tax` calls `billing/tax_rates.py` `` — now correctly reports
+`CONTRADICTED` instead of vanishing. See ADR-0018 for the full write-up,
+including the residual limitations that remain open (loose relation
+phrasing, and accuracy bounded by how completely the ingester resolves
+`from package import submodule`-style imports).
 
 ---
 

docs/BENCHMARK_PROTOCOL.md

@@ -56,10 +56,13 @@ detection/`, ADR-0018) — not a Family A/B comparison, but the same
 "real data before publishing" discipline applied to a different engine.
 Real agents, shown only one file of a small codebase, produced a genuine
 mix of true and hallucinated claims about it. Result: 100% recall on
-invented symbol names (14/14 caught), a confirmed blind spot on
-function-to-file relation claims (0% caught, structurally invisible to the
-extractor), and a real bug in decision resurfacing — a corpus of one or two
-decisions always normalized its closest match to 100% confidence regardless
+invented symbol names (14/14 caught); a blind spot on function-to-file
+relation claims (0% caught at the time, structurally invisible to the
+extractor), closed the same day by extending the relation target pattern
+to accept file paths and checking such claims against the file-level
+IMPORTS graph instead of a symbol-level CALLS edge; and a real bug in
+decision resurfacing — a corpus of one or two decisions always normalized
+its closest match to 100% confidence regardless
 of actual relevance — found, fixed, and regression-tested.
 
 ## The two families

docs/adr/0018-consistency-engine-first-measurement.md

@@ -92,11 +92,26 @@ synthetic-fixture trap's blind spots.
 - **Symbol-existence checking has a clean, confirmed 100% recall** on
   invented function names in this pilot — the engine's strongest result so
   far.
-- **The function-to-file relation blind spot is real and unfixed.** Closing
-  it would mean extending `claims.py`'s relation extraction to recognize
-  file targets and to tolerate explanatory text between the relation verb
-  and its arguments — a real change to the extractor's scope, deliberately
-  left for a future pilot/ADR rather than rushed into this one.
+- **The function-to-file relation blind spot is fixed** (same-day
+  follow-up): `claims.py`'s relation target pattern now accepts file paths
+  (`[\w./-]+` instead of `[\w.]+`), and `check.py` gained
+  `check_symbol_calls_file`, which checks a file-targeted relation claim
+  against the file-level `IMPORTS` graph instead of a symbol-level `CALLS`
+  edge. Verified against the exact probe this ADR used to demonstrate the
+  gap: `` `apply_tax` calls `billing/tax_rates.py` `` now reports
+  `CONTRADICTED` ("the file defining 'apply_tax' does not import
+  'billing/tax_rates.py'") instead of silently vanishing into two
+  independent existence checks. Regression tests added in both
+  `test_claims.py` and `test_consistency_check.py`
+  (`TestSymbolCallsFileRelation`). One residual, honestly-stated limitation:
+  the fix's accuracy is bounded by how completely the ingester resolves
+  imports — `from package import submodule`-style imports were observed, in
+  the real `billing/tax.py` fixture, to register only an edge to the
+  package's `__init__.py`, not to the submodule file. That is a pre-existing
+  ingester limitation, not introduced by this fix, and is not addressed
+  here. The extractor's other named gap (tolerating explanatory text
+  between the relation verb and its arguments, e.g. "likely calls a helper
+  in") remains open, deliberately out of scope for this pass.
 - **Backtick-quoted local variable names are a genuine nuisance-false-
   positive source** in real usage — anyone writing a normal technical
   summary with `` `some_var` `` for readability will trigger a false

src/verityai/consistency/check.py

@@ -24,7 +24,7 @@
 
 from pathlib import Path
 
-from verityai.consistency.claims import extract_claims
+from verityai.consistency.claims import extract_claims, looks_like_path
 from verityai.context.rank import bm25_rank
 from verityai.core.models import (
     CheckStatus,
@@ -34,6 +34,8 @@
     ConsistencyReport,
     DecisionStatus,
     Evidence,
+    GraphNode,
+    NodeKind,
 )
 from verityai.graph.query import GraphQuery
 from verityai.graph.store import EdgeKind
@@ -73,8 +75,62 @@ def check_symbol_exists(claim: Claim, query: GraphQuery) -> ClaimCheck:
     )
 
 
+def check_symbol_calls_file(claim: Claim, query: GraphQuery) -> ClaimCheck:
+    """Does the file defining `claim.subject` actually import `claim.target`?
+
+    A relation claim whose target is a file, not a function, asserts a
+    module-level dependency, not a call edge -- there is no CALLS edge to a
+    file node. Checked instead against the file-level IMPORTS graph
+    (`GraphQuery.file_dependencies` uses the same edges). This closes the
+    blind spot found in ADR-0018: "`apply_tax` calls `billing/tax_rates.py`"
+    used to decompose into two independent, both-true existence checks and
+    vanish -- neither symbol/file existence check has any way to see that
+    the claimed relationship between them is false.
+    """
+    subject_nodes = query.define(claim.subject)
+    if not subject_nodes:
+        return ClaimCheck(
+            claim=claim,
+            status=CheckStatus.CONTRADICTED,
+            confidence=1.0,
+            explanation=f"no definition of {claim.subject!r} found",
+        )
+
+    target_path = claim.target or ""
+    target_file = query.store.get_node(GraphNode.make_id(NodeKind.FILE, target_path))
+    if target_file is None:
+        return ClaimCheck(
+            claim=claim,
+            status=CheckStatus.CONTRADICTED,
+            confidence=1.0,
+            explanation=f"no file at {target_path!r} found in the graph",
+        )
+
+    for subject_node in subject_nodes:
+        subject_file_id = GraphNode.make_id(NodeKind.FILE, subject_node.path)
+        imports = query.store.neighbours(subject_file_id, kinds=[EdgeKind.IMPORTS], direction="out")
+        if any(node.id == target_file.id for node in imports):
+            return ClaimCheck(
+                claim=claim,
+                status=CheckStatus.SUPPORTED,
+                confidence=1.0,
+                explanation=f"{subject_node.path!r} imports {target_path!r}",
+                evidence=[Evidence(kind="file", locator=subject_node.path)],
+            )
+
+    return ClaimCheck(
+        claim=claim,
+        status=CheckStatus.CONTRADICTED,
+        confidence=0.9,
+        explanation=(f"the file defining {claim.subject!r} does not import {target_path!r}"),
+    )
+
+
 def check_symbol_relation(claim: Claim, query: GraphQuery) -> ClaimCheck:
     """Does the graph actually contain the claimed relationship?"""
+    if looks_like_path(claim.target or ""):
+        return check_symbol_calls_file(claim, query)
+
     edge_kind = _RELATION_EDGE_KINDS.get(claim.relation or "")
     if edge_kind is None:
         return ClaimCheck(

src/verityai/consistency/claims.py

@@ -70,15 +70,26 @@
 _RELATION_PATTERN = re.compile(
     r"`(?P<subject>[\w.]+)`\s*(?:class|function|method|object)?\s+(?P<relation>"
     + "|".join(re.escape(phrase) for phrase in _RELATIONS)
-    + r")\s+`(?P<target>[\w.]+)`",
+    # The target may be a symbol (`[\w.]+`) or a file path -- `/` and `-`
+    # are needed for the latter (e.g. `billing/tax_rates.py`). Without them
+    # a claim like "`apply_tax` calls `billing/tax_rates.py`" never matches
+    # at all and silently decomposes into two independent, both-true
+    # existence checks -- see ADR-0018.
+    + r")\s+`(?P<target>[\w./-]+)`",
     re.IGNORECASE,
 )
 
 
-def _looks_like_path(token: str) -> bool:
+def looks_like_path(token: str) -> bool:
     return "/" in token or token.lower().endswith(_FILE_EXTENSIONS)
 
 
+# Kept as an alias: this was private until check.py needed the same test to
+# route a relation claim to the file-import checker instead of the
+# symbol-relation checker (ADR-0018's fix).
+_looks_like_path = looks_like_path
+
+
 def _looks_like_symbol(token: str) -> bool:
     return bool(_SYMBOL.match(token)) and (
         "." in token or "_" in token or token.endswith("()") or any(c.isupper() for c in token[1:])

tests/unit/test_claims.py

@@ -105,6 +105,20 @@ def test_extends_is_recognised(self):
         relation = next(c for c in claims if c.kind is ClaimKind.SYMBOL_RELATION)
         assert relation.relation == "inherits"
 
+    def test_a_file_path_target_is_recognised_as_a_relation(self):
+        """Regression (ADR-0018): a claim that a function calls INTO a file,
+        not another function, used to fall outside the target pattern
+        entirely and silently decompose into two independent, both-true
+        existence claims -- the false relation itself went unchecked."""
+        claims = extract_claims("`apply_tax` calls `billing/tax_rates.py` to resolve the rate.")
+
+        relation = next(c for c in claims if c.kind is ClaimKind.SYMBOL_RELATION)
+        assert relation.subject == "apply_tax"
+        assert relation.target == "billing/tax_rates.py"
+        # And it must not ALSO appear as a bare FILE_EXISTS claim -- same
+        # subsumption rule as a symbol-to-symbol relation.
+        assert not any(c.kind is ClaimKind.FILE_EXISTS for c in claims)
+
     def test_an_unmapped_verb_is_not_extracted_as_a_relation(self):
         """'depends on' and 'uses' are ambiguous about which edge kind they
         mean, so they are left unrecognised rather than mapped to a guess."""

tests/unit/test_consistency_check.py

@@ -146,6 +146,62 @@ def test_an_unresolved_ambiguous_call_is_unverifiable(self, tmp_path):
         store.close()
 
 
+class TestSymbolCallsFileRelation:
+    """ADR-0018 found that a relation claim targeting a FILE, not a function,
+    silently decomposed into two independent, both-true existence checks and
+    vanished. These reproduce the exact real-world case: a function whose
+    file does not import the file it's claimed to "call into"."""
+
+    @pytest.fixture
+    def two_file_project(self, tmp_path):
+        (tmp_path / "rates.py").write_text("REGION_RATES = {}\n")
+        (tmp_path / "policy.py").write_text("ACTIVE_POLICY = {}\n")
+        (tmp_path / "tax.py").write_text(
+            "from rates import REGION_RATES\n\n\ndef apply_tax(subtotal, region):\n    return subtotal\n"
+        )
+        return tmp_path
+
+    @pytest.fixture
+    def two_file_query(self, two_file_project):
+        store = GraphStore()
+        ingest_repo(two_file_project, store)
+        yield GraphQuery(store)
+        store.close()
+
+    def test_an_actual_import_is_supported(self, two_file_query):
+        result = check_symbol_relation(
+            relation_claim("apply_tax", "calls", "rates.py"), two_file_query
+        )
+
+        assert result.status is CheckStatus.SUPPORTED
+
+    def test_a_file_never_imported_is_contradicted(self, two_file_query):
+        """apply_tax's file (tax.py) never imports policy.py -- the exact
+        shape of hallucination ADR-0018 found real agents producing."""
+        result = check_symbol_relation(
+            relation_claim("apply_tax", "calls", "policy.py"), two_file_query
+        )
+
+        assert result.status is CheckStatus.CONTRADICTED
+        assert "does not import" in result.explanation
+
+    def test_a_nonexistent_target_file_is_contradicted(self, two_file_query):
+        result = check_symbol_relation(
+            relation_claim("apply_tax", "calls", "nonexistent.py"), two_file_query
+        )
+
+        assert result.status is CheckStatus.CONTRADICTED
+        assert "no file at" in result.explanation.lower()
+
+    def test_missing_subject_is_still_contradicted(self, two_file_query):
+        result = check_symbol_relation(
+            relation_claim("nonexistent_fn", "calls", "rates.py"), two_file_query
+        )
+
+        assert result.status is CheckStatus.CONTRADICTED
+        assert "nonexistent_fn" in result.explanation
+
+
 class TestFileExistence:
     def test_an_existing_file_is_supported(self, project):
         result = check_file_exists(file_claim("src/limits.py"), project)