fix(graph): resolve from package import submodule to the real submodule file

While verifying ADR-0018's function-to-file relation fix against the real
billing fixture, found that `billing/tax.py` (which does `from billing
import tax_rates` and genuinely reads `tax_rates.REGION_RATES`) was
recorded in the graph as importing only `billing/__init__.py`, never
`billing/tax_rates.py`. Root cause in `graph/ingest.py`'s
`visit_ImportFrom`: only the bare package name reached `_add_import`,
never the imported names themselves. Python's grammar cannot distinguish
"importing a submodule via its package" from "importing a plain symbol
defined in the package's __init__.py" -- both are
`ImportFrom(module="billing", names=["tax_rates"])`. Only a post-walk
check of whether `billing.tax_rates` corresponds to a real first-party
file can tell them apart, and `_resolve` already does exactly this kind of
lookup for other edge kinds -- it just was never given the candidate to
try.

Fix: `visit_ImportFrom` now records one IMPORTS edge per imported name
(matching plain `import a, b`'s existing per-alias granularity), each
carrying the qualified candidate (`module.name`) in its metadata.
`_resolve` tries that candidate first -- if it's a real first-party file,
that IS what Python binds at runtime, not a guess -- falling back to the
bare package (the `__init__.py`-symbol case) exactly as before when it
isn't.

This changed the correct answer to ADR-0018's own demonstration probe:
`apply_tax` calls `billing/tax_rates.py` now reports SUPPORTED, because
the import genuinely exists -- the CONTRADICTED verdict reported moments
earlier (in the immediately preceding commit) was itself an artifact of
this ingester bug, not the true answer. A second probe against a file
`apply_tax`'s module never imports still correctly reports CONTRADICTED,
confirming the mechanism works once the underlying import graph is
accurate. README.md, BENCHMARK_PROTOCOL.md and ADR-0018 updated to reflect
the corrected sequence rather than leaving a now-stale claim standing.

2 new regression tests in test_graph_ingest.py (submodule-import case;
plain __init__.py-symbol case confirming no regression). No test in the
suite asserted an exact import-edge count, so the per-name edge
granularity change is safe.

520 tests passing (518 + 2 new).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Author: Juanpacol

README.md

@@ -282,16 +282,22 @@ as strongly as an actual paraphrase of it. Confirmed with an isolated probe,
 fixed by normalizing against the checked text's own best-possible score
 instead of the in-corpus max, and locked in with a regression test.
 
-**The function-to-file relation blind spot was closed the same day**: the
-relation target pattern now accepts file paths, and a new check
-(`check_symbol_calls_file`) verifies the claim against the file-level
-`IMPORTS` graph instead of a symbol-level `CALLS` edge. The exact probe
-this measurement used to demonstrate the gap —
-`` `apply_tax` calls `billing/tax_rates.py` `` — now correctly reports
-`CONTRADICTED` instead of vanishing. See ADR-0018 for the full write-up,
-including the residual limitations that remain open (loose relation
-phrasing, and accuracy bounded by how completely the ingester resolves
-`from package import submodule`-style imports).
+**The function-to-file relation blind spot, and a real ingester bug behind
+it, were both closed the same day.** The relation target pattern now
+accepts file paths, and a new check (`check_symbol_calls_file`) verifies
+the claim against the file-level `IMPORTS` graph. That surfaced a second,
+independent bug: the ingester recorded `from billing import tax_rates` as
+importing only the bare package, never the real submodule file —
+`billing/tax.py` genuinely does import `billing/tax_rates.py`, but the
+graph didn't know it. Fixed in `graph/ingest.py` by trying the more
+specific candidate first. The exact probe this measurement used to
+demonstrate the original gap — `` `apply_tax` calls `billing/tax_rates.py` ``
+— now correctly reports **`SUPPORTED`** (the import genuinely exists);
+a second probe against a file `apply_tax`'s module never imports still
+correctly reports `CONTRADICTED`, confirming the mechanism works once the
+underlying import graph is accurate. See ADR-0018 for the full sequence,
+including the one relation-extraction gap that remains open (loose
+phrasing like "likely calls a helper in").
 
 ---
 

docs/BENCHMARK_PROTOCOL.md

@@ -60,10 +60,13 @@ invented symbol names (14/14 caught); a blind spot on function-to-file
 relation claims (0% caught at the time, structurally invisible to the
 extractor), closed the same day by extending the relation target pattern
 to accept file paths and checking such claims against the file-level
-IMPORTS graph instead of a symbol-level CALLS edge; and a real bug in
-decision resurfacing — a corpus of one or two decisions always normalized
-its closest match to 100% confidence regardless
-of actual relevance — found, fixed, and regression-tested.
+IMPORTS graph instead of a symbol-level CALLS edge -- which in turn
+surfaced a second, independent ingester bug (`from package import
+submodule` resolved only to the package, never the real submodule file),
+also fixed the same day; and a real bug in decision resurfacing — a corpus
+of one or two decisions always normalized its closest match to 100%
+confidence regardless of actual relevance — found, fixed, and
+regression-tested.
 
 ## The two families
 

docs/adr/0018-consistency-engine-first-measurement.md

@@ -97,21 +97,38 @@ synthetic-fixture trap's blind spots.
   (`[\w./-]+` instead of `[\w.]+`), and `check.py` gained
   `check_symbol_calls_file`, which checks a file-targeted relation claim
   against the file-level `IMPORTS` graph instead of a symbol-level `CALLS`
-  edge. Verified against the exact probe this ADR used to demonstrate the
-  gap: `` `apply_tax` calls `billing/tax_rates.py` `` now reports
-  `CONTRADICTED` ("the file defining 'apply_tax' does not import
-  'billing/tax_rates.py'") instead of silently vanishing into two
-  independent existence checks. Regression tests added in both
-  `test_claims.py` and `test_consistency_check.py`
-  (`TestSymbolCallsFileRelation`). One residual, honestly-stated limitation:
-  the fix's accuracy is bounded by how completely the ingester resolves
-  imports — `from package import submodule`-style imports were observed, in
-  the real `billing/tax.py` fixture, to register only an edge to the
-  package's `__init__.py`, not to the submodule file. That is a pre-existing
-  ingester limitation, not introduced by this fix, and is not addressed
-  here. The extractor's other named gap (tolerating explanatory text
-  between the relation verb and its arguments, e.g. "likely calls a helper
-  in") remains open, deliberately out of scope for this pass.
+  edge. First verified against the exact probe this ADR used to demonstrate
+  the gap — `` `apply_tax` calls `billing/tax_rates.py` `` went from
+  vanishing entirely to reporting `CONTRADICTED`, because at that point the
+  ingester itself had a separate bug (below) that failed to record the real
+  import. Regression tests added in both `test_claims.py` and
+  `test_consistency_check.py` (`TestSymbolCallsFileRelation`).
+- **The ingester's own import-resolution gap — also fixed, same day.**
+  `billing/tax.py` does `from billing import tax_rates` and genuinely reads
+  `tax_rates.REGION_RATES` — a real import of `billing/tax_rates.py`. But
+  `graph/ingest.py`'s `visit_ImportFrom` recorded only the bare package name
+  (`billing`) as the import target, never trying `billing.tax_rates` as a
+  candidate — Python's grammar cannot tell "importing a submodule via its
+  package" from "importing a symbol defined in the package's `__init__.py`"
+  apart syntactically; only a post-walk check of whether `billing.tax_rates`
+  is a real first-party file can. Fixed by recording that more specific
+  candidate per imported name (one `IMPORTS` edge per name, matching plain
+  `import a, b`'s existing granularity) and having `_resolve` try it first.
+  **This changed the probe's correct answer**: with the ingester fixed,
+  `billing/tax.py` is now correctly recorded as importing
+  `billing/tax_rates.py`, so `` `apply_tax` calls `billing/tax_rates.py` ``
+  now reports `SUPPORTED` — the CONTRADICTED verdict reported moments
+  earlier was itself an artifact of the ingester bug, not the correct
+  answer. A second probe, `` `apply_tax` calls `billing/policy.py` `` (a
+  file `tax.py` genuinely never imports), still correctly reports
+  `CONTRADICTED`, confirming the mechanism distinguishes real from false
+  file relations once the underlying import graph is accurate. Two
+  regression tests added in `test_graph_ingest.py`, one confirming the
+  submodule case resolves to the submodule file, one confirming a plain
+  `__init__.py`-symbol import is unaffected. The extractor's remaining named
+  gap (tolerating explanatory text between the relation verb and its
+  arguments, e.g. "likely calls a helper in") remains open, deliberately
+  out of scope for this pass.
 - **Backtick-quoted local variable names — addressed, deliberately without
   changing any verdict** (same-day follow-up). Investigating a fix
   surfaced a harder fact: a real local variable name (`with_tax`) and a

experiments/consistency_pilot_1_hallucination_detection/README.md

@@ -8,9 +8,15 @@ ADR-0007. What was actually missing was the same thing Context Engine
 lacked before ADR-0009: a measurement against real, not hand-authored,
 input. This experiment is that measurement.
 
-**Status: complete, and it found a real bug**, fixed in
-`src/verityai/consistency/check.py` (see `docs/adr/0018-consistency-engine-
-first-measurement.md`).
+**Status: complete, and it found three real bugs**, all fixed the same day
+(see `docs/adr/0018-consistency-engine-first-measurement.md` for the full
+sequence): the decision-resurfacing normalization bug below, the
+function-to-file relation blind spot, and — found while fixing that blind
+spot — a separate ingester bug that had been silently making the blind-spot
+fix's own verdicts *less* accurate. Note: this README describes what Part A
+found at measurement time; the specific "0% caught" relation-blind-spot
+result and its example probe were both since fixed, and the exact probe's
+correct answer changed as a result (see the ADR).
 
 ## Part A — inducing real hallucinations, not writing them by hand
 

src/verityai/graph/ingest.py

@@ -166,13 +166,23 @@ def visit_ImportFrom(self, node: ast.ImportFrom) -> None:
         if node.level:
             module = "." * node.level + module
 
+        # One edge per name (matching `visit_Import`'s granularity), because
+        # `from pkg import a, b` cannot be resolved as a single unit: `a`
+        # might be a submodule file and `b` a plain symbol in the same
+        # statement. Python's grammar cannot tell "importing a submodule via
+        # its package" from "importing a symbol defined in the package's
+        # __init__.py" -- both are `ImportFrom(module="pkg", names=["x"])`.
+        # `submodule_candidate` carries the more specific guess (`pkg.x`) for
+        # `_resolve` to try first, once every file has been walked and it can
+        # check whether that candidate is a real first-party file.
         for alias in node.names:
             local = alias.asname or alias.name
-            self.imported_names[local] = f"{module}.{alias.name}" if module else alias.name
-        self._add_import(module or ".", node.lineno)
+            qualified = f"{module}.{alias.name}" if module else alias.name
+            self.imported_names[local] = qualified
+            self._add_import(module or ".", node.lineno, submodule_candidate=qualified)
         self.generic_visit(node)
 
-    def _add_import(self, module: str, line: int) -> None:
+    def _add_import(self, module: str, line: int, submodule_candidate: str | None = None) -> None:
         """Record an imported module as an EXTERNAL node.
 
         Every import becomes a node even when the target is a first-party
@@ -205,7 +215,10 @@ def _add_import(self, module: str, line: int) -> None:
                 # resolver needs the name to look the module up among
                 # first-party files; without this it can only ever see
                 # "external:b" and no first-party import would ever resolve.
-                metadata={"module": module},
+                # `submodule_candidate` (from `from X import Y`) is `X.Y`,
+                # a strictly more specific guess than `module` alone -- see
+                # `_resolve`.
+                metadata={"module": module, "submodule_candidate": submodule_candidate},
                 line=line,
             )
         )
@@ -524,7 +537,16 @@ def _resolve(
 
     if edge.kind is EdgeKind.IMPORTS:
         module = edge.metadata.get("module", "")
-        target_id = module_ids.get(module)
+        # `from pkg import name` is ambiguous by grammar alone: `name` might
+        # be a submodule file (`pkg/name.py`) or a plain symbol defined in
+        # `pkg/__init__.py`. Try the more specific candidate first -- if
+        # `pkg.name` really is a first-party file, that IS what Python binds
+        # at runtime, not a guess. Falls back to the package itself when the
+        # candidate isn't a real file (the __init__.py-symbol case).
+        candidate = edge.metadata.get("submodule_candidate")
+        target_id = module_ids.get(candidate) if candidate else None
+        if target_id is None:
+            target_id = module_ids.get(module)
         if target_id is None and module:
             # `from pkg.core import X` names a module that is a file; `import
             # pkg` names a package whose file is `pkg/__init__.py`. Try the

tests/unit/test_graph_ingest.py

@@ -297,6 +297,42 @@ def test_tests_edges_are_derived_from_calls(self, repo, store):
         testers = store.neighbours(factory.id, [EdgeKind.TESTS], direction="in")
         assert [n.name for n in testers] == ["test_service_runs"]
 
+    def test_from_package_import_submodule_resolves_to_the_submodule_file(self, tmp_path, store):
+        """Regression (ADR-0018): `from billing import tax_rates` is a real
+        import of `billing/tax_rates.py`, not of `billing/__init__.py` --
+        but grammatically it looks identical to importing a plain symbol
+        defined inside `__init__.py`. Only a post-walk file-existence check
+        can tell them apart, and it must prefer the submodule file when one
+        exists."""
+        (tmp_path / "billing").mkdir()
+        (tmp_path / "billing" / "__init__.py").write_text("")
+        (tmp_path / "billing" / "tax_rates.py").write_text("REGION_RATES = {}\n")
+        (tmp_path / "billing" / "tax.py").write_text(
+            "from billing import tax_rates\n\n\ndef apply_tax():\n    return tax_rates.REGION_RATES\n"
+        )
+
+        ingest_repo(tmp_path, store)
+
+        tax_file = next(n for n in store.all_nodes(NodeKind.FILE) if n.path == "billing/tax.py")
+        imports = store.neighbours(tax_file.id, [EdgeKind.IMPORTS], direction="out")
+        assert "billing/tax_rates.py" in {n.path for n in imports}
+        assert "billing/__init__.py" not in {n.path for n in imports}
+
+    def test_from_package_import_symbol_still_resolves_to_the_package(self, tmp_path, store):
+        """The other half of the same ambiguity: `from billing import
+        HELPER` where `HELPER` is a plain symbol defined in
+        `billing/__init__.py`, not a submodule file -- must still resolve
+        to the package, unchanged from before this fix."""
+        (tmp_path / "billing").mkdir()
+        (tmp_path / "billing" / "__init__.py").write_text("HELPER = 1\n")
+        (tmp_path / "billing" / "tax.py").write_text("from billing import HELPER\n")
+
+        ingest_repo(tmp_path, store)
+
+        tax_file = next(n for n in store.all_nodes(NodeKind.FILE) if n.path == "billing/tax.py")
+        imports = store.neighbours(tax_file.id, [EdgeKind.IMPORTS], direction="out")
+        assert "billing/__init__.py" in {n.path for n in imports}
+
 
 class TestIncrementality:
     def test_unchanged_files_are_skipped_on_reingest(self, repo, store):