fix(resolver): resolve Python builtins through the formal tier (DEBT-005)

Root cause was investigated during earlier work: tree-sitter-stack-graphs-
python 0.3.0's bundled src/builtins.py ships empty, so FormalResolver's
merged builtins graph had nothing in it — any reference to len/print/range/
etc. fell through to the resolved/textual tier instead of formal, despite
stack-graphs.tsg wiring up a "<builtins>" push-symbol fallback edge for
exactly this case.

Fix required no TSG grammar patch. build_python_builtins_graph() builds a
StackGraph from a synthetic PYTHON_BUILTINS_STUB (~70 common builtins as
plain def/class stubs) through the same compiled `sgl` rules the upstream
crate already provides, using FILE_PATH="<builtins>.py". That exact path
satisfies stack-graphs.tsg's own per-file module-path rule (the regex branch
that turns a file's relative path into a pop_symbol chain anchored at
ROOT_NODE), producing a pop_symbol="<builtins>" node — the missing
counterpart to every file's push_symbol="<builtins>" fallback edge. Verified
by two new tests (len()/print() resolve; a genuinely undefined name still
doesn't) and a real `ci index` smoke test showing call_sites.confidence =
'formal' for len/print/enumerate/isinstance/sorted calls.

Known scope limit, documented in pattern-debt-registry.yaml and the README:
this is correct at the call_sites storage layer, but rebuild_graph filters
call_sites -> call_edges by name-match against indexed project symbols,
independent of confidence — builtins never match (they aren't project
symbols) so they still don't produce call_edges rows or show up in
callers/path/caller_count_by_confidence. Making that visible would need
builtins to become pseudo-symbols, which risks skewing hub_count/coreness
project-wide (len/print called everywhere) — deliberately left as a
separate, explicitly-tracked follow-up rather than folded in here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WTXEBnJdVd1o2egp1bTBaX

Author: claude

README.md

@@ -20,12 +20,11 @@ tính graph metrics (coreness/hubs), và phục vụ qua SQLite FTS5 + semantic
 2. **Call graph phân cấp** — mỗi edge mang một mức tin cậy:
    - `resolved` — khớp file symbol / import / alias (tier-1, conservative resolver).
    - `inferred` — method call phân giải theo kiểu của receiver (tier-2: `self`/`this` → class bao quanh; biến typed → `type_map`).
-   - `formal` — phân giải phạm vi tĩnh qua `stack-graphs` (tier-3, hiện hỗ trợ Python). Được bảo vệ bởi **hai deadline độc lập**: một cho bước build stack-graph (TSG) và một cho bước path-stitching, cộng thêm cap `MAX_WORK_PER_PHASE = 4096` để chống DoS.
-     > **Lưu ý `formal` tier**: `formal` **không resolve được builtin Python** (`len`, `print`, `range`...) —
-     > `src/builtins.py` bundled trong `tree-sitter-stack-graphs-python` 0.3.0 rỗng, và rule TSG upstream
-     > push symbol `"<builtins>"` nhưng không có rule nào pop lại — dead-end ở tầng grammar, không phải bug
-     > phía `ci`. Repo upstream (`github/stack-graphs`) đã bị archive từ 9/2025, không còn nhận fix. Các
-     > reference gọi builtin sẽ fall back về `resolved`/`textual` tier như bình thường.
+   - `formal` — phân giải phạm vi tĩnh qua `stack-graphs` (tier-3, hiện hỗ trợ Python). Được bảo vệ bởi **hai deadline độc lập**: một cho bước build stack-graph (TSG) và một cho bước path-stitching, cộng thêm cap `MAX_WORK_PER_PHASE = 4096` để chống DoS. Python builtins (`len`, `print`, `range`...) resolve qua tier này nhờ `build_python_builtins_graph` tự build (bundled `src/builtins.py` của `tree-sitter-stack-graphs-python` 0.3.0 rỗng, không dùng được trực tiếp).
+     > **Lưu ý**: builtin call hiện được gắn đúng `edge_confidence: formal` ở tầng lưu trữ nội bộ
+     > (`call_sites`), nhưng **chưa hiển thị qua `callers`/`path`/`caller_count_by_confidence`** —
+     > các tool đó chỉ đọc `call_edges`, vốn chỉ chứa cạnh giữa 2 symbol đã index trong project;
+     > builtin không phải project symbol nên không tạo `call_edges` dù ở tier nào.
    - `textual` — chỉ khớp tên (fallback).
 3. **Import graph** — `import_edges` (file→module/file) cho tool `dependencies`.
 4. **Graph metrics** — `coreness` (k-core, O(V+E)) và `is_hub` để AI biết đâu là lõi hệ thống.

crates/ci-core/src/resolver/formal.rs

@@ -55,6 +55,111 @@ const RESOLVE_TIMEOUT: Duration = Duration::from_secs(3);
 /// before the deadline check below ever gets a chance to fire.
 const MAX_WORK_PER_PHASE: usize = 4096;
 
+/// Synthetic source for a virtual "<builtins>.py" file, standing in for
+/// `tree-sitter-stack-graphs-python` 0.3.0's own bundled `src/builtins.py`
+/// (which ships empty — see DEBT-005 / the regression test below for the
+/// original investigation). Bodies are `pass` — only the *names* need to
+/// exist as top-level definitions for references to resolve to; nothing
+/// calls into these bodies. Not exhaustive, but covers the builtins that
+/// show up in real code by a wide margin.
+const PYTHON_BUILTINS_STUB: &str = r#"
+def print(*args, **kwargs): pass
+def len(obj): pass
+def range(*args): pass
+def isinstance(obj, cls): pass
+def issubclass(cls, classinfo): pass
+def super(*args): pass
+def open(*args, **kwargs): pass
+def enumerate(iterable, start=0): pass
+def zip(*iterables): pass
+def map(func, *iterables): pass
+def filter(func, iterable): pass
+def sorted(iterable, *, key=None, reverse=False): pass
+def reversed(seq): pass
+def min(*args, **kwargs): pass
+def max(*args, **kwargs): pass
+def sum(iterable, start=0): pass
+def abs(x): pass
+def round(number, ndigits=None): pass
+def all(iterable): pass
+def any(iterable): pass
+def iter(obj, *args): pass
+def next(iterator, *args): pass
+def hasattr(obj, name): pass
+def getattr(obj, name, *args): pass
+def setattr(obj, name, value): pass
+def delattr(obj, name): pass
+def callable(obj): pass
+def repr(obj): pass
+def format(value, spec=""): pass
+def id(obj): pass
+def hash(obj): pass
+def vars(*args): pass
+def dir(*args): pass
+def input(*args): pass
+def staticmethod(func): pass
+def classmethod(func): pass
+def property(*args): pass
+
+class object: pass
+class type: pass
+class str: pass
+class int: pass
+class float: pass
+class bool: pass
+class complex: pass
+class list: pass
+class dict: pass
+class set: pass
+class frozenset: pass
+class tuple: pass
+class bytes: pass
+class bytearray: pass
+
+class BaseException: pass
+class Exception: pass
+class ValueError: pass
+class TypeError: pass
+class KeyError: pass
+class IndexError: pass
+class AttributeError: pass
+class StopIteration: pass
+class RuntimeError: pass
+class NotImplementedError: pass
+class ZeroDivisionError: pass
+class NameError: pass
+class ImportError: pass
+class OSError: pass
+class FileNotFoundError: pass
+class KeyboardInterrupt: pass
+"#;
+
+/// Builds a `StackGraph` holding definitions for `PYTHON_BUILTINS_STUB`,
+/// reusing the *same* compiled TSG rules (`sgl`) the upstream crate uses for
+/// ordinary files — no grammar patch needed. The FILE_PATH `"<builtins>.py"`
+/// is the load-bearing part: the grammar's per-file module-path rule (the
+/// branch that turns a file's relative path into a `pop_symbol` chain
+/// anchored at ROOT_NODE) turns this exact path into a single
+/// `pop_symbol = "<builtins>"` node hanging directly off ROOT_NODE — which is
+/// precisely the counterpart every file's `push_symbol = "<builtins>"`
+/// fallback edge (the one stack-graphs.tsg wires up for any reference that
+/// falls through local scope) was missing.
+fn build_python_builtins_graph(sgl: &StackGraphLanguage) -> anyhow::Result<StackGraph> {
+    let mut graph = StackGraph::new();
+    let file = graph.get_or_create_file("<builtins>.py");
+
+    let mut globals = Variables::new();
+    globals
+        .add("FILE_PATH".into(), "<builtins>.py".into())
+        .map_err(|_| anyhow::anyhow!("Failed to set FILE_PATH global for builtins"))?;
+
+    let deadline = TsgCancelAfterDuration::new(RESOLVE_TIMEOUT);
+    sgl.build_stack_graph_into(&mut graph, file, PYTHON_BUILTINS_STUB, &globals, &deadline)
+        .map_err(|e| anyhow::anyhow!("Failed to build Python builtins stack graph: {e:?}"))?;
+
+    Ok(graph)
+}
+
 /// Same as `ForwardPartialPathStitcher::find_minimal_partial_path_set_in_file`
 /// (stack-graphs 0.14), but with `max_work_per_phase` bounded — see
 /// `MAX_WORK_PER_PHASE` for why.
@@ -150,11 +255,16 @@ impl FormalResolver {
     pub fn load_python(&mut self) -> anyhow::Result<()> {
         let lc = tree_sitter_stack_graphs_python::try_language_configuration(cancellation_flag())
             .map_err(|e| anyhow::anyhow!("Failed to load Python stack-graphs config: {e}"))?;
+        // Upstream's own `lc.builtins` is built from its bundled (empty)
+        // src/builtins.py — replace it with our own, built through the same
+        // `sgl` rules. See `build_python_builtins_graph` for why this alone
+        // is enough to make builtins resolve, with no grammar patch.
+        let builtins = build_python_builtins_graph(&lc.sgl)?;
         self.configs.insert(
             "python".to_string(),
             FormalLanguageConfig {
                 sgl: lc.sgl,
-                builtins: lc.builtins,
+                builtins,
                 no_similar_paths_in_file: lc.no_similar_paths_in_file,
             },
         );
@@ -354,18 +464,10 @@ def bar():
     /// unnoticed since it was never touched.
     ///
     /// NOTE on scope: this only verifies the merge happens correctly (file
-    /// count grows, no `add_from_graph` error). It deliberately does NOT
-    /// assert that a Python builtin like `len()` resolves end-to-end —
-    /// investigating that surfaced two independent upstream gaps in the
-    /// pinned `tree-sitter-stack-graphs-python` 0.3.0: (1) its bundled
-    /// `src/builtins.py` is empty (`include_str!` yields 0 bytes), and (2)
-    /// even with a synthetic non-empty builtins file built through the same
-    /// real `sgl` rules, `stack-graphs.tsg`'s `global -> ROOT_NODE` edge
-    /// pushes the symbol `"<builtins>"` but no node anywhere in that grammar
-    /// pops `"<builtins>"` — the binding is a dead end at the grammar level,
-    /// not something this merge can route around. Fixing that needs a newer/
-    /// different `tree-sitter-stack-graphs-python` version or a project-
-    /// authored builtins.py + matching tsg rule, both out of scope here.
+    /// count grows, no `add_from_graph` error) — DEBT-005 covers the actual
+    /// builtin *resolution* (see `test_resolve_file_resolves_python_builtins`
+    /// below), since `config.builtins` here is now `ci`'s own
+    /// `build_python_builtins_graph` output, not upstream's empty one.
     #[test]
     fn test_resolve_file_merges_builtins_without_error() {
         let mut resolver = FormalResolver::new();
@@ -378,7 +480,7 @@ def bar():
         // would pass vacuously.
         assert!(
             builtins_file_count > 0,
-            "builtins graph should contain at least the <builtins> file, even though its source is empty"
+            "builtins graph should contain at least the <builtins> file"
         );
 
         let mut graph = StackGraph::new();
@@ -401,6 +503,60 @@ def bar():
         );
     }
 
+    /// DEBT-005: `len()` and `print()` must resolve to the synthetic
+    /// `build_python_builtins_graph` definitions through the `formal` tier —
+    /// the actual fix, not just "the merge doesn't crash" (see
+    /// `test_resolve_file_merges_builtins_without_error` above).
+    #[test]
+    fn test_resolve_file_resolves_python_builtins() {
+        let mut resolver = FormalResolver::new();
+        resolver.load_python().unwrap();
+
+        let edges = resolver
+            .resolve_file(
+                "python",
+                "test.py",
+                "def use_builtins():\n    print(len([1, 2, 3]))\n",
+            )
+            .unwrap();
+
+        assert!(
+            edges
+                .iter()
+                .any(|e| e.reference_symbol == "len" && e.definition_symbol == "len"),
+            "len() must resolve through the formal tier. Edges: {edges:?}"
+        );
+        assert!(
+            edges
+                .iter()
+                .any(|e| e.reference_symbol == "print" && e.definition_symbol == "print"),
+            "print() must resolve through the formal tier. Edges: {edges:?}"
+        );
+    }
+
+    /// A genuinely undefined name must still fail to resolve — the builtins
+    /// fix must not make FormalResolver resolve *everything*.
+    #[test]
+    fn test_resolve_file_does_not_resolve_undefined_name() {
+        let mut resolver = FormalResolver::new();
+        resolver.load_python().unwrap();
+
+        let edges = resolver
+            .resolve_file(
+                "python",
+                "test.py",
+                "def use_undefined():\n    return totally_undefined_xyz()\n",
+            )
+            .unwrap();
+
+        assert!(
+            !edges
+                .iter()
+                .any(|e| e.reference_symbol == "totally_undefined_xyz"),
+            "a genuinely undefined name must not resolve. Edges: {edges:?}"
+        );
+    }
+
     #[test]
     fn test_resolve_python_no_refs() {
         let mut resolver = FormalResolver::new();

docs/pattern-debt-registry.yaml

@@ -54,24 +54,27 @@ items:
     owner_hint: testing
 
   DEBT-005-formal-python-builtins-unresolved:
-    status: open
-    urgency: low
+    status: resolved
+    urgency: done
+    resolved_at: "2026-07-01"
     description: >
       FormalResolver (formal.rs) không resolve được Python builtins (len, print,
-      range...). Root cause đã điều tra trong test_resolve_file_merges_builtins_
-      without_error (formal.rs): src/builtins.py bundled trong
+      range...). Root cause: src/builtins.py bundled trong
       tree-sitter-stack-graphs-python 0.3.0 rỗng, và stack-graphs.tsg push symbol
-      "<builtins>" nhưng không rule nào pop lại — dead-end ở tầng grammar upstream,
-      không sửa được từ phía merge builtins graph. Verified 2026-07-01: 0.3.0 vẫn
-      là bản mới nhất (12/2024), repo github/stack-graphs đã bị archive 9/2025 nên
-      sẽ không có upstream fix. Không ảnh hưởng correctness — reference gọi
-      builtin fall back về resolved/textual tier bình thường, chỉ mất confidence
-      tier cao nhất.
-    current_control: "README caveat dưới bullet formal tier; fallback resolver vẫn hoạt động"
+      "<builtins>" mà không có source builtins nào tận dụng đúng cơ chế per-file
+      module-path để pop lại nó. Fix: build_python_builtins_graph() tự build
+      StackGraph cho một PYTHON_BUILTINS_STUB (~70 builtin phổ biến) bằng đúng
+      `sgl` upstream, với FILE_PATH="<builtins>.py" — path này khớp regex
+      "([^/]+)\.py$" của chính stack-graphs.tsg, tự nhiên tạo ra node
+      pop_symbol="<builtins>" gắn thẳng vào ROOT_NODE. KHÔNG cần vá/fork TSG rule
+      nào — chỉ cần gọi đúng API public (build_stack_graph_into) với nội dung và
+      FILE_PATH phù hợp. Verified bằng test thật (test_resolve_file_resolves_
+      python_builtins) + smoke test CLI thật (index file dùng len/print/enumerate/
+      isinstance/sorted → call_sites.confidence='formal' cho cả 5).
+    current_control: "build_python_builtins_graph() trong formal.rs; 2 regression test (resolves builtins + vẫn từ chối tên thật sự undefined)"
     remaining:
-      - "Xác nhận tree_sitter_stack_graphs::StackGraphLanguage::from_str + tsg_source() đủ để tự vá TSG rule (pop \"<builtins>\") mà không cần fork toàn crate"
-      - "Tự viết builtins.py (định nghĩa len/print/range/...) + override rule TSG bù chỗ thiếu pop \"<builtins>\""
-      - "Regression test: len()/print() resolve tới definition_symbol qua formal tier, không chỉ textual"
+      - "GIỚI HẠN ĐÃ BIẾT: call_sites.confidence='formal' cho builtin KHÔNG tạo ra call_edges row — rebuild_graph() (pipeline.rs) lọc theo tên khớp symbols đã index trong project, độc lập với confidence, nên builtin (không phải project symbol) không bao giờ có call_edges dù tier nào. Nghĩa là callers/path/CallerCountByConfidence chưa thấy được thay đổi này — chỉ đúng ở tầng lưu trữ call_sites, sẵn sàng cho consumer tương lai."
+      - "Nếu muốn builtin hiển thị qua tool: cần thiết kế riêng (builtin thành pseudo-symbol?) — rủi ro thật: len/print sẽ có caller_count khổng lồ khắp mọi codebase Python, có thể làm lệch hub_count/coreness. Cố ý chưa làm, cần bàn kỹ trước khi động vào."
     owner_hint: ci-core/resolver
 
   DEBT-006-ty-subprocess-premise-invalid: