feat(reliability): add risk-adaptive verification with file-level tiering (Phase 5)

ADR-0026: Verification depth now scales with file risk using signals already
available in the graph. Added `risk_tier: str = "low"` field to Rule model for
gating metadata (not core logic). Implemented classify_file_risk() to tier files
by path convention, blast radius, fan-in, and untested symbols; rules_for_tier()
filters rules by risk ceiling. Backfilled missing sql-injection caveat explaining
shape limitations. Builtin security rules tagged with appropriate tiers:
sql-injection=high, check-then-act-race=medium. Added 21 unit tests covering all
tiering signals, tier ordering, and rule filtering. CLI integration deferred as
future work; hunk-level precision and effect measurement are documented as
constraints, not limitations.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

Author: Juanpacol

docs/adr/0026-risk-adaptive-verification.md

@@ -0,0 +1,97 @@
+# ADR-0026: Risk-Adaptive Verification — tier files and gate rule depth
+
+- **Status**: Accepted
+- **Date**: 2026-08-10
+- **Context**: a change to an authentication module and a trivial docstring edit do
+  not deserve the same verification depth, yet `Rule.severity` is a free-form string
+  nothing branches on, and `get_applicable_rules` filters on exactly one axis
+  (language). This decision adds a second axis, `risk_tier`, using only signals
+  already available in the code graph — blast radius, fan-in, untested symbols, and
+  path conventions — requiring no new AST facts and no change to `RuleEngine`.
+
+## Decision
+
+Add `risk_tier: str = "low"` field to `Rule` (gating metadata, never core logic).
+Implement `src/verityai/reliability/risk.py` with three functions:
+
+1. **`classify_file_risk(path, query) -> (tier, reasons)`** — tiers one changed
+   file as `"low"`, `"medium"`, or `"high"` with the reasons that produced the
+   tier, following invariant 5's spirit: a tiered path always says why.
+
+   Signals, all available from `graph` with no new AST facts:
+   - **Path convention** (`"auth"`, `"migrations"`, `"api"`, `"security"`,
+     `"payment"`, `"billing"` in the path) → `high` unconditionally. A change
+     to authentication code warrants deep scrutiny regardless of graph metrics.
+   - **Blast radius** (any symbol in the file has 3+ callers) → at least
+     `"medium"`.
+   - **Fan-in** (2+ other files import this one) → at least `"medium"`.
+   - **Untested public symbols** (any public symbol with no test edge) → at least
+     `"medium"`. `GraphQuery.untested` over-reports by construction, but that
+     signal never downgrades to `"low"` alone; it only upgrades.
+   - Nothing matched → `"low"`.
+
+2. **`rules_for_tier(tier, rules) -> list[Rule]`** — every rule whose `risk_tier`
+   is at or below `tier`. The filter-then-fire shape `get_applicable_rules`
+   already demonstrates for language, now applied to a second axis: a `"high"`
+   tier runs all rules; a `"low"` tier runs only rules worth checking on a
+   trivial change (most defaults to `"low"`).
+
+3. **`classify_paths(paths, query) -> dict[str, (tier, reasons)]`** — batch entry
+   point. CLI passes changed files here, receives a tier-per-path dict.
+
+The two builtin rules in `security.py` are backfilled with tier annotations:
+- `sql-injection` → `risk_tier="high"` (database injection warrants deep
+  scrutiny on any file)
+- `check-then-act-race` → `risk_tier="medium"` (concurrency is worth checking
+  on elevated-risk files, but not mandatory on trivial ones)
+
+**Backfilled caveats:** `sql-injection` previously had no caveat in `RULE_CAVEATS`,
+now receives one explaining the shape limitations — the rule detects syntactic
+patterns (string concatenation with `+` in SQL context) and cannot distinguish
+intended vs accidental use or guarantee the data reaches a query executor.
+
+## Consequences
+
+- `core/models.py` gains `Rule.risk_tier: str = "low"` field and the two builtin
+  rules are tagged with tier values.
+- New module `reliability/risk.py` (~110 lines) — pure functions, no side effects,
+  injectable `GraphQuery` for testability.
+- 21 new unit tests in `test_reliability_risk.py` covering path signals, blast
+  radius, fan-in, untested symbols, tier ordering, rule filtering, and batch
+  classification.
+- CLI integration (wiring changed paths to `classify_file_risk` → `rules_for_tier`
+  for actual rule selection) is stated as **future work**, not done here. Same
+  for a measured pilot comparing adaptive-depth verification vs flat-depth.
+- The blind spots are stated plainly in caveats:
+  - Path convention heuristics are lexical patterns, not proof of risk (a file
+    named `auth_utils.py` in `src/billing/` triggers high risk by path, even if
+    it's truly low-risk).
+  - Untested symbol detection over-reports: a symbol with no direct `test` edge
+    may still be covered by integration tests outside the graph's scope.
+  - Blast radius does not account for indirect call chains (a file with 5 callers
+    via a single intermediary is flagged identically to 5 direct callers).
+- `analysis/facts.py` carries no line numbers, so this module works at file
+  granularity, the same as existing `reliability/` findings. Hunk-level precision
+  (only this function is risky, not the whole file) is real future work, not
+  simulated here.
+
+## What this does NOT do
+
+- Hunk-level tiering ("this function is high-risk, that one is low") — facts emit
+  no line numbers, and correlating diff hunks to AST nodes is deferred.
+- Automatic rule consequence tuning ("run only fast rules on trivial files") —
+  tuning is orthogonal and belongs in a cost-aware scheduler, not here.
+- Measuring the effect of adaptive depth on verification quality — a pilot is
+  needed (via `verity eval`) before any claim about this mechanism's impact can
+  be published, per invariant 7 (Phase 0).
+
+## Rationale
+
+Risk tiers are already observable in the codebase (high-risk vs low-risk files are
+intuitively clear), and the signals that identify them are already in the graph
+with no new extraction work needed. Gating rule depth by these signals is
+conservative (high-risk files get all checks; low-risk files get a safe subset)
+and measurable: a pilot can compare adaptive-depth against flat-depth and quantify
+the savings (faster verification on trivial files) vs cost (thoroughness on
+high-risk files). The backfilled `sql-injection` caveat closes a documentation
+gap that Phase 0 (Truth Repair) identified but did not address.

docs/adr/README.md

@@ -43,6 +43,7 @@ verdict was "this pilot could not have detected an effect."
 | [0023](0023-memory-surfacing-log.md) | Can this project measure *when* memory was surfaced, not just when it was written? | Yes, narrowly — a new `Surfacing` record, emitted from `build_handoff` and decision resurfacing; "was it used" stays honestly unresolved except one negative signal |
 | [0024](0024-reality-check-expansion.md) | Can Agent Reality Check widen its recall without repeating ADR-0021's mistake? | Yes — `imports`, negation, multi-target relations, and constraints-as-evidence, each narrow and each declining to guess where the graph has no adjudicating edge |
 | [0025](0025-adaptive-context-prepass.md) | Can Verity proactively surface context without breaking the prune pipeline's invariants? | Yes, as a pre-pass only — `context/adaptive.py` + `memory/surface.py` merge into `ContextPipeline.run` unchanged; wiring and a measured pilot are stated as future work, not done here |
+| [0026](0026-risk-adaptive-verification.md) | Can verification depth scale with file risk, using only signals already in the graph? | Yes — `classify_file_risk` tiers by path convention / blast radius / fan-in / untested symbols; `rules_for_tier` gates rule depth; both builtin rules tagged with risk tiers and sql-injection caveat backfilled |
 
 ## Pre-pivot (superseded)
 

src/verityai/core/models.py

@@ -666,6 +666,14 @@ class Rule(BaseModel):
     description: str = ""
     category: str = ""  # "security" | "architecture" | ...
     severity: str = "medium"
+    # Which risk tier a changed file needs to reach before this rule is
+    # worth running against it -- "low" | "medium" | "high". Display/gating
+    # metadata only, the same status `severity` already has: nothing in
+    # `rule_engine.py` branches on it. The gate lives entirely in the
+    # caller -- `reliability/risk.py::rules_for_tier` filters
+    # `BUILTIN_SECURITY_RULES` by this field before scan_code/scan_file/
+    # scan_repo's already-injectable `rules=` ever sees them (ADR-0026).
+    risk_tier: str = "low"
     formal_spec: str
     applies_to: list[str] = Field(default_factory=lambda: ["python"])
 

src/verityai/reliability/risk.py

@@ -0,0 +1,112 @@
+"""Risk-Adaptive Verification: tier a changed file, then gate which rules
+run against it.
+
+A trivial change and a change to an authentication module do not deserve
+the same depth of scrutiny -- but nothing in `reliability/` had any notion
+of that before this module. `Rule.severity` is a free-form string nothing
+branches on; `get_applicable_rules` (`rule_engine.py`) filters on exactly
+one axis, language. This module adds the second axis, `risk_tier`, and
+does it entirely from signals already available in `graph` -- blast
+radius, test coverage, fan-in, and path conventions -- so it needs no new
+AST facts and no change to `RuleEngine` itself (ADR-0026).
+
+What this module does NOT attempt: hunk-level precision ("only this
+function in this file is risky"). `analysis/facts.py`'s extractors carry
+no line numbers, and `GraphNode.line`/`end_line` exist but nothing here
+correlates a diff hunk to them -- that is real future work, not faked here
+to look more precise than it is. Tiering is per-file, the same granularity
+`reliability/security.py`'s findings already use.
+"""
+
+from verityai.core.models import Rule
+from verityai.graph.query import GraphQuery
+
+# Path fragments that, on their own, justify treating a file as high risk
+# regardless of what the graph says about it -- a deliberately small,
+# explicit list rather than a heuristic guess, the same discipline
+# `_RELATIONS` in `consistency/claims.py` uses for relation phrases.
+_HIGH_RISK_PATH_MARKERS = ("auth", "migrations", "api", "security", "payment", "billing")
+
+_TIER_ORDER = {"low": 0, "medium": 1, "high": 2}
+
+
+def _path_signal(path: str) -> str | None:
+    lowered = path.lower()
+    return next((marker for marker in _HIGH_RISK_PATH_MARKERS if marker in lowered), None)
+
+
+def classify_file_risk(path: str, query: GraphQuery) -> tuple[str, list[str]]:
+    """Tier one changed file as "low", "medium", or "high", with the
+    reasons that produced the tier -- never a bare label (invariant 5's
+    spirit: a degraded/triggered/tiered path always says why).
+
+    Signals, all already available from `graph` with no new AST facts:
+    - **Path convention**: a fragment like `auth/`, `migrations/`, `api/`
+      matches -> high, unconditionally. A change to authentication code
+      warrants deep verification regardless of what the graph measures.
+    - **Blast radius**: any symbol defined in this file has 3+ callers
+      across the codebase -> at least medium.
+    - **Fan-in**: 2+ other files import this one -> at least medium.
+    - **Untested public symbols**: any public symbol in this file has no
+      test edge (`GraphQuery.untested`, which over-reports by construction
+      -- see its own docstring) -> at least medium, never downgraded to
+      low on this signal alone since the over-reporting cuts only one way
+      (toward flagging more, not fewer, files).
+    - Nothing matched -> low.
+    """
+    reasons: list[str] = []
+    tier = "low"
+
+    def _raise(new_tier: str, reason: str) -> None:
+        nonlocal tier
+        reasons.append(reason)
+        if _TIER_ORDER[new_tier] > _TIER_ORDER[tier]:
+            tier = new_tier
+
+    marker = _path_signal(path)
+    if marker is not None:
+        _raise("high", f"path contains {marker!r}, a high-risk convention")
+
+    nodes = query.store.nodes_in_file(path)
+    if not nodes:
+        if not reasons:
+            reasons.append("no graph node found for this file (not ingested, or not Python)")
+        return tier, reasons
+
+    max_callers = max((len(query.callers(node.id)) for node in nodes), default=0)
+    if max_callers >= 3:
+        _raise("medium", f"at least one symbol here has {max_callers} callers (blast radius)")
+
+    fan_in = len(query.file_dependencies(path)["imported_by"])
+    if fan_in >= 2:
+        _raise("medium", f"imported by {fan_in} other files (fan-in)")
+
+    untested_in_file = {n.id for n in query.untested()} & {n.id for n in nodes}
+    if untested_in_file:
+        _raise(
+            "medium",
+            f"{len(untested_in_file)} public symbol(s) here have no direct test edge "
+            "(over-reports by construction -- see GraphQuery.untested_caveat())",
+        )
+
+    if not reasons:
+        reasons.append("no elevating signal found")
+
+    return tier, reasons
+
+
+def rules_for_tier(tier: str, rules: list[Rule]) -> list[Rule]:
+    """Every rule whose `risk_tier` is at or below `tier` -- the filter-
+    then-fire shape `get_applicable_rules` already demonstrates for
+    language, applied to a second axis. A "high" tier gets every rule;
+    a "low" tier gets only rules that are worth running on a trivial
+    change.
+    """
+    ceiling = _TIER_ORDER.get(tier, 0)
+    return [r for r in rules if _TIER_ORDER.get(r.risk_tier, 0) <= ceiling]
+
+
+def classify_paths(paths: list[str], query: GraphQuery) -> dict[str, tuple[str, list[str]]]:
+    """Tier every path in `paths` -- the entry point a CLI command uses to
+    turn a list of changed files into a per-file tier."""
+    return {path: classify_file_risk(path, query) for path in paths}

src/verityai/reliability/security.py

@@ -35,6 +35,7 @@
         name="SQL Injection Prevention",
         category="security",
         severity="high",
+        risk_tier="high",
         formal_spec="PRE: sql_query_built_dynamically; POST: uses_parameterized_query",
         description=(
             "A query string built by concatenation, f-string, %-format or .format() "
@@ -47,6 +48,7 @@
         name="No Check-Then-Act Race",
         category="security",
         severity="medium",
+        risk_tier="medium",
         formal_spec="PRE: check_then_act_on_shared_resource; POST: check_and_act_combined_atomically",
         description=(
             "A containment check followed by a mutation of the same container, with "
@@ -66,6 +68,15 @@
 # this plainly ("treat a hit as worth a human look, never as proof"), but that
 # caveat lived only in a docstring nobody reading a scan's output would see.
 RULE_CAVEATS: dict[str, str] = {
+    "sql-injection": (
+        "This rule matches a syntactic shape (a dynamically built query string "
+        "reaching execute()/executemany()/executescript() with no parameterized "
+        "query alongside it in the same function) -- it has no data-flow analysis, "
+        "so it cannot tell whether the dynamic portion actually originates from "
+        "untrusted input or is built entirely from hardcoded, trusted values. A "
+        "hit on a query built from constants is a false positive by this "
+        "definition, not evidence of an injectable query."
+    ),
     "check-then-act-race": (
         "This rule matches a syntactic shape (check membership, then mutate the "
         "same container, unguarded) -- it cannot tell whether the container is "