feat(context): add an adaptive surfacing pre-pass, never a pipeline bypass (Phase 4)

Everything in context/ is reactive: given a context and a task, decide
what to drop or how to order it. Nothing computed a trigger (is now the
moment to push something), a what-to-add (candidates not already in the
context), or a budget policy beyond a flat int|None from the caller.

Adds context/adaptive.py (should_surface, plan_budget, select) and
memory/surface.py (candidates_for), forced into memory/ rather than
context/ by CLAUDE.md's dependency rule. The rule that matters more than
the functions themselves: this module only ever produces a
list[ContextItem] for the caller to merge and hand to the existing
ContextPipeline.run, unchanged -- it never bypasses that pipeline or
injects between its stages. Three failure modes this prevents, each
traced to a specific prune.py mechanism: ledger corruption (only _stage
writes the token accounting), the dedup-before-classify trap (a pushed
duplicate is dropped before classify can protect it as CRITICAL), and the
missing-task trap (_enforce_budget's rank_score only populates when a
task was actually ranked against -- select() refuses to run without one
rather than silently collapse drop order).

plan_budget's conservative default (15% of window) is a direct response
to Gloaguen et al. (arXiv:2602.11988): unconditional context injection
measured elsewhere as +20% cost with no success gain.

New tests run adaptive output through the real ContextPipeline.run and
assert invariants 1 and 2 still hold post-merge, rather than inventing new
guarantees. Explicitly not done here: CLI/MCP wiring and a verity eval
pilot comparing adaptive surfacing to a no-injection control -- stated as
future work, required before any effect claim per invariant 7.

New ADR-0025 (renumbered from the plan's 0024, which Phase 3 used first).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Author: Juanpacol

docs/adr/0025-adaptive-context-prepass.md

@@ -0,0 +1,95 @@
+# ADR-0025: An adaptive context pre-pass — proactive, but never a bypass
+
+- **Status**: Accepted
+- **Date**: 2026-08-10
+- **Note on numbering**: the implementation plan this work follows
+  originally reserved ADR-0024 for this decision; Phase 3 (Reality Check
+  expansion) used 0024 first, so this is 0025. Risk-Adaptive Verification
+  (a later phase) shifts to 0026 for the same reason.
+- **Context**: everything in `context/` is reactive — given a context and a
+  task, `prune.py`/`rank.py`/`classify.py` decide what to drop or how to
+  order it. An engineering map done to scope the "Adaptive Context Engine"
+  proposal found three things genuinely missing: no function computes a
+  **trigger** (is now the moment to push something), no function returns
+  **candidates not already in the context**, and no **budget policy**
+  beyond a flat `int | None` passed in by the caller.
+
+## Decision
+
+Add `context/adaptive.py` as a thin pre-pass — three pure functions,
+`should_surface`, `plan_budget`, `select` — plus `memory/surface.py` for
+candidate sourcing (forced into `memory/` rather than `context/` by
+CLAUDE.md's dependency rule: `context/` must never import `memory/`).
+
+**The rule that matters more than the functions themselves:** this module
+only ever produces a `list[ContextItem]` for the caller to merge with the
+rest of the context and hand to the existing `ContextPipeline.run`,
+unchanged. It never bypasses that pipeline and never injects between its
+stages. Three concrete failure modes this rule prevents, each traced to a
+specific mechanism in `prune.py`:
+
+1. **Ledger corruption (invariant 2).** `ContextPipeline._stage` is the
+   sole writer of the token ledger — each stage's `tokens_before` is
+   recomputed from exactly the list the previous stage returned. Injecting
+   items outside that chokepoint (before `measured`, between two `_stage`
+   calls, or after `_place`) corrupts `tokens_before`, `tokens_after`, or
+   both, and therefore `reduction_ratio` — this project's headline number.
+2. **The dedup-before-classify trap.** `dedup` runs first and keys on
+   `content_hash`. A pushed memory record duplicating existing context text
+   is dropped at that stage, before `classify` ever gets to protect it as
+   `CRITICAL` — so a naive merge-then-classify assumption silently loses
+   exactly the content a push was meant to add.
+3. **The missing-task trap.** `_enforce_budget` sorts candidates by
+   `rank_score`, populated only when the rank stage actually ran against a
+   non-empty `task` (`ContextPipeline.run` skips ranking entirely for an
+   empty task). `select()` refuses to run with an empty task rather than
+   silently populate `rank_score=0` for everything and collapse drop order
+   to newest-first — covered by
+   `test_no_task_is_refused_rather_than_silently_scored_zero`.
+
+## What each function does
+
+- **`should_surface(health) -> ContextTrigger | None`** — pure over
+  `compute_health`'s existing output. Fires on high window usage (≥75%) or
+  low relevant ratio (≤50%); both thresholds are stated as round numbers
+  with no pilot behind them yet, the same honesty `deterministic.py`'s
+  `_SUSPICIOUS_DUPLICATE_SHARE` states about itself.
+- **`plan_budget(counter, health, ratio=0.15) -> BudgetPlan`** — `basis` is
+  never omitted, mirroring invariant 3's rule for `TokenCount`. The default
+  ratio (15% of the window) is deliberately conservative: Gloaguen et al.
+  (arXiv:2602.11988) found unconditional repository-level context injection
+  raised inference cost over 20% with no task-success gain, and a careless
+  adaptive-push policy risks reproducing exactly that finding instead of
+  avoiding it.
+- **`select(candidates, task, plan, ranker) -> SurfaceDecision`** — ranks
+  candidates against `task` via the existing `ContextRanker`, then keeps
+  what fits `plan.budget` greedily in rank order. Returns items that are
+  RANKED, not yet BUDGETED or PROTECTED — that happens only once the caller
+  hands the merged list to `ContextPipeline.run`.
+- **`memory/surface.py::candidates_for(store, task, counter)`** — converts
+  active decisions, hard/soft constraints, discoveries, and unresolved
+  failures into `ItemKind.MEMORY` items, which `classify.py:230-231`
+  already protects as `CRITICAL` unconditionally. Uses a local content hash
+  rather than importing `context/classify.py`'s, since only
+  `memory -> context.tokenizer` is a declared dependency edge — adding
+  `memory -> context.classify` would be an undeclared one.
+
+## Consequences
+
+- `core/models.py` gains `ContextTrigger`, `BudgetPlan`, `SurfaceDecision`
+  — no existing model changed.
+- New tests (`test_adaptive.py`, `test_memory_surface.py`) include two that
+  run adaptive output through the real `ContextPipeline.run` and assert
+  invariant 1 (critical retention) and invariant 2 (ledger chaining) still
+  hold post-merge — not new guarantees this module invents, but
+  confirmation that it doesn't break the ones that already exist.
+- **Not done here, and stated plainly:** this ADR does not wire
+  `should_surface`/`select` into `cli/main.py` or `mcp/server.py`, and does
+  not run a `verity eval` pilot comparing adaptive-surfacing against a
+  no-injection control. Both are natural next steps — the second is
+  actually necessary before any claim about this mechanism's effect could
+  be published, per invariant 7 (Phase 0) — but they are future work, not
+  claimed as complete by adding these three functions.
+- The threshold constants (`_HIGH_WINDOW_USAGE`, `_LOW_RELEVANT_RATIO`,
+  `_DEFAULT_BUDGET_RATIO`) are placeholders pending exactly that pilot —
+  they should not be read as tuned values.

docs/adr/README.md

@@ -42,6 +42,7 @@ verdict was "this pilot could not have detected an effect."
 | [0022](0022-verity-eval-harness.md) | Can a real trial harness replace hand-run, hand-scored pilots? | Yes — `verity eval` retains a content-hashed artifact per trial and flags degenerate noise floors instead of hiding them; reproduced pilot 8's result exactly |
 | [0023](0023-memory-surfacing-log.md) | Can this project measure *when* memory was surfaced, not just when it was written? | Yes, narrowly — a new `Surfacing` record, emitted from `build_handoff` and decision resurfacing; "was it used" stays honestly unresolved except one negative signal |
 | [0024](0024-reality-check-expansion.md) | Can Agent Reality Check widen its recall without repeating ADR-0021's mistake? | Yes — `imports`, negation, multi-target relations, and constraints-as-evidence, each narrow and each declining to guess where the graph has no adjudicating edge |
+| [0025](0025-adaptive-context-prepass.md) | Can Verity proactively surface context without breaking the prune pipeline's invariants? | Yes, as a pre-pass only — `context/adaptive.py` + `memory/surface.py` merge into `ContextPipeline.run` unchanged; wiring and a measured pilot are stated as future work, not done here |
 
 ## Pre-pivot (superseded)
 

src/verityai/context/adaptive.py

@@ -0,0 +1,145 @@
+"""A proactive pre-pass: decide what to surface and when, before pruning.
+
+Everything in `prune.py`, `rank.py`, and `classify.py` is reactive -- given
+a context and a task, decide what to drop or how to order it. Nothing
+computes a *trigger* (is now the moment to push something), a *what-to-add*
+(candidates not already in the context), or a *budget policy* (how much
+room a push gets, and why). This module is that missing layer, and it is
+deliberately thin: every function here is pure, and the one rule that
+matters more than any of them is in `select()`'s docstring below.
+
+**Hard rule, non-negotiable (ADR-0025):** this module only ever produces a
+`list[ContextItem]` for the caller to hand to the existing
+`ContextPipeline.run`, unchanged. It never bypasses that pipeline and never
+injects items between its stages. `prune.py`'s ledger chains (invariant 2)
+only because `ContextPipeline._stage` is the sole writer of token
+accounting; injecting outside that chokepoint corrupts `tokens_before`,
+`tokens_after`, and therefore `reduction_ratio` -- the project's headline
+number. `context/` must never import `memory/` (CLAUDE.md's dependency
+rule), so candidate *sourcing* lives in `memory/surface.py`; this module
+only ranks and budgets what it is handed.
+"""
+
+from verityai.context.rank import ContextRanker
+from verityai.context.tokenizer import TokenCounter
+from verityai.core.models import (
+    BudgetPlan,
+    ContextHealth,
+    ContextItem,
+    ContextTrigger,
+    SurfaceDecision,
+)
+
+# Health thresholds that justify a proactive push. Chosen the same way
+# `bench/deterministic.py`'s `_SUSPICIOUS_DUPLICATE_SHARE` was: a round
+# number that names a real condition worth looking at, not an empirically
+# tuned cutoff -- there is no pilot behind these yet (see ADR-0025's stated
+# limits).
+_HIGH_WINDOW_USAGE = 0.75
+_LOW_RELEVANT_RATIO = 0.5
+
+# Default share of the window a push may use. Conservative on purpose: the
+# AGENTS.md finding this project's own research review cites (arXiv:2602.11988)
+# is that unconditional context injection cost over 20% with no success
+# gain -- a budget policy that pushes too eagerly risks reproducing exactly
+# that result instead of avoiding it.
+_DEFAULT_BUDGET_RATIO = 0.15
+
+
+def should_surface(health: ContextHealth) -> ContextTrigger | None:
+    """Does this context's health justify pushing something into it now?
+
+    Pure over `compute_health`'s existing output -- no new state, no new
+    measurement. Returns `None` when nothing in `health` crosses a
+    threshold; a caller must not surface anything without a stated reason,
+    the same discipline every other degraded/triggered path in this
+    codebase follows (invariant 5).
+    """
+    if health.window_usage >= _HIGH_WINDOW_USAGE:
+        return ContextTrigger(
+            reason=f"window usage {health.window_usage:.0%} >= {_HIGH_WINDOW_USAGE:.0%}",
+            health_snapshot={"window_usage": health.window_usage},
+        )
+    if health.relevant_ratio <= _LOW_RELEVANT_RATIO:
+        return ContextTrigger(
+            reason=f"relevant ratio {health.relevant_ratio:.0%} <= {_LOW_RELEVANT_RATIO:.0%}",
+            health_snapshot={"relevant_ratio": health.relevant_ratio},
+        )
+    return None
+
+
+def plan_budget(
+    counter: TokenCounter,
+    health: ContextHealth,
+    ratio: float = _DEFAULT_BUDGET_RATIO,
+) -> BudgetPlan:
+    """How many tokens a proactive push gets, and the reasoning behind it.
+
+    `basis` is never omitted -- a budget number without it is exactly the
+    bare-int mistake invariant 3 exists to prevent for `TokenCount`, applied
+    to a different field.
+    """
+    window = counter.window
+    budget = max(0, int(window * ratio))
+    return BudgetPlan(
+        budget=budget,
+        window=window,
+        basis=(
+            f"{ratio:.0%} of the {window:,}-token window, deliberately conservative "
+            "(Gloaguen et al., arXiv:2602.11988, found unconditional repository-level "
+            "context injection raised inference cost >20% with no task-success gain; "
+            "see ADR-0025)"
+        ),
+    )
+
+
+def select(
+    candidates: list[ContextItem],
+    task: str,
+    plan: BudgetPlan,
+    ranker: ContextRanker | None = None,
+) -> SurfaceDecision:
+    """Rank `candidates` against `task` and keep what fits `plan.budget`.
+
+    Requires a non-empty `task`: `prune.py`'s `_enforce_budget` sorts by
+    `rank_score`, which is only populated when ranking actually ran against
+    a task (`ContextPipeline.run` skips the rank stage entirely for an
+    empty task). A caller that surfaces items with no task string gets
+    `rank_score` defaulting to zero for all of them, which silently
+    collapses drop order to newest-first -- this function refuses that
+    case outright rather than let it happen invisibly.
+
+    The returned items are RANKED, not yet BUDGETED or PROTECTED: this is a
+    pre-pass. The caller must still hand `SurfaceDecision.items` (merged
+    with the rest of the context) to `ContextPipeline.run`, which is what
+    actually classifies, dedups, and enforces the budget. Selecting here and
+    skipping that step would bypass every invariant that pipeline provides.
+    """
+    if not task:
+        return SurfaceDecision(
+            items=[],
+            plan=plan,
+            degraded_reason=(
+                "no task provided -- ranking against nothing would score every "
+                "candidate zero and select in an undefined order"
+            ),
+        )
+    if not candidates:
+        return SurfaceDecision(items=[], plan=plan, degraded_reason=None)
+
+    ranker = ranker or ContextRanker()
+    ranking = ranker.rank(task, candidates)
+
+    selected: list[ContextItem] = []
+    used = 0
+    for scored in ranking.items:
+        if used + scored.item.token_count > plan.budget:
+            continue
+        selected.append(scored.item)
+        used += scored.item.token_count
+
+    return SurfaceDecision(
+        items=selected,
+        plan=plan,
+        degraded_reason=ranking.degraded_reason,
+    )

src/verityai/core/models.py

@@ -800,3 +800,46 @@ class TrialRecord(BaseModel):
     @property
     def succeeded(self) -> bool:
         return self.scorer_exit_code == 0
+
+
+# --- Adaptive context (proactive surfacing pre-pass) ------------------------
+
+
+class ContextTrigger(BaseModel):
+    """The observation that decided *now* is the moment to surface memory.
+
+    Mirrors `relevance_reason` on `ContextItem`: a trigger is always
+    auditable, never a bare boolean. `should_surface` (`context/adaptive.py`)
+    is the only producer of these.
+    """
+
+    reason: str
+    health_snapshot: dict[str, float] = Field(default_factory=dict)
+
+
+class BudgetPlan(BaseModel):
+    """How much room a proactive surfacing pass has to work with, and why.
+
+    `basis` exists for the same reason `TokenCount` is a pair, never a bare
+    int (invariant 3): a budget number without the reasoning that produced
+    it invites more confidence than it earned. Never a bare `int`.
+    """
+
+    budget: int
+    window: int
+    basis: str
+
+
+class SurfaceDecision(BaseModel):
+    """What an adaptive pre-pass chose to add, and under what plan.
+
+    Deliberately holds `items` for `ContextPipeline.run` to consume
+    unchanged (ADR-0025) -- this is a pre-pass's output, never a substitute
+    for running the pipeline. `degraded_reason` follows the same rule as
+    every other degraded path in this codebase (invariant 5).
+    """
+
+    items: list[ContextItem] = Field(default_factory=list)
+    trigger: ContextTrigger | None = None
+    plan: BudgetPlan | None = None
+    degraded_reason: str | None = None

src/verityai/memory/surface.py

@@ -0,0 +1,74 @@
+"""Candidate context items sourced from memory, for a proactive surfacing pass.
+
+Lives in `memory/`, not `context/`, because CLAUDE.md's dependency rule
+says `context/` must never import `memory/` — ranking a context and
+persisting a decision are independent operations, and an adaptive
+"decide what to push" pre-pass (`context/adaptive.py`, ADR-0025) still
+needs *something* to source candidates from. This module is that source:
+it turns active memory records into `ContextItem`s, and `adaptive.py` hands
+the result to the existing `ContextPipeline.run` unchanged. Nothing here
+ranks, prunes, or decides a budget — that stays in `context/`.
+"""
+
+import hashlib
+
+from verityai.context.tokenizer import TokenCounter
+from verityai.core.models import ContextItem, ItemKind
+from verityai.memory.store import MemoryStore
+
+
+# Local, not `context/classify.py`'s `content_hash` -- importing it would
+# add a `memory -> context.classify` edge the dependency table does not
+# declare (only `memory -> context.tokenizer` is, for handoff's token
+# budget). `ContextItem.content_hash` only needs to be a stable identifier
+# for dedup, not byte-identical to classify.py's own hashing scheme.
+def _content_hash(text: str) -> str:
+    normalized = " ".join(text.split())
+    return hashlib.sha256(normalized.encode("utf-8")).hexdigest()[:16]
+
+
+def candidates_for(
+    store: MemoryStore,
+    task: str,
+    counter: TokenCounter | None = None,
+) -> list[ContextItem]:
+    """Active decisions, constraints, discoveries, and unresolved failures,
+    as `ContextItem`s -- candidates for a proactive surfacing pass to offer
+    `context/adaptive.py::select`.
+
+    Every item is `ItemKind.MEMORY`, which `classify.py:230-231` already
+    classifies `CRITICAL` unconditionally -- these records exist specifically
+    because they must not be silently dropped, so this reuses that existing
+    protection rather than inventing a new one. `task` is accepted for a
+    future ranking pass to use; this function itself does not rank or filter
+    by relevance, only converts what memory has into the shape `context/`
+    already knows how to measure and protect.
+    """
+    counter = counter or TokenCounter()
+
+    records: list[tuple[str, str]] = []
+    for decision in store.decisions():
+        records.append((f"decision: {decision.statement}", str(decision.id)))
+    for constraint in store.constraints():
+        marker = "hard constraint" if constraint.hard else "soft constraint"
+        records.append((f"{marker}: {constraint.statement}", str(constraint.id)))
+    for discovery in store.discoveries():
+        records.append((f"discovery: {discovery.statement}", str(discovery.id)))
+    for failure in store.failures(include_resolved=False):
+        records.append((f"already tried, did not work: {failure.attempted}", str(failure.id)))
+
+    items: list[ContextItem] = []
+    for index, (text, record_id) in enumerate(records):
+        count = counter.count(text)
+        items.append(
+            ContextItem(
+                kind=ItemKind.MEMORY,
+                content=text,
+                token_count=count.tokens,
+                token_method=count.method,
+                original_index=index,
+                content_hash=_content_hash(text),
+                metadata={"task": task, "record_id": record_id},
+            )
+        )
+    return items