[FEATURE] Knowledge-Centric Self-Improvement: knowledge base as persistent improving object with cross-model transfer

[Lexus2016]

Feature Description

Problem Statement

The evolution pipeline produces skills (executable) and issues (bug reports) but does not maintain a distilled knowledge base of cross-task heuristics, constraints, and pitfalls that any agent can consult. Improvements are currently tied to the generating model and do not transfer across model families. When the model is upgraded, previously learned heuristics are lost.

A paper (arXiv:2607.19592) demonstrates Knowledge-Centric Self-Improvement: rather than treating the agent as the object that improves, a curated knowledge base is the persistent improving object while agents remain generic and disposable. The key result: distilled knowledge transfers across held-out tasks AND across LLM families — the improvement is not tied to the generating model.

Proposed Solution

Add a complementary improvement channel: a distilled knowledge base alongside the skill library.

1. Curate a knowledge base from each research cycle: after each cycle, distill the findings into a structured knowledge asset containing cross-task heuristics, constraints, pitfalls, and validation strategies. This is the "inspectable, transferable" layer between the model and the skills.
2. Test cross-model transfer: the pipeline's skills are generated by one model and used by the same model. Knowledge-Centric SI's transfer result suggests testing whether distilled knowledge (not skills) transfers across model families — if it does, the knowledge base becomes a model-agnostic asset that survives model upgrades.
3. Task-conditioned adapter: convert the shared knowledge asset into a short memo tailored to the current task, similar to how the research skill already produces task-specific findings but from a persistent knowledge base rather than fresh research each cycle.

The protocol: agents attempt one task, contribute evidence-grounded insights to a shared knowledge base via task-level and cross-task forums, followed by knowledge distillation. The distilled knowledge is a structured asset that any agent can leverage.

Value Proposition

• Impact: Medium-High (0.7) — model-agnostic asset that compounds across model upgrades
• Effort: Low-Medium (0.2) — protocol adoption, not infrastructure
• Priority Score: 1.12

Research Evidence

• arXiv:2607.19592 — Knowledge-Centric Self-Improvement; code open-sourced
• Results: improves solve rates while reducing dollar cost across ARC-AGI-1/2, SWE-bench Pro, Terminal-Bench 2; transfers to held-out tasks and across LLM families

Implementation Plan

1. Define a knowledge-base schema (heuristics + constraints + pitfalls + validation strategies) as a JSON or markdown structure
2. After each research cycle, run a distillation step that extracts structured knowledge from the cycle findings and appends to the knowledge base
3. Add a task-conditioned adapter that selects relevant knowledge entries for the current task and produces a short memo
4. Test cross-model transfer: generate knowledge with model A, use it with model B, measure whether the knowledge base improves outcomes regardless of the model

Success Criteria

• A structured knowledge base exists alongside the skill library
• Research cycle findings are distilled into the knowledge base each cycle
• Knowledge base entries are consulted by the pipeline and improve downstream outcomes
• Cross-model transfer is tested and confirmed (knowledge generated by model A helps model B)

[Lexus2016]

Triage note from the 2026-07-27/28 backlog pass — recording the state of the backlog this proposal is joining, so it is not planned against stale assumptions.

Checked and clear: this does not duplicate anything merged in the last two days, and does not depend on the trajectory capture that #1363 blocks. It is implementable on its own terms.

Two things worth knowing before it is picked up:

1. The measurement layer moved. Three false-positive mechanisms were found and fixed in introspection_extract.py (fix: two review defects — dead-end detector silent on alternating calls, recovery metric counted deflection #1374, fix: three defects surviving round 1 — recovery metric inflated 4x, dead-end check unreachable, gate metrics blindable #1430) — deflection and explanation counted as refusal recovery, and a pivot clause that matched inside its own refusal. On the same corpus the recovery rate went 0.3566 → 0.2168 → 0.0839. Any proposal that cites a digest number as its baseline should re-read it rather than trust a figure from before 2026-07-28.

2. Seven issues are queued behind [BLOCKER] No production capture of real agent tool calls — three issues and six PRs have failed on this one missing input #1363 (no production capture of real agent tool calls): [IMPROVEMENT] Adversarial evaluator floor test — null-agent baseline for evolution metrics (BenchJack) #1267, [IMPROVEMENT] Tool-use competency diagnostic rubric — discovery, parameterization, error recovery, efficiency (MCP-Atlas) #1268, [IMPROVEMENT] Selective memory addition gate with history-based utility deletion (Experience-Following Property) #1270, [FEATURE] ERL heuristic extraction from cross-task traces (Child A of #1303) #1359/[FEATURE] ERL judge-reranked heuristic retrieval (Child B of #1303) #1360/[FEATURE] ERL system-prompt injection of top-k heuristics (Child C of #1303) #1361, [FEATURE] Experience Memory Graph (EMG): graph-matched one-shot error correction from paired success/failure trajectories #1436, [IMPROVEMENT] Failure-trajectory 3-timestamp framework for diagnosing pipeline stalls (t_err, t_lock, t_obs) #1442. Six PRs have already been closed as dead code or incoherent for implementing against a synthetic substitute. If this proposal's implementation ends up wanting per-run traces, it joins that queue rather than proceeding — worth checking early, since it changes the effort estimate entirely.

No action needed on this issue right now; it stays open and untriaged-for-priority.

[Lexus2016]

Rejected by evolution-analysis: out-of-scope — Knowledge-Centric Self-Improvement — speculative research framework, no concrete implementation path on this codebase. Fails focus-test.