Agent Relay v0.2 — Design Spec

Status: approved 2026-04-28 Author: project owner Branch: v0.2-lessons

Why

v0.1 ships a working multi-agent state machine with file-based artifacts. The pitch — "Docker Compose for AI agents" — is forgettable, and the unique value (workflow state in your repo, tool-agnostic, human-in-the-loop) doesn't differentiate from a crowded 2026 shelf (LangGraph, CrewAI, AutoGen, claude-flow / Ruflo, claude-code subagents, agentic-stack, AGENTS.md).

The honest gap that no other tool fills cleanly: a coding agent that gets measurably better at the same task class over successive runs because past plans, reviewer rejections, and auditor catches are compiled into typed lessons that the next planner reads. Hermes Agent does skill-from-experience but inside a closed runtime. AGENTS.md is hand-written. Mem0 / Letta / Zep operate at the conversation level, not at structured-workflow-artifact level. None expose role-typed lessons as in-repo, human-editable markdown with run-id provenance.

v0.2 closes this gap and reframes the project around compounding improvement.

Non-goals (v0.2)

Embedding-based lesson retrieval (heuristic relevance only)
Cross-repo lesson portability
Conflict resolution between contradictory lessons (flag, don't resolve)
A separate relay-skills or relay-recipes repo (templates ship inside agent-relay)
Strategic / positioning content in this OSS repo — kept in private notes

Core additions

1. Persisted run history

Today. Artifacts live at .relay/workflows/<name>/artifacts/*.md and are overwritten on every iteration. State.yml advances in place. After a workflow completes, you have one snapshot — the last one.

v0.2. When a workflow reaches a terminal stage, the artifact set + final state.yml + workflow.yml + roles snapshot are copied to .relay/history/<run-id>/. Run-id format: <YYYYMMDD-HHMM>-<short-slug> where short-slug is sourced from state.metadata["run_slug"] if set, otherwise the workflow name.

Backward compat. History is opt-in via relay.yml: history.enabled: true (default true for new workflows; existing workflows keep prior behavior unless flag added).

2. Lessons compiler (`relay distill`)

A new command that walks .relay/history/ and produces:

.relay/LESSONS.md — human-readable, organized by role
.relay/lessons.json — typed, machine-readable

Two extraction modes:

Heuristic (default, deterministic, no LLM) — parses review/audit artifact files for known headers (## Required Changes, ## Concerns, ## Catches, ## Verdict) and extracts bullet points as lesson candidates. Severity inferred from header (Required Changes → warn, Catches in audit → error, Suggestions → info). Tag inference: filenames mentioned in the bullet text. Deterministic — same history dir produces the same lessons.json. CI-friendly.
LLM (--llm, optional) — if a backend is configured, calls the LLM with a structured prompt to distill higher-quality typed lessons from the same artifacts. Falls back to heuristic if no backend or call fails.

Lesson schema (Pydantic v2):

class Lesson(BaseModel):
    id: str                 # stable hash of (role, claim, evidence_run_id)
    role: str               # "planner" | "reviewer" | "implementer" | "auditor" | other
    claim: str              # one-sentence statement
    severity: Literal["info", "warn", "error"]
    evidence_run_id: str    # which run this came from
    evidence_excerpt: str   # the bullet text it was extracted from
    observed_at: datetime
    tags: list[str] = []    # filenames or keywords inferred from the bullet

Idempotency: re-running relay distill with the same history produces the same lesson IDs. Manual edits to LESSONS.md are preserved by appending # manual: ... lines, which relay distill reads back on next run.

3. Auto-load lessons into planner prompts

prompt.py extended with an optional ## Lessons from past runs section. Loaded from .relay/lessons.json when:

The role spec opts in via inject_lessons: true (defaults to false)
Lessons file exists

Filtering: lessons matching the current role get a "from prior X" section; lessons whose tags overlap with files in the current role's reads list are highlighted as "highly relevant." Capped at top-N (default 10) to avoid prompt bloat.

The injected section is markdown:

## Lessons from past runs

Bullets below were distilled from previous workflow runs. They are not
absolute rules — apply judgment.

**Highly relevant to this task** (touch files you're working with):
- [warn, planner] When modifying state.py, the reviewer rejected plans that didn't
  account for the iteration_counts dict mutation. (run 20260428-0930-fix-bug, ...)

**Other lessons from past planner work:**
- [info, planner] ...

4. Two new templates + worked examples

bug-rca-fix — 5-stage workflow for systematic bug fixes:

reproduce (rca_agent) → produces repro.md
hypothesize (rca_agent) → produces hypothesis.md
fix_plan (planner) → produces plan.md, branches: approve | reject
implement (implementer) → produces build_log.md
verify (auditor) → produces audit.md, branches: approve | reject (back to fix_plan)

rfc-then-implement — for design-heavy work:

rfc_draft (architect) → produces rfc.md
rfc_review (reviewer) → branches: approve | reject (back to rfc_draft)
implement (implementer) → produces build_log.md
audit (auditor) → branches: approve | done | reject (back to implement)

plan-review-implement-audit — kept, with inject_lessons: true added to planner role.

Each template ships with templates/<name>/example/ containing realistic artifact files from a synthetic-but-plausible run, plus a populated LESSONS.md to demonstrate the compiled-knowledge output.

5. Claude Code exporter (`relay export claude-code`)

Mirrors the existing Cursor exporter. Generates:

.claude/agents/<role>.md — one per role, with frontmatter (name, description, model, optional tools field) and the system prompt as body. This is Anthropic's standard subagent format.
.claude/commands/relay-next.md, relay-advance.md, relay-status.md — slash commands wrapping the CLI for users who want to drive the workflow from inside Claude Code.

6. README rewrite

New headline: "Multi-agent workflows that learn from their own past runs."

Sections (in order):

Headline + 2-sentence value
30-second example (init → next → paste → advance, plus the v0.2 "after 3 runs the planner inherits 12 lessons" line)
What's different (4 bullets — git-committable artifacts, role-typed lessons, tool-agnostic, human-editable knowledge)
Templates (3 templates with one-paragraph descriptions, linked to templates/<name>/example/)
CLI reference table
Honest "vs" table — LangGraph, CrewAI, claude-code subagents, agentic-stack, AGENTS.md
Status block (test count, version, "used to ship every PR in this repo" once we have one)

No emoji headers, no clipart. Lead with the value, not the feature list.

Architecture summary

.relay/
  relay.yml                        # adds: history.enabled, lessons.max_per_role
  LESSONS.md                       # NEW — human-readable compiled lessons
  lessons.json                     # NEW — typed lessons with provenance
  history/                         # NEW — frozen run snapshots
    20260428-0930-fix-bug-1/
      state.yml
      workflow.yml
      roles/
      artifacts/
        plan.md
        plan_review.md
        build_log.md
        audit.md
    20260428-1145-fix-bug-2/
      ...
  workflows/
    default/
      workflow.yml
      state.yml                    # state.metadata["run_slug"] used for history dirs
      roles/
        planner.yml                # gains optional inject_lessons: true
        ...
      artifacts/                   # working set, overwritten each iteration as before

New module: src/relay/lessons.py. New module: src/relay/exporters/claude_code.py. CLI gains relay distill and relay export claude-code.

Test plan

Unit (tests/unit/)
- test_lessons_heuristic.py — given a fixture history dir, distill produces deterministic lessons.json
- test_lessons_relevance.py — relevance filter prefers tag-matching lessons
- test_history_snapshot.py — terminal stage triggers a history dir copy
- test_prompt_lessons_injection.py — when inject_lessons=true, lessons appear in compose_prompt output; when false, they don't
- test_claude_code_exporter.py — produces .claude/agents/.md + .claude/commands/.md with valid frontmatter
Integration (tests/integration/)
- test_distill_cli.py — relay distill produces both LESSONS.md and lessons.json
- test_export_claude_code.py — full CLI round-trip
- test_init_new_templates.py — bug-rca-fix and rfc-then-implement init cleanly
e2e (tests/e2e/)
- test_lessons_compounding.py — synthetic 2-run scenario where run 1's review.md mentions a fix; run 2's planner prompt includes that lesson

Target: 119 (current) + ~25 new tests = ~144. All deterministic, no live LLM calls in CI.

Acceptance criteria

pytest -q passes with ≥140 tests on v0.2-lessons branch
relay init --template bug-rca-fix works; same for rfc-then-implement
relay distill (heuristic mode) on a synthetic 2-run history produces a non-empty LESSONS.md and valid lessons.json
A planner role with inject_lessons: true shows the lessons section in relay next output
relay export claude-code produces files that pass a JSON-schema check on the frontmatter
README rewrites land on this branch, version bumped to 0.2.0 in pyproject.toml
Branch pushed to origin/v0.2-lessons (no merge to main, no PR creation)

Risks and mitigations

Risk	Mitigation
Heuristic distill is too dumb to be valuable	Ship LLM mode as opt-in; the heuristic is the floor, not the ceiling. Worked examples show what good output looks like
Lessons inject too much noise	Top-N cap (default 10), filter by role + tags, opt-in per role
History grows unbounded	v0.2 doesn't prune. Document a `relay history prune --keep N` for v0.3. Acceptable for v0.2 because most users won't have >50 runs in the first month
Claude Code subagent format changes	Mirror the current docs format; if Anthropic changes the schema, the exporter is one file to update
Breaks v0.1 users	History opt-in via config; existing tests must keep passing unchanged

Out of scope (v0.3+)

Embedding-based relevance for lessons
Lesson conflict detection / merging
relay history prune
Cross-repo lesson packs
A web UI for lesson review
Programmatic API (current usage is CLI-first by design)

FilesExpand file tree

2026-04-28-v0.2-design.md

Latest commit

History