feat(guide): RAC Guide MCP server, onboarding, and grounding demo [roadmap:v0.10.x]

[tcballard]

Summary

Implements the v0.10.x Guide series: rac/roadmaps/v0.10.x-guide/v0.10.0-guide-foundation.md, v0.10.1-guide-onboarding.md, and v0.10.2-guide-grounding-demo.md.

Adds:

• rac mcp [--root PATH] — a stdio MCP server that serves RAC repository knowledge to coding agents, shipped inside the existing package as a standard dependency (mcp>=1.0)
• Exactly four read-only tools — get_artifact, search_artifacts, get_related, get_summary — with descriptions shipped verbatim from the guide-tool-surface design
• A shared per-response character budget (default 10,000, configurable at server startup) with whole-item truncation and pinned truncated / omitted / hint markers
• Structured not-found / duplicate / unreadable errors as data, never protocol exceptions
• Onboarding: README Guide section, docs/mcp.md (per-client configuration, team CLAUDE.md routing setup, troubleshooting), and configuration blocks for Claude Code, Claude Desktop, and Cursor
• Examples corpus examples/guide/ (one connected requirement, decision, design, roadmap under its own GUIDE repository key) wired into the dogfood CI gate
• Grounding demo: the Meridian code task whose naive implementation violates the corpus's soft-delete decision, and examples/guide/demo.md — a stranger-runnable contrast script with measurement protocol, run-log template, and recording shot list
• 60 new tests: a new mcp CI battery (server, tool contracts, truncation boundaries, error paths, isolation) plus dogfood coverage of the examples corpus and demo searchability
• A manually triggered TestPyPI rehearsal workflow (test-publish.yml) mirroring the release pipeline, so builds can be exercised end-to-end without publishing to PyPI
• README "Lore" header banner, theme-matched dark/light variants served via an HTML picture element from rac/assets/images/ (ADR-019 asset convention)

Roadmap / ADR Trace

Roadmap:

• rac/roadmaps/v0.10.x-guide/v0.10.0-guide-foundation.md
• rac/roadmaps/v0.10.x-guide/v0.10.1-guide-onboarding.md
• rac/roadmaps/v0.10.x-guide/v0.10.2-guide-grounding-demo.md

Relevant ADRs:

• rac/decisions/adr-029-guide-delivery-surface.md — stdio only, standard dependency, rac mcp subcommand
• rac/decisions/adr-030-guide-tools-only-surface.md — exactly four tools, no resources or prompts
• rac/decisions/adr-031-guide-in-process-core-consumption.md — in-process service consumption, read-only by construction
• rac/decisions/adr-032-guide-stateless-reads.md — every call re-reads the repository, no caching
• rac/decisions/adr-033-guide-response-budget.md — character budget and truncation contract (clarified in this PR: the unit is characters of serialized JSON; UTF-8 wire bytes may exceed it for non-ASCII content)
• rac/decisions/adr-034-guide-agent-reasoning-boundary.md — no judgment-shaped tools
• rac/decisions/adr-007-json-contract-stability.md, adr-015-explorer-as-consumer.md, adr-019-asset-management.md, adr-026-opaque-artifact-identities.md, adr-027-ci-test-topology.md

Designs: rac/designs/guide-tool-surface.md (pinned tool surface; gains the unreadable error shape and a structuredContent scope note in this PR), rac/designs/guide-grounding-demo.md (demo specification).

Scope

Included

• src/rac/mcp/ package: server.py (FastMCP factory build_server(root, budget) and run_server), budget.py, errors.py; only this package imports the MCP SDK
• CLI subcommand with pinned exit codes; diagnostics to stderr only, stdout reserved for the protocol
• Zero-flag startup hardening: an empty or artifact-free root starts the server, emits a one-line helpful stderr notice, and get_summary reports the empty state
• Contract tests pinning all four tool output shapes, truncation boundaries, error shapes, CLI-JSON equivalence, and repository freshness; an AST isolation battery enforcing the dependency direction; a walk-count regression test pinning one corpus walk per get_related call
• Demo assets: examples/guide/task/ (minimal Meridian user-service slice with the delete() stub) and examples/guide/demo.md
• Release tooling: test-publish.yml, a workflow_dispatch rehearsal of the publish pipeline targeting TestPyPI via trusted publishing (strips the setuptools-scm local version segment, which PyPI-family indexes reject)
• Launch branding: Lore header banner (dark + light) in rac/assets/images/, referenced from the README top

Excluded

• MCP resources, prompts, and HTTP/SSE transport (ADR-030, ADR-029)
• Conflict detection or any judgment-shaped tool (ADR-034)
• Caching, file watching, or session state (ADR-032); the single-walk snapshot inside one get_related call is per-call only
• Body-text search; search_artifacts pins find_artifacts metadata-field matching only (design open question, deferred)
• Per-call budget override or a budget CLI flag; the budget is a build_server startup parameter only
• Alternative runners (uvx etc.); the documented path is pip install
• The broad tool-per-command surface from the old v1.2-mcp-server stub, superseded by ADR-030
• Structured output via structuredContent: tools return serialized JSON text (the pinned contract); FastMCP's structuredContent envelope is explicitly out of contract in v1 (noted in the design)
• Any project/package rename: the "Lore" banner is launch branding only; package, CLI, and docs naming are unchanged (a rename, if wanted, needs its own decision)
• Human-only release-tail steps, deliberately not faked in this PR (see Notes For Reviewer)

Product / Architecture Decisions

• get_summary returns PortfolioSummary.to_dict() unchanged, including the health block: the design's prose pins "the same contract rac portfolio --json emits"; its illustrative JSON had merely elided the field. An equivalence test asserts byte-for-byte parity with the CLI.
• get_related serves resolution, outgoing, and incoming from ONE corpus walk per call: the response is an atomic snapshot of repository state (no mid-call drift), and incoming edges filter Core's own resolved_path — the server invents no matching semantics (ADR-031). Caching across calls remains forbidden (ADR-032); the walk count is pinned by a regression test.
• An artifact that resolves but cannot be read (deleted between walk and read, permissions, non-UTF-8) returns a structured {"error": "unreadable", "id": ..., "path": ...} body — an additive contract change documented in the design — keeping the "errors as data, never protocol exceptions" rule airtight.
• Tool errors otherwise are exactly ResolutionResult.to_dict() for the failure outcomes, so Guide errors cannot drift from rac resolve --json errors.
• The truncated marker is absent (not false) on complete responses; marker names and placement are pinned by contract tests. Each tool's truncation hint is standalone prose; get_summary has its own (raise the server budget), since "request a specific artifact" made no sense there.
• The budget counts characters of the serialized JSON, not UTF-8 bytes — clarified in ADR-033: character counting keeps the unit deterministic across models; wire bytes may exceed the budget for non-ASCII corpora.
• Truncation drops whole items from the tail only (matches, incoming entries, content characters), keeping output deterministic and always valid JSON.
• The MCP SDK import in the CLI is lazy, so every other rac command avoids its import cost.
• Empty-corpus startup is a stderr notice, not an error: absence of a corpus is a valid served state (get_summary reports zero artifacts).
• The demo corpus decision (ADR-001: Soft-Delete User Records) was authored to the design's true-to-life bar — audit/GDPR motivation, named technologies, recorded failure history in the alternatives — so the demo cannot read as a strawman.

User-Facing Contract

CLI

rac mcp                  # serve the current directory over stdio
rac mcp --root PATH      # serve PATH

Human Output

• Nothing on stdout (it belongs to the MCP protocol); a one-line stderr notice when the root contains no recognized artifacts.

JSON Output

All tool payloads carry "schema_version": "1". Shapes reuse existing to_dict contracts (SearchResult, PortfolioSummary, ResolutionResult); get_artifact adds content (file text exactly as stored); get_related adds outgoing / incoming. Error bodies: not-found, duplicate (resolver shapes), and unreadable (resolved but unreadable file, with id and path). Truncated responses add:

{
  "truncated": true,
  "omitted": 12,
  "hint": "Narrow the query or request a specific artifact ID."
}

Exit Codes

• 0: clean server shutdown (client disconnect)
• 2: usage error (--root is not a directory)
• Tool-level failures are structured tool results, never exit codes.

Verification

Ran

python -m pytest -q                                   # 921 passed
python -m ruff check src/ tests/                      # clean
python -m ruff format --check src/ tests/             # clean
python -m mypy src/                                   # no issues, 64 files
rac validate rac/                                     # 99 valid, 0 invalid
rac relationships rac/ --validate                     # 239 checked, 0 issues
rac validate examples/guide/rac/                      # 4 valid, 0 invalid
rac relationships examples/guide/rac/ --validate      # 10 checked, 0 issues
rac review rac/                                       # no priority 1-2 findings

Plus a live stdio smoke test: an mcp SDK client spawning rac mcp exercised all four tools against the dogfood corpus (219 artifacts) and the examples corpus (4 artifacts), including the not-found error path; get_related output was verified identical before and after the one-walk refactor. The setuptools-scm no-local-version override used by the rehearsal workflow was verified locally (0.1.dev64+gb6cf33a8e builds as 0.1.dev64).

Covered

• All four tool response shapes pinned against fixture corpora, including CLI-JSON equivalence for the same fixture state
• Truncation at each whole-item boundary (matches, incoming entries, content tail), marker absence on complete responses, envelope-exceeds-budget fallbacks, unicode content through the character budget, type filter combined with truncation
• Not-found, duplicate, and unreadable error shapes; ambiguous references contribute no incoming edge; zero-outgoing artifacts return {}
• Freshness: a fixture edit between calls is observed (ADR-032, no caching); exactly one corpus walk per get_related call (regression-pinned)
• Isolation by AST: rac.core / rac.services import neither mcp nor rac.mcp; rac.mcp imports no write-capable service
• CLI: factory construction, registration, --root handling, both exit codes
• Empty-corpus startup: helpful stderr, functional zero-state get_summary, no spurious notice on populated corpora
• Examples corpus validity and demo searchability (delete user, delete, soft-delete each surface the demo decision as the top decision match)

Review Path

1. src/rac/mcp/server.py — the four tools, the consumer boundary, the one-walk snapshot
2. src/rac/mcp/budget.py, src/rac/mcp/errors.py — the pinned response mechanics
3. src/rac/cli.py, pyproject.toml, .github/workflows/tests.yml — delivery surface and battery registration
4. tests/test_mcp_tools.py, test_mcp_server.py, test_mcp_isolation.py — the contract pins
5. examples/guide/ — corpus, task, and demo.md
6. README.md, docs/mcp.md — onboarding, team routing setup, header banner
7. .github/workflows/test-publish.yml — release rehearsal tooling

Notes For Reviewer

• The three client configuration blocks carry explicit "TODO: verify against [client] [version] before release" markers: the roadmap requires verification against live clients (one by someone other than the author), which cannot be done in the implementation environment. Complete these before release.
• examples/guide/demo.md ends with the human-only release-tail checklist: the 10-run measurement (gate: at least 8 correct decision-ID citations), the 90-second recording, registry submissions, and the announcement. None of these is pre-filled or faked.
• The TestPyPI rehearsal workflow needs one-time setup before its first run: a trusted publisher on test.pypi.org pointing at test-publish.yml and the testpypi environment, plus that environment created in the repository settings.
• Demo-quality watch item: the task code necessarily shows the deleted_at IS NULL pattern (the compliant fix is impossible without knowing the column exists), so a strong ungrounded run could occasionally infer soft-delete unaided. If measurement shows a weak contrast, adjust the task wording — not the tool surface.
• The "Lore" banner introduces a product name the corpus does not yet record; if the name is to outlive the launch, capture it in a decision artifact before it spreads further.

Implementation Process

Implemented with AI assistance under the roadmap contract. Final scope, review, and acceptance decisions were made by the maintainer.

[tcballard]

Test-installing the Guide build from TestPyPI

Once the Test Publish (TestPyPI) workflow has been run against this branch (Actions tab → Test Publish (TestPyPI) → Run workflow → select claude/rac-v0.10-implementation-12w6e1):

1. Find the rehearsal version

The exact version string is printed in the run's Build rehearsal distributions step (e.g. requirements_as_code-0.9.1.dev12.tar.gz), or listed at https://test.pypi.org/project/requirements-as-code/#history. It is a .devN version derived from the latest tag — unique per commit, never a real release number.

2. Install into a clean environment

python -m venv ~/rac-rehearsal && source ~/rac-rehearsal/bin/activate
pip install --index-url https://test.pypi.org/simple/ \
    --extra-index-url https://pypi.org/simple/ \
    "requirements-as-code==<rehearsal version>"

The --extra-index-url is required: dependencies (mcp, markdown-it-py, pyyaml) live on real PyPI, not TestPyPI. Omitting the version pin also works but may resolve to an older rehearsal upload — pinning is safer.

3. Smoke-check the server binary

rac --version
rac mcp --root /nonexistent ; echo "exit: $?"   # expect usage error, exit 2

4. Wire it into Claude Code and test against the examples corpus

Use the venv's absolute rac path so the global PATH doesn't matter:

claude mcp add rac-guide -- ~/rac-rehearsal/bin/rac mcp --root /path/to/clone/examples/guide
claude mcp list    # rac-guide should be listed and connected

In a Claude Code session, /mcp should show rac-guide with four tools (get_summary, search_artifacts, get_artifact, get_related). Ask "what decisions are recorded in this repository?" and watch get_summary / search_artifacts fire — a grounded answer should cite ADR-001: Soft-Delete User Records for deletion topics.

5. While you're there (release-gate two-for-one)

A successful step 4 is the v0.10.1 "verified against Claude Code" checkbox — note the client version (claude --version) and replace the TODO: verify marker in README/docs/mcp.md for the Claude Code block. From the same setup, examples/guide/demo.md runs the full grounded demo and the 10-run measurement.

Cleanup

claude mcp remove rac-guide
deactivate && rm -rf ~/rac-rehearsal

Remember claude mcp remove rac-guide in particular — a configured server pointing at the demo corpus will skew the ungrounded demo run later.