Skip to content

Synthetic e2e environment for testing claudectl end-to-end #387

Description

@mercurialsolo

Goal

A throwaway environment that exercises claudectl's real discovery → monitor → process → brain pipeline against controllable fixtures, for both automated e2e assertions and interactive "experience" testing — without real Claude Code agents or spend.

Key insight

Everything claudectl treats as reality derives from $HOME (dirs_home()), so the isolation lever is a temp $HOME (redirects both .claude and .claudectl). Surfaces to fake:

Surface Where Synthesize
Session registry $HOME/.claude/sessions/*.json write JSON
Transcript $HOME/.claude/projects/<enc-cwd>/<id>.jsonl append JSONL on a timeline
Liveness/CPU real ps — a PID only counts if its command contains "claude" (process.rs:54) spawn a placeholder process argv'd as claude
Brain HTTP to brain.endpoint mock LLM server returning canned {action,confidence}
coord/bus $HOME/.claudectl/* fresh temp DBs; drive via supervisor run

The non-obvious constraint: in-memory fakes (demo.rs) bypass the real pipeline; faithful e2e needs real PIDs.

Design — two tiers

  • Tier A (CI): SynthEnv (temp $HOME) + SynthSession (placeholder claude proc + session JSON + emit_* methods) + scenario library (healthy/stall/loop/budget/conflict/needs-input) → run headless claudectl --json/--watch and assert on output.
  • Tier B (experience): same env, launch the TUI with a scenario driver mutating fixtures over time; capture with recorder/session_recorder.

Components

  1. SynthEnv (temp HOME + teardown that kills procs)
  2. SynthSession (spawn + JSON + emit_message/emit_tool_use/emit_usage/set_needs_input/stall/finish)
  3. Scenario library (port demo.rs timelines to real fixtures)
  4. Mock brain server (~30-line stub)
  5. Runner (headless-assert + TUI/record)

Build order

Tier A first (SynthEnv + one scenario + a headless assertion) — it's the reusable foundation. Lives in tests/e2e/ (+ optionally a claudectl simulate subcommand for Tier B).

Related: #373 (demo tour, in-memory) is the lighter-weight cousin.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions