Skip to content

Move the tarot review gate host-side; serve tarot's own authoring skill - #403

Merged
dkrattiger merged 1 commit into
dimitri/pending-fixesfrom
panopticon/host-side-tarot-gate
Sep 1, 2026
Merged

Move the tarot review gate host-side; serve tarot's own authoring skill#403
dkrattiger merged 1 commit into
dimitri/pending-fixesfrom
panopticon/host-side-tarot-gate

Conversation

@dkrattiger

Copy link
Copy Markdown
Contributor

Why

capabilities.tarot_review has never been flippable on any repo. The gate #348 shipped runs as an
in-container PreToolUse hook that shells out to tarot — a binary no task image has ever
contained. Turning the capability on today would refuse every advance on that repo, forever.

The obvious fix is to bake tarot into the base image. This doesn't do that: it would couple
panopticon's image to tarot's release cadence and create version skew with the operator's own
install. Instead the enforcement moves to the host, where tarot already lives — the same place
the dashboard's v review hook has been running it since #350.

That works because of one fact: a task's clone (task.clone) is the directory bind-mounted into
its container at /workspace. A check run host-side reads exactly what the agent just wrote — no
copy, no sync, no staleness window — and tarot diffs with the working tree included, so uncommitted
work counts.

What changed

  • core/tarot.py (new) — the LLM-free tarot adapter: argv for every invocation, exit-code
    handling (0 valid / 1 violations / 2 structural), binary resolution
    (override → $PANOPTICON_TAROT_BIN → PATH), and the tarot.base ladder. Injectable
    command-runner, mirroring core/git.py.
  • taskservice/tarot_gate.py (new) — the policy. On an advance whose workflow declares the
    review responsibility for an opted-in repo, it runs the checks against the clone and either
    resolves the responsibility MET or refuses the transition with the checks' own output, recorded
    as the responsibility's comment.
  • Three MCP passthroughs — tarot's authoring skill has seven steps, four of which invoke the
    CLI. tarot_strand_seed (strands suggest --json, read-only), tarot_check (both checks,
    without attempting a transition), and tarot_tour_scaffold (writes the stub tour) cover
    exactly those four. The other three steps are file editing, which the agent already does.
  • skills() serves tarot's own packaged SKILL.md for opted-in repos, prefixed only by a
    name-mapping table (tarot strands suggesttarot_strand_seed, …). Panopticon keeps no second
    copy of tarot's file formats — they change with tarot's validators, and the judgment the skill
    teaches (what to retitle, what a description is for, what makes a tour note worth reading) is
    precisely what a schema summary would lose.
  • container/tarot_gate.py → an allow-everything shim, unwired from hooks.py. The module
    can't just be deleted: .claude/settings.json lives in a task's persisted config volume and
    update_json_config merges rather than prunes, so a respawned task would keep a hook entry
    pointing at it and fail every apply_operation.
  • terminal/console.py now resolves the binary and the base ref through the same helpers, so
    v and the gate can't disagree about which tarot to run or which base to diff against.

No Dockerfile, Makefile, or image changes. No repo capability is flipped.

Two bugs caught before this landed

Spike also has an ITERATING state. Gating on the state label would have refused every spike
advance on an opted-in repo — with no responsibility to satisfy and no way out but a free move. The
gate keys off whether the workflow declares the responsibility instead.

A clone that pins its own tarot.base would have been diffed against HEAD, which reads as a
zero-line diff and would have waved every such task through as "trivial" — exactly the silent pass
this gate exists to remove. One config read now feeds both the diff ref and tarot's arguments. (The
in-container gate had a sibling of this bug: it summed a failed numstat to 0 and auto-resolved
MET. A failed numstat is now "unknown, not trivial" and the checks run.)

Operator runbook

1. Host prerequisite. tarot on the task service host's PATH — the existing
uv tool install tarot-review covers it; $PANOPTICON_TAROT_BIN overrides. No image rebuild and
no container respawn
, which is the point of moving host-side. Resolution is lazy and re-probes
while missing, so installing tarot in response to a refusal takes effect on the next advance
without restarting the service.

2. Flip a repo — GET, merge, PATCH. TaskService.update_repo does replace(existing, **changes),
so capabilities is replaced wholesale; patching only tarot_review silently wipes
docker_in_docker.

curl -s  $PANOPTICON_SERVICE_URL/repos/<id> | jq '.capabilities'
curl -X PATCH $PANOPTICON_SERVICE_URL/repos/<id> \
  --header 'Content-Type: application/json' \
  --data '{"capabilities": {"docker_in_docker": false, "tarot_review": true}}'

3. Threshold. Add "tarot_review_threshold": <int> to the same map to move the trivial-diff
cutoff off its default of 20 changed lines. Below it, the checks are skipped and the responsibility
auto-resolves.

4. Rollback is the same PATCH with "tarot_review": false. Effective on the next advance; no
restart.

5. Which repo first. Start with a single-language Python repo. tarot's changed-function set
comes from its language adapter (git config tarot.language, else python), so unsupervised-main
being a mixed monorepo makes it the last one to enable, not the first.

Limits worth knowing

  • Remote runners. The gate runs where the task service runs, but a clone is created on the
    runner's host (ADR 0011). With runner_host set the checks can't reach it and the gate refuses
    rather than passing. The dashboard's v already degrades this way — but v degrading is a
    convenience loss, whereas the gate degrading blocks the advance, so don't enable this for
    repos whose tasks run on remote runners until the check runs runner-side.
  • The gate can't tell a good tour from a worthless one. check_tour verifies that steps
    resolve, not that they say anything. Serving tarot's judgment text and exposing tour scaffold
    (whose blast-radius steps and real trail/cursor no hand enumeration produces) is what pushes
    against that, but quality stays a human-review concern.
  • This puts a subprocess in the control plane. The determinism invariant is about LLM calls so
    it isn't broken, and core/git.py plus console.py's existing tarot invocation are the
    precedent — but the task service is described as "pure, no I/O" and this widens that. Deliberate,
    flagged for the reviewer.

Follow-ups (tarot-side, none blocking)

  • tarot skill show to print the packaged SKILL.md — panopticon currently has to
    skill install --target <tempdir> and read it back.
  • A no-CLI authoring path in tarot's own SKILL.md, which would shrink the preamble here to nothing.
  • --json on strands check / tour check / tour scaffold: today panopticon scrapes
    human-readable output, which quietly makes tarot's prose formatting an API.

🤖 Generated with Claude Code

The `tarot_review` capability has never been flippable: the gate ran as an
in-container `PreToolUse` hook that shelled out to a `tarot` binary no task
image has ever contained. Rather than bake tarot into the image — coupling
panopticon's image to tarot's release cadence — run the checks on the host,
against the task's per-task clone. That clone is the very directory
bind-mounted into the container at /workspace, so a host-side check sees
exactly what the agent just wrote.

- `core/tarot.py`: the LLM-free tarot adapter (argv, exit codes, binary
  resolution, the shared `tarot.base` ladder) behind an injectable
  command-runner, mirroring `core/git.py`.
- `taskservice/tarot_gate.py`: the policy. Refuses an `advance` whose review
  artifacts don't check out, with the checks' own output, and records it as
  the responsibility's comment. The refusal is the enforcement — a
  FAILED-with-comment promise counts as *resolved*, so the gate runs on every
  attempt, not only while it's pending.
- Three read-mostly MCP passthroughs (`tarot_strand_seed`, `tarot_check`,
  `tarot_tour_scaffold`) cover the four CLI-bound steps of tarot's authoring
  skill, so the agent can author and iterate without a local tarot.
- `skills()` serves tarot's **own** packaged SKILL.md for an opted-in repo,
  prefixed only by a name-mapping table. Panopticon keeps no second copy of
  tarot's file formats: they change with tarot's validators, and the judgment
  the skill teaches is what a schema summary would lose.
- `container/tarot_gate.py` becomes an allow-everything shim (a respawned
  task's persisted settings.json may still name it) and `hooks.py` stops
  wiring it.
- `terminal/console.py` resolves the binary and the base ref through the same
  helpers, so the dashboard's `v` and the gate can't disagree about which
  tarot to run or which base to diff against.

No repo capability is flipped here, and no image or Makefile changes: the one
prerequisite is `tarot` on the task service host's PATH.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@dkrattiger
dkrattiger merged commit 4d110c0 into dimitri/pending-fixes Sep 1, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants