Move the tarot review gate host-side; serve tarot's own authoring skill - #403
Merged
Merged
Conversation
The `tarot_review` capability has never been flippable: the gate ran as an in-container `PreToolUse` hook that shelled out to a `tarot` binary no task image has ever contained. Rather than bake tarot into the image — coupling panopticon's image to tarot's release cadence — run the checks on the host, against the task's per-task clone. That clone is the very directory bind-mounted into the container at /workspace, so a host-side check sees exactly what the agent just wrote. - `core/tarot.py`: the LLM-free tarot adapter (argv, exit codes, binary resolution, the shared `tarot.base` ladder) behind an injectable command-runner, mirroring `core/git.py`. - `taskservice/tarot_gate.py`: the policy. Refuses an `advance` whose review artifacts don't check out, with the checks' own output, and records it as the responsibility's comment. The refusal is the enforcement — a FAILED-with-comment promise counts as *resolved*, so the gate runs on every attempt, not only while it's pending. - Three read-mostly MCP passthroughs (`tarot_strand_seed`, `tarot_check`, `tarot_tour_scaffold`) cover the four CLI-bound steps of tarot's authoring skill, so the agent can author and iterate without a local tarot. - `skills()` serves tarot's **own** packaged SKILL.md for an opted-in repo, prefixed only by a name-mapping table. Panopticon keeps no second copy of tarot's file formats: they change with tarot's validators, and the judgment the skill teaches is what a schema summary would lose. - `container/tarot_gate.py` becomes an allow-everything shim (a respawned task's persisted settings.json may still name it) and `hooks.py` stops wiring it. - `terminal/console.py` resolves the binary and the base ref through the same helpers, so the dashboard's `v` and the gate can't disagree about which tarot to run or which base to diff against. No repo capability is flipped here, and no image or Makefile changes: the one prerequisite is `tarot` on the task service host's PATH. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
capabilities.tarot_reviewhas never been flippable on any repo. The gate #348 shipped runs as anin-container
PreToolUsehook that shells out totarot— a binary no task image has evercontained. Turning the capability on today would refuse every
advanceon that repo, forever.The obvious fix is to bake tarot into the base image. This doesn't do that: it would couple
panopticon's image to tarot's release cadence and create version skew with the operator's own
install. Instead the enforcement moves to the host, where tarot already lives — the same place
the dashboard's
vreview hook has been running it since #350.That works because of one fact: a task's clone (
task.clone) is the directory bind-mounted intoits container at
/workspace. A check run host-side reads exactly what the agent just wrote — nocopy, no sync, no staleness window — and tarot diffs with the working tree included, so uncommitted
work counts.
What changed
core/tarot.py(new) — the LLM-free tarot adapter: argv for every invocation, exit-codehandling (0 valid / 1 violations / 2 structural), binary resolution
(
override → $PANOPTICON_TAROT_BIN → PATH), and thetarot.baseladder. Injectablecommand-runner, mirroring
core/git.py.taskservice/tarot_gate.py(new) — the policy. On anadvancewhose workflow declares thereview responsibility for an opted-in repo, it runs the checks against the clone and either
resolves the responsibility
METor refuses the transition with the checks' own output, recordedas the responsibility's comment.
CLI.
tarot_strand_seed(strands suggest --json, read-only),tarot_check(both checks,without attempting a transition), and
tarot_tour_scaffold(writes the stub tour) coverexactly those four. The other three steps are file editing, which the agent already does.
skills()serves tarot's own packagedSKILL.mdfor opted-in repos, prefixed only by aname-mapping table (
tarot strands suggest→tarot_strand_seed, …). Panopticon keeps no secondcopy of tarot's file formats — they change with tarot's validators, and the judgment the skill
teaches (what to retitle, what a description is for, what makes a tour note worth reading) is
precisely what a schema summary would lose.
container/tarot_gate.py→ an allow-everything shim, unwired fromhooks.py. The modulecan't just be deleted:
.claude/settings.jsonlives in a task's persisted config volume andupdate_json_configmerges rather than prunes, so a respawned task would keep a hook entrypointing at it and fail every
apply_operation.terminal/console.pynow resolves the binary and the base ref through the same helpers, sovand the gate can't disagree about which tarot to run or which base to diff against.No Dockerfile, Makefile, or image changes. No repo capability is flipped.
Two bugs caught before this landed
Spikealso has anITERATINGstate. Gating on the state label would have refused every spikeadvance on an opted-in repo — with no responsibility to satisfy and no way out but a free move. The
gate keys off whether the workflow declares the responsibility instead.
A clone that pins its own
tarot.basewould have been diffed againstHEAD, which reads as azero-line diff and would have waved every such task through as "trivial" — exactly the silent pass
this gate exists to remove. One config read now feeds both the diff ref and tarot's arguments. (The
in-container gate had a sibling of this bug: it summed a failed numstat to 0 and auto-resolved
MET. A failed numstat is now "unknown, not trivial" and the checks run.)Operator runbook
1. Host prerequisite.
taroton the task service host'sPATH— the existinguv tool install tarot-reviewcovers it;$PANOPTICON_TAROT_BINoverrides. No image rebuild andno container respawn, which is the point of moving host-side. Resolution is lazy and re-probes
while missing, so installing tarot in response to a refusal takes effect on the next advance
without restarting the service.
2. Flip a repo — GET, merge, PATCH.
TaskService.update_repodoesreplace(existing, **changes),so
capabilitiesis replaced wholesale; patching onlytarot_reviewsilently wipesdocker_in_docker.3. Threshold. Add
"tarot_review_threshold": <int>to the same map to move the trivial-diffcutoff off its default of 20 changed lines. Below it, the checks are skipped and the responsibility
auto-resolves.
4. Rollback is the same PATCH with
"tarot_review": false. Effective on the next advance; norestart.
5. Which repo first. Start with a single-language Python repo. tarot's changed-function set
comes from its language adapter (
git config tarot.language, else python), sounsupervised-mainbeing a mixed monorepo makes it the last one to enable, not the first.
Limits worth knowing
runner's host (ADR 0011). With
runner_hostset the checks can't reach it and the gate refusesrather than passing. The dashboard's
valready degrades this way — butvdegrading is aconvenience loss, whereas the gate degrading blocks the advance, so don't enable this for
repos whose tasks run on remote runners until the check runs runner-side.
check_tourverifies that stepsresolve, not that they say anything. Serving tarot's judgment text and exposing
tour scaffold(whose blast-radius steps and real trail/cursor no hand enumeration produces) is what pushes
against that, but quality stays a human-review concern.
it isn't broken, and
core/git.pyplusconsole.py's existing tarot invocation are theprecedent — but the task service is described as "pure, no I/O" and this widens that. Deliberate,
flagged for the reviewer.
Follow-ups (tarot-side, none blocking)
tarot skill showto print the packaged SKILL.md — panopticon currently has toskill install --target <tempdir>and read it back.--jsononstrands check/tour check/tour scaffold: today panopticon scrapeshuman-readable output, which quietly makes tarot's prose formatting an API.
🤖 Generated with Claude Code