Skip to content

Commit 4d110c0

Browse files
authored
Move the tarot review gate host-side; serve tarot's own authoring skill (#403)
Runs `tarot strands check` / `tarot tour check` on the host against the task's clone — the same directory bind-mounted into the container at /workspace — instead of from an in-container hook that needed a `tarot` binary no task image has ever had. That is what makes `capabilities.tarot_review` flippable at all. - core/tarot.py: the LLM-free tarot adapter behind an injectable command-runner. - taskservice/tarot_gate.py: the policy — refuses an `advance` whose review artifacts don't check out, with the checks' own output, recorded as the responsibility's comment. The refusal is the enforcement (a FAILED-with-comment promise counts as resolved), so it runs on every attempt. - Three MCP passthroughs (tarot_strand_seed / tarot_check / tarot_tour_scaffold) cover the four CLI-bound steps of tarot's authoring skill. - skills() serves tarot's own packaged SKILL.md under a name-mapping table; panopticon keeps no second copy of tarot's file formats. - container/tarot_gate.py becomes an allow-everything shim; console.py shares the binary and base-ref resolution so `v` and the gate can't disagree. No image or Makefile changes, and no repo capability flipped.
1 parent 8e2051d commit 4d110c0

18 files changed

Lines changed: 1956 additions & 494 deletions

File tree

AGENTS.md

Lines changed: 27 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,10 @@ src/panopticon/
2020
core/ # domain models, state classes, the Workflow interface (the state
2121
# machine: resolution, queries, start_task/apply_transition),
2222
# store & artifact interfaces — pure, no I/O EXCEPT git.py (local
23-
# branch/worktree ops; LLM-free, behind an injectable command-runner)
23+
# branch/worktree ops) and tarot.py (the host-side `tarot` CLI adapter:
24+
# strands/tour check, strands suggest --json, tour scaffold, the packaged
25+
# authoring skill, the shared `tarot.base` ladder) — both LLM-free, behind
26+
# an injectable command-runner
2427
workflows/ # built-in Workflow subclasses (Spike seed; GithubPeerReviewed [formerly Parity]
2528
# = cloude-cade lifecycle; GithubSelfReviewed = same, sans the peer-review state,
2629
# the user self-reviews; both share the GithubForgeWorkflow base = gh tool/layer/skills;
@@ -30,7 +33,10 @@ src/panopticon/
3033
# its shell_script in a host tmux session (here: `claude setup-token`)) +
3134
# discovery.py = scan the package + an optional path for Workflow subclasses
3235
# (the registry build_app runs on; drop a module in → registered, ADR 0004)
33-
taskservice/ # control plane: TaskService, FastAPI REST API, the SQLAlchemy store
36+
taskservice/ # control plane: TaskService, FastAPI REST API, tarot_gate.py (the host-side
37+
# review-artifact gate + the served tarot-authoring skill; refuses an ITERATING
38+
# `advance` for an opted-in repo whose `.tarot/` artifacts don't check out),
39+
# the SQLAlchemy store
3440
# adapter (in-memory or on-disk SQLite), filesystem artifact store, MCP
3541
# server (mcp.py: operations=tools, artifacts=resources; FastMCP) mounted at /mcp
3642
sessionservice/ # the runner: Runner ABC + StubRunner (in-process) + LocalRunner
@@ -68,10 +74,9 @@ src/panopticon/
6874
# heartbeat liveness) + agent.py (`-m panopticon.container.agent` = the tmux
6975
# pane's launcher: render skills + operations, point claude at the /mcp server,
7076
# put the workflow overview in its system prompt → exec `claude`) — the ONLY LLM pkg;
71-
# tarot_gate.py = the `PreToolUse` hook on `apply_operation` that real-verifies
72-
# (not self-attests) the opt-in tarot review-artifact responsibility for the
73-
# GitHub-forge workflows (`Repo.capabilities.tarot_review`, docs/repos.md) — runs
74-
# `tarot strands check` / `tarot tour check` and denies `advance` on failure
77+
# tarot_gate.py = an allow-everything SHIM; the review-artifact gate moved
78+
# host-side to taskservice/tarot_gate.py (the module survives only because a
79+
# respawned task's persisted .claude/settings.json may still name it)
7580
docker/Dockerfile # base task-container image (ADR 0005 base layer): python + git + bash +
7681
# the panopticon package + the `claude` CLI the agent execs; runs as the
7782
# unprivileged `panopticon` user. docker/entrypoint.sh = remap that user to the
@@ -179,6 +184,22 @@ on every PR (the same commands the Makefile wraps).
179184
(formerly `parity`; cloude-cade's lifecycle): the full `PLANNING→…→COMPLETE` path, the fg/bg
180185
`advanced_by` policy, per-stage gating, going back to coding as an ungated free move
181186
(`set_state`), and drop. Extend it when you touch the github-peer-reviewed flow.
187+
- `tests/taskservice/test_tarot_gate.py` — the golden spec for the **host-side tarot review gate**:
188+
pass → advance + `MET`; fail → refusal with the checks' output, `FAILED` comment, and the task
189+
**left where it was**; a *second* attempt still refused (the refusal is the enforcement — a
190+
`FAILED`-with-comment promise counts as resolved); honest refusals for a missing binary, an
191+
absent clone, and a remote-runner clone; the trivial-diff threshold (incl. a failed numstat
192+
meaning *unknown*, not *trivial*) and a clone that pins its own `tarot.base`; and every case the
193+
gate must NOT run (capability off, drop, `set_state`, another state). Two tests drive a stub
194+
`tarot` through the real subprocess runner so the exit-code contract is pinned for real.
195+
- `tests/taskservice/test_tarot_tools.py` — the three authoring passthroughs over MCP
196+
(`tarot_strand_seed`/`tarot_check`/`tarot_tour_scaffold`): the read-only pair writes nothing,
197+
`tarot_check` returns violations without touching task state, and every refusal is a plain
198+
message. Also covers serving **tarot's own** packaged skill (verbatim + panopticon's name
199+
mapping) only for an opted-in repo with tarot present.
200+
- `tests/core/test_tarot.py` — the tarot adapter: the exact argv of every invocation, exit-code
201+
handling (0/1/2 and a missing binary), binary resolution order, the `tarot.base` ladder, and
202+
`authoring_skill()` degrading to `None` rather than raising when tarot's layout moves.
182203
- `tests/test_store.py` — store **contract tests run against in-memory and on-disk SQLite**,
183204
proving the interface is backend-agnostic (and that rows/domain models stay in sync).
184205
- `tests/test_discovery.py` — workflow discovery (Slice 8): the built-in package + an optional

docs/repos.md

Lines changed: 35 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -79,20 +79,41 @@ a repo opts in only when its tasks genuinely need to run Docker.
7979

8080
For the GitHub-forge workflows (`github-peer-reviewed`, `github-self-reviewed`), setting
8181
`capabilities.tarot_review` adds a real, verified ITERATING responsibility: before `advance` out
82-
of ITERATING is allowed, the task must author `.tarot/strands.json` and (for non-trivial changes)
83-
a tour, passing `tarot strands check` / `tarot tour check`. Unlike every other responsibility
84-
(agent self-attested), this one is checked by an in-container hook that runs the actual commands
85-
and denies the `advance` call — with the checks' output as the reason — on failure. A trivial
86-
diff (below a changed-line threshold, default 20; override with the integer capability
87-
`tarot_review_threshold`) skips the checks and resolves the responsibility automatically.
88-
89-
The `tarot` CLI itself isn't installed by panopticon — a repo that opts in must add it to its own
90-
`image_layer_file` (for example `RUN pip install tarot-review`), since installing it into the
91-
shared workflow layer would force it onto every forge repo whether opted in or not. If `tarot`
92-
isn't on `PATH` for an opted-in repo, the gate denies with a message pointing back here.
93-
94-
Off by default, like `docker_in_docker` — a repo opts in only once it has installed `tarot` via
95-
its image layer.
82+
of ITERATING is allowed, the task must author its `.tarot/` review artifacts (a strand seed and a
83+
tour) and pass `tarot strands check` / `tarot tour check`. Unlike every other responsibility
84+
(agent self-attested), this one is checked for real, and a failure **refuses the advance** with
85+
the checks' own output — which lands in the agent's context the way a failing test's would — and
86+
records it as the responsibility's comment. A trivial diff (below a changed-line threshold,
87+
default 20; override with the integer capability `tarot_review_threshold`) skips the checks and
88+
resolves the responsibility automatically.
89+
90+
**The checks run on the host, not in the container.** The task service shells out to the
91+
operator's own `tarot` against the task's per-task clone — the same directory bind-mounted into
92+
the container at `/workspace`, so it sees exactly what the agent just wrote. Nothing needs to be
93+
installed in any task image, and no image rebuild or container respawn is involved in enabling
94+
this.
95+
96+
So the one prerequisite is host-side: **`tarot` on the task service host's `PATH`**
97+
(`uv tool install tarot-review`), or `$PANOPTICON_TAROT_BIN` pointing at it. That is the same
98+
binary the dashboard's `v` review hook uses. If it isn't there, an opted-in repo's advances are
99+
refused with a message saying so — the gate never silently passes.
100+
101+
The agent authors the artifacts without a local `tarot` via three MCP tools, which run host-side
102+
against the clone: `tarot_strand_seed` (tarot's detector-built seed, as JSON — writes nothing),
103+
`tarot_check` (the same checks the gate runs, without attempting a transition — the iteration
104+
loop), and `tarot_tour_scaffold` (writes `.tarot/tours/<id>.json` from the *edited* seed). The
105+
instructions come from **tarot's own packaged authoring skill**, served automatically for an
106+
opted-in repo — panopticon keeps no second copy of tarot's file formats, since they change with
107+
tarot's validators and the judgment the skill teaches is what a schema summary would lose.
108+
109+
Two limits worth knowing. The gate runs where the **task service** runs; if a task's clone lives
110+
on a remote runner's host (`runner_host` set), the checks can't reach it and the gate refuses
111+
rather than passing — so don't enable this for repos whose tasks run on remote runners yet. And
112+
tarot picks its language adapter from `git config tarot.language` (else python), so a polyglot
113+
repo will want that set in its clones. A clone that sets `git config tarot.base` has its own diff
114+
base respected, exactly as the dashboard's `v` does.
115+
116+
Off by default, like `docker_in_docker`.
96117

97118
## Host hook (`hook_file`)
98119

src/panopticon/container/hooks.py

Lines changed: 4 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,10 @@
99
the agent is asking the user something (so the dashboard shows input is required) and back to the
1010
*agent* once it's answered. ``AskUserQuestion`` is a mid-turn tool call — it never fires ``Stop``
1111
— so without this the turn would wrongly read *agent* the whole time the question is pending;
12-
- **PreToolUse** matched to the `apply_operation` MCP tool also runs the tarot review-artifact
13-
gate (:mod:`panopticon.container.tarot_gate`) — a real, opt-in check (not just turn bookkeeping)
14-
that can deny an `advance` call; see that module for what it does and why.
12+
The tarot review-artifact gate used to hang off **PreToolUse** on `apply_operation`; it now runs
13+
host-side in the task service (:mod:`panopticon.taskservice.tarot_gate`), so nothing is wired here
14+
for it. :mod:`panopticon.container.tarot_gate` remains as an allow-everything shim for containers
15+
whose persisted settings still name it.
1516
1617
claude-specific (`.claude/settings.json`); M3 revisits for other CLIs. Pure — the turn-flip
1718
callback is :mod:`panopticon.container.hook`. `:blocked:` is preserved by construction: the
@@ -27,8 +28,6 @@
2728

2829
#: The command claude runs for each hook event (sets the turn via the task service).
2930
HOOK_COMMAND = "python -m panopticon.container.hook"
30-
#: The command claude runs on every `apply_operation` call (the tarot review-artifact gate).
31-
TAROT_GATE_COMMAND = "python -m panopticon.container.tarot_gate"
3231

3332

3433
def settings() -> dict[str, Any]:
@@ -60,10 +59,6 @@ def run(actor: str, event: str | None = None, *, matcher: str | None = None) ->
6059
# The agent stops to ask the user → flip to user; once answered → back to agent.
6160
"PreToolUse": [
6261
run("user", matcher="AskUserQuestion"),
63-
{
64-
"matcher": "mcp__panopticon__apply_operation",
65-
"hooks": [{"type": "command", "command": TAROT_GATE_COMMAND}],
66-
},
6762
],
6863
"PostToolUse": [run("agent", matcher="AskUserQuestion")],
6964
},
Lines changed: 21 additions & 176 deletions
Original file line numberDiff line numberDiff line change
@@ -1,188 +1,33 @@
1-
"""The tarot review-artifact gate — a `PreToolUse` hook on `apply_operation` (advance).
2-
3-
For a repo that opts in (`Repo.capabilities["tarot_review"]`, see
4-
:mod:`panopticon.workflows.github_forge`), every other ITERATING responsibility is agent
5-
self-attested, but this one is real-verified: this hook intercepts the `advance` operation while
6-
the task is in ITERATING and runs `tarot strands check` / `tarot tour check` in `/workspace`,
7-
denying the tool call (with the checks' output as the reason, so it lands in the agent's context
8-
like a failed test would) unless they pass. A trivial diff (below a changed-line threshold) skips
9-
the checks and auto-resolves the responsibility instead — no tour to write for a one-line fix.
10-
11-
Registered unconditionally in `container/hooks.py` (like the turn-flip hooks), regardless of
12-
workflow — irrelevant calls (a different operation, a non-ITERATING state, a non-opted-in repo)
13-
resolve in the first couple of checks below and allow immediately. Deterministic and LLM-free:
14-
only subprocess + REST calls, so it's unit-tested with a fake command runner and a fake client
15-
(no real `tarot` binary needed), the same shape as :mod:`panopticon.container.hook`.
1+
"""Compatibility shim: the tarot review-artifact gate no longer runs in the container.
2+
3+
Enforcement moved **host-side** (:mod:`panopticon.taskservice.tarot_gate`), where the operator's
4+
`tarot` is installed and the task's clone already lives — the same directory the container sees at
5+
``/workspace``. Nothing in a task image needs tarot any more, and the agent reaches tarot through
6+
the `tarot_strand_seed` / `tarot_check` / `tarot_tour_scaffold` MCP tools instead.
7+
8+
This module survives only for **already-provisioned tasks**: `.claude/settings.json` lives in a
9+
task's persisted config volume and :func:`panopticon.container.config.update_json_config` merges
10+
rather than prunes, so a task respawned after this change still has a `PreToolUse` entry pointing
11+
at ``python -m panopticon.container.tarot_gate``. Deleting the module would make every
12+
`apply_operation` call fail with a hook error in those containers; allowing unconditionally here
13+
makes the stale entry a harmless no-op. Newly rendered settings don't wire it at all
14+
(:mod:`panopticon.container.hooks`), so this can be deleted once no such container remains.
1615
"""
1716

1817
from __future__ import annotations
1918

20-
import json
21-
import os
22-
import subprocess
19+
import contextlib
2320
import sys
24-
from collections.abc import Sequence
25-
from dataclasses import dataclass
26-
from typing import Any, Protocol, TextIO
27-
28-
import httpx
29-
30-
from panopticon.client import JsonObj, TaskServiceClient
31-
from panopticon.core.models import Status
32-
33-
#: The `Repo.capabilities` key that opts a repo into this gate (mirrors `docker_in_docker`).
34-
TAROT_REVIEW_CAPABILITY = "tarot_review"
35-
#: Optional per-repo override (an int) of the trivial-diff line threshold, under `capabilities`.
36-
TAROT_REVIEW_THRESHOLD_CAPABILITY = "tarot_review_threshold"
37-
#: The ITERATING responsibility this gate verifies (see `GithubForgeWorkflow.TAROT_REVIEW_ARTIFACTS`).
38-
RESPONSIBILITY_KEY = "tarot-review-artifacts"
39-
#: Below this many total changed lines (git `diff --numstat`, added + removed), the diff is
40-
#: considered trivial and the tarot checks are skipped entirely.
41-
DEFAULT_TRIVIAL_THRESHOLD = 20
42-
WORKSPACE = "/workspace"
43-
44-
45-
@dataclass(frozen=True)
46-
class CommandResult:
47-
"""The outcome of running one external command — never raises; the caller inspects it."""
48-
49-
returncode: int
50-
output: str # combined stdout+stderr
51-
found: bool = True # False when the executable itself wasn't found on PATH
52-
53-
54-
class CommandRunner(Protocol):
55-
def __call__(self, args: Sequence[str], *, cwd: str | None = None) -> CommandResult: ...
56-
57-
58-
def _subprocess_run(args: Sequence[str], *, cwd: str | None = None) -> CommandResult:
59-
try:
60-
proc = subprocess.run(list(args), cwd=cwd, capture_output=True, text=True, check=False)
61-
except FileNotFoundError:
62-
return CommandResult(returncode=127, output=f"{args[0]}: command not found", found=False)
63-
return CommandResult(returncode=proc.returncode, output=proc.stdout + proc.stderr)
64-
65-
66-
def _read_payload(stdin: TextIO) -> dict[str, Any]:
67-
"""Tolerantly parse the hook's stdin JSON; empty/invalid input yields an empty payload."""
68-
try:
69-
raw = stdin.read()
70-
except (OSError, ValueError):
71-
return {}
72-
if not raw or not raw.strip():
73-
return {}
74-
try:
75-
data = json.loads(raw)
76-
except json.JSONDecodeError:
77-
return {}
78-
return data if isinstance(data, dict) else {}
79-
80-
81-
def _allow() -> int:
82-
"""No stdout, exit 0 — Claude Code lets the tool call through unmodified."""
83-
return 0
21+
from typing import TextIO
8422

8523

86-
def _deny(reason: str) -> int:
87-
"""Structured `PreToolUse` denial: the reason string lands in the agent's context as the
88-
tool call's failure, the same seam a failed test's output would use."""
89-
print(
90-
json.dumps(
91-
{
92-
"hookSpecificOutput": {
93-
"hookEventName": "PreToolUse",
94-
"permissionDecision": "deny",
95-
"permissionDecisionReason": reason,
96-
}
97-
}
98-
)
99-
)
24+
def main(*, stdin: TextIO | None = None) -> int:
25+
"""Drain the hook payload and allow the tool call. No output, exit 0."""
26+
# A closed/absent stdin is not a reason to fail a tool call.
27+
with contextlib.suppress(OSError, ValueError):
28+
(stdin or sys.stdin).read()
10029
return 0
10130

10231

103-
def _opted_in(repo: JsonObj) -> bool:
104-
return bool((repo.get("capabilities") or {}).get(TAROT_REVIEW_CAPABILITY))
105-
106-
107-
def _threshold(repo: JsonObj) -> int:
108-
value = (repo.get("capabilities") or {}).get(TAROT_REVIEW_THRESHOLD_CAPABILITY)
109-
return value if isinstance(value, int) else DEFAULT_TRIVIAL_THRESHOLD
110-
111-
112-
def _changed_line_count(run: CommandRunner, *, base_ref: str) -> int:
113-
"""Total added+removed lines between ``base_ref`` and ``HEAD`` (a `diff --numstat` sum) — the
114-
trivial-diff heuristic. Binary files report ``-`` counts, which don't parse as digits and are
115-
skipped rather than counted."""
116-
result = run(["git", "-C", WORKSPACE, "diff", "--numstat", f"{base_ref}...HEAD"])
117-
total = 0
118-
for line in result.output.splitlines():
119-
added, _, rest = line.partition("\t")
120-
removed, _, _path = rest.partition("\t")
121-
for count in (added, removed):
122-
if count.isdigit():
123-
total += int(count)
124-
return total
125-
126-
127-
def _run_tarot_checks(run: CommandRunner) -> CommandResult | None:
128-
"""Run `tarot strands check` then `tarot tour check`; stop at the first failure (nothing to
129-
gain running both once one has already failed). ``None`` means both passed."""
130-
for args in (["tarot", "strands", "check"], ["tarot", "tour", "check"]):
131-
result = run(args, cwd=WORKSPACE)
132-
if result.returncode != 0:
133-
return result
134-
return None
135-
136-
137-
def main(
138-
*,
139-
client: TaskServiceClient | None = None,
140-
stdin: TextIO | None = None,
141-
run: CommandRunner = _subprocess_run,
142-
) -> int:
143-
payload = _read_payload(stdin or sys.stdin)
144-
tool_input = payload.get("tool_input")
145-
if not isinstance(tool_input, dict) or tool_input.get("operation") != "advance":
146-
return _allow()
147-
148-
env = os.environ
149-
task_id = env["PANOPTICON_TASK_ID"]
150-
client = client or TaskServiceClient(httpx.Client(base_url=env["PANOPTICON_SERVICE_URL"]))
151-
152-
task = client.get_task(task_id)
153-
if task.get("state") != "ITERATING":
154-
return _allow()
155-
156-
repo = client.get_repo(task["repo_id"])
157-
if not _opted_in(repo):
158-
return _allow()
159-
160-
base_ref = f"origin/{repo.get('default_base', 'main')}"
161-
if _changed_line_count(run, base_ref=base_ref) < _threshold(repo):
162-
client.resolve_responsibility(
163-
task_id,
164-
RESPONSIBILITY_KEY,
165-
Status.MET,
166-
comment="trivial diff — tarot review skipped",
167-
)
168-
return _allow()
169-
170-
failure = _run_tarot_checks(run)
171-
if failure is None:
172-
client.resolve_responsibility(
173-
task_id,
174-
RESPONSIBILITY_KEY,
175-
Status.MET,
176-
comment="verified by tarot strands check / tarot tour check",
177-
)
178-
return _allow()
179-
if not failure.found:
180-
return _deny(
181-
"`tarot` is not installed in this container. An opted-in repo must install it via "
182-
"its `image_layer_file` (see docs/repos.md)."
183-
)
184-
return _deny(failure.output)
185-
186-
18732
if __name__ == "__main__": # pragma: no cover
18833
raise SystemExit(main())

0 commit comments

Comments
 (0)