Skip to content

auto: a planned node's checklist and its tool grants are written together and never checked against each other #213

Description

@jitokim

Found by dogfooding auto on #204 option 2 — run 20260819-223604.575080000-1.

What happened

The run's last node, verify, FAILED. Not because the code was wrong — it was
asked to run gofmt and had no grant for it:

gofmt -l internal/serve → never ran. Both gofmt -l internal/serve and
gofmt -l ./internal/serve were refused:
Permission to use Bash has been denied because Claude Code is running in don't ask mode.
make fmt-check was refused the same way. I did not route around the denial
(e.g. via $GOROOT/bin/gofmt), so formatting is unverified — not
observed-clean, and not observed-dirty.

The node behaved exactly right: it refused to route around the denial and
refused to claim a check it could not run. The planner is what failed it.

The grants, from the run's own graph.json

impl      Read Grep Glob Edit Write  Bash(go *) Bash(git *) Bash(make *)
tests     Read Grep Glob Edit Write  Bash(go *) Bash(git *) Bash(make *)
review    Read Grep Glob             Bash(go *) Bash(git *) Bash(grep *)
changelog Read Grep Edit             Bash(git *) Bash(cat *)
verify    Read Grep                  Bash(git *) Bash(go *) Bash(grep *)

verify is the one node given neither Bash(make *) nor any gofmt grant —
and it is the node whose prompt lists a formatting check. impl and tests,
which need make less, both have it.

Why the existing lint cannot see this

handoff.LintToolGrants (internal/handoff/tool_grant_lint.go:54) warns when a
node declares no allowed_tools and no success_check.verify. Here every
node declared a list — an incomplete one. The predicate is "declared nothing",
so "declared less than my own prompt asks for" is invisible to it.

Its own message already names this failure mode in the other direction: "a tool
your settings do not pre-authorise is denied silently, and the node's own
result_matches passes on the prose explaining it."
This run is the honest
variant — the node failed loudly instead of passing on prose — and it still cost
the run.

The shape of a fix, not a decision

The candidate predicate is the #154 shape, which measured well: extract the
shell commands a planned node's prompt names (backticked gofmt -l …,
make local, go test …) and check each against that node's grants; warn on a
command the node is told to run and not permitted to.

It should be measured before it is shipped, exactly as #154 was: run it over
the existing plan corpus in ~/.oh-my-graph/runs/*/graph.json, count hits, and
hand-check the noise rate. A lint that cries wolf on planned graphs would be
worse than this bug — the repo's own claims linter had to be narrowed twice for
that reason.

Note the denominator is small (#16 in the notes): the plan corpus is still
accumulating, so the measurement may not be conclusive yet.

Cost of the bug

$8.35, and the run's verdict, for a formatting check. The code itself was fine —
make local passes on the branch (#212).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions