Skip to content

feat(work): keep AI pull requests in draft until review is confident - #215

Merged
TitusKirch merged 9 commits into
devfrom
ai/211-draft-pr-until-review-confident
Aug 3, 2026
Merged

feat(work): keep AI pull requests in draft until review is confident#215
TitusKirch merged 9 commits into
devfrom
ai/211-draft-pr-until-review-confident

Conversation

@TitusKirch

@TitusKirch TitusKirch commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Summary

CI here is gated on the pull request not being a draft, so a draft costs no CI minutes — but work-implement opened its PRs ready for review, so every implement round triggered a full run, the rounds a review was about to hand straight back included. The review loop forms its opinion about whether the work is finished before CI ever reports, so the expensive signal was being spent ahead of the cheap one.

The draft state now carries the loop's confidence:

  • work-implement opens every PR as a draft and leaves it there — first round and re-work rounds alike. It never marks a PR ready for review; an already-ready PR is left ready, never pushed back into draft.
  • work-review marks it ready only where its own review would accept, then waits for CI: green → ai: done, red → ai: changes requested with the failure as the feedback. Every other verdict leaves the draft alone, so a round the review is rejecting costs no CI at all.
  • Once ready, always ready. A CI failure never re-drafts, so the round answering it gets CI feedback directly instead of going dark again.

Net effect: CI runs once per "the review thinks this is finished" moment rather than once per push.

Reading CI after the un-draft

Two properties of that wait are easy to get subtly wrong, and both are specified rather than left to the recipe's shape.

A skipping check is never a pass, so those rows are dropped rather than counted. A draft gate on the job — which is what all three of this repo's workflows use — does not report nothing while the PR is a draft; it reports skipping, one row per gated job. gh pr ready fires a fresh run for the same head SHA, but that run takes seconds to register, so a poll inside that window sees a full, non-failing list of stale rows. The poll therefore discards every skipping bucket outright and reads only the jobs that actually answered: the list falls empty and the existing "none yet is not none at all" rule handles it. A job declining on a condition of its own (a paths filter, a matrix exclusion) is dropped by the same rule and for the same reason — it said nothing about this head — and lands in the fourth table row, where the accept rests on the verify this review ran itself.

The wait's base exists on every round, and bounds the wait only. It is the later of the ready_for_review event and this head's own commit — not the event alone, which is written once: the round answering a CI failure finds the PR already ready, writes no new event, and would otherwise recompute its deadline from a flip hours old and escalate before CI reported. A PR that never carried the event at all (one a human opened ready) has the head as its base. An underivable base escalates to ai: needs human rather than being read as an elapsed one — the same treatment an unreadable round count and an unreadable draft state get. Because that base falls back to the head commit it is deliberately not used to decide which check results count; getting it wrong can only make the wait too long or too short.

The two open questions in the issue

How long to wait, and what a timeout means. A new work.review.timeout (seconds, default 600) bounds the wait, mirroring release.timeout. It is per-repo because CI duration is: a fixed number would escalate every issue in a repo whose pipeline outlasts it. Still pending when it elapses → ai: needs human, never an accept and never back into draft — no verdict may rest on a gate that has not reported. Its description says what it is measured from, since on a re-work round the budget starts at the push and the review's own worktree, install and verify are spent inside it.

"No checks at all" needed no new rule: it is the existing unknown case, and step 5's second source already answers it — the review runs the repo's own verify against the pushed head, so the accept rests on that, not on CI's silence. What did need stating is that none yet and none triggerable are different readings, told apart by reading which workflows the base runs before reading their result.

codeql.yml's missing draft gate is in scope, because without it the issue's premise does not hold: CodeQL ran on every draft push and — lacking ready_for_review in its trigger types — would never have re-run at the moment the review un-drafts. It now carries the same gate, written with an event-name guard because it also runs on push, on a schedule and on dispatch; none of those is gated, so nothing on main/dev goes unanalysed.

Type of change

  • Bug fix
  • New skill
  • Skill update
  • Breaking change
  • Documentation
  • Internal / chore

Checklist

  • pnpm verify passes locally
  • Tests added or updated — or not applicable
  • Affected skill's SKILL.md updated (description, examples)
  • Docs updated (README / skill README / CHANGELOG entry handled by release-please)
  • Commit messages follow Conventional Commits

Test discipline

Mostly a prose change (SKILL.md / REFERENCE.md across both loops), which drives tdd not at all. One slice does touch something a test reaches — the work.review.timeout config key — and that slice was driven red-green: the seam is the schema contract the issue's "how long should work-review wait" implies, the failing assertions went into test/schema.test.ts first (/work/review must NOT have additional properties), and the schema key followed.

Rounds 2 and 3 are prose-only — every finding was about what the REFERENCE's recipe specifies, not about the schema — so tdd drove nothing in them, which is the defined outcome for a prose change rather than a skipped step. Round 3's claims were measured against this pull request rather than assumed: codeql.yml was read at the head and confirmed to carry no paths: filter, which is what the corrected CI-gotchas bullet in CLAUDE.md / AGENTS.md now says.

atomic-commit was driven for the commit split; its confirmation step could not run unattended and was skipped, which is the documented deviation. The PR was opened with the forge CLI rather than the pull-request skill, same base and head — the stated fallback.

Related issues

Refs #211

The other two pull-request workflows already decline to run while a PR is a
draft; CodeQL did not, so a draft still paid for a full analysis. Add the same
gate plus the ready_for_review trigger type the un-draft needs, written with an
event-name guard because this workflow also runs on push, on a schedule and on
dispatch — none of which is gated, so nothing on main or dev goes unanalysed.

Refs #211
The implement loop opened its PR ready for review, so every round triggered a
full CI run — including the rounds the review was about to hand straight back.
The review forms its opinion before CI ever reports, so the expensive signal was
being spent ahead of the cheap one.

Let the draft state carry the loop's confidence instead: work-implement opens
every PR as a draft and never un-drafts it, and work-review marks one ready only
where its own review would accept, then waits for CI — green accepts, red hands
back the failure as feedback. Once ready a PR stays ready, so the round
answering a CI failure sees CI rather than going dark again.

The wait needs a bound, and CI duration is per-repo, so work.review.timeout
(default 600s) sizes it; still pending when it elapses escalates to needs human,
because no verdict may rest on a gate that has not reported. A head that
triggers no workflow at all is the existing unknown case and is already answered
by the verify run the review performs itself.

Refs #211
The CI-gotchas bullet read as an incidental fact about draft PRs. The work loop
now depends on it deliberately, and each pull-request workflow needs both halves
— the draft gate and ready_for_review in its trigger types — or the un-draft
fires nothing.

Refs #211
@TitusKirch

Copy link
Copy Markdown
Owner Author

AI review — changes requested (round 1 of 3, head eeaa36b)

The gate. pnpm verify run against the pushed head in a throwaway worktree, installed from the head's own lockfile (pnpm install --frozen-lockfile) — green, 180/180 tests, including the new the review CI-wait timeout suite. The forge's own checks are unknown rather than green here: all three PR workflows report skipping because the PR is a draft, which is the mechanic working as intended.

Scope is met. Both loops carry the rule, work-implement never un-drafts, work-review un-drafts only on an accept and never re-drafts, and all three open questions from the issue are answered — work.review.timeout for the wait, the existing unknown case for "no checks at all", and codeql.yml given both halves of the gate. Three things in the new prose do not hold up.

1. A stale skipping list reads as green, so an accept can be written with CI never having run (blocking)

The poll's case in work-review/REFERENCE.md sends everything that is not fail/pending/cancel to green, and its own comment states the assumption: "every bucket is pass or skipping". But skipping is exactly what a repo with a job-level draft gate reports before the un-draft. Measured on this pull request, right now, while it is still a draft:

$ gh pr checks 215 --json name,bucket
[{"bucket":"skipping","name":"Verify","workflow":"CI"},
 {"bucket":"skipping","name":"Analyze (...)","workflow":"CodeQL"},
 {"bucket":"skipping","name":"Conformance","workflow":"Skill conformance"}]

gh pr ready fires a fresh run for the same head SHA, but that run takes seconds to register. Poll inside that window and raw holds three skipping lines — no fail, no pending, and not empty — so the recipe answers green and the verdict is done on a head CI never analysed. The prose guards the adjacent case ("An empty check list moments after gh pr ready is usually a workflow that has not registered its run yet"), but the list is not empty here, so that guard never engages. This change also widens the surface: adding the draft gate to codeql.yml turns a third check into a skipping entry that previously reported a real result.

The reading that is missing is "these results predate the un-draft". Something has to tie the buckets to a run that started after the ready flip — the check run's startedAt against the ready_for_review timestamp, or treating skipping as unknown rather than folding it into green.

2. The derived deadline only works for the round that un-drafts (blocking)

"The deadline is derived, not carried" measures the wait from the ready_for_review timeline event. That event is written once. On the round that answers a CI failure — the path this change deliberately creates ("the following rounds get CI feedback directly rather than going dark again") — the PR is already ready, gh pr ready is correctly skipped, no new event is written, and the deadline is recomputed from an event hours or days old. It is already past on the first poll, so any CI still running yields needs human with no wait at all: the round designed to see CI is the round most likely to escalate before CI reports.

The same paragraph also has no answer for a PR that never carried a ready_for_review event — one a human opened ready, or any PR already in flight when this lands — where the deadline is not derivable at all. Elsewhere this REFERENCE is careful that an unreadable value is not a zero (the round count); the deadline needs the same treatment. A base defined on every round fixes both: the later of the ready_for_review event and the head commit's push, or the newest matching run's own start time.

3. The recipe emits a fifth reading that neither verdict table consumes (minor)

*cancel*) echo unknown is a real outcome — both ci.yml and skills-conformance.yml set cancel-in-progress: true, so a run superseded mid-wait is routine here — but the four-row table in REFERENCE.md and the four-row table in SKILL.md both stop at green / red / pending-at-deadline / none-triggerable. A reviewer that reads unknown has no verdict to write. State what it does (poll on, since a supersede implies a replacement, and escalate if none appears by the deadline) or fold it into pending.

Nit, non-blocking

In work-review/SKILL.md both new links read [the draft gate](REFERENCE.md#marking-ready-then-waiting-for-ci). They resolve, but "The draft gate" is the section in work-implement's REFERENCE; the anchor is this skill's own "Marking ready, then waiting for CI". Either relabel the link or point it at the other file.

Checked and fine

  • ci.yml and skills-conformance.yml already carried ready_for_review before this change, so the CLAUDE.md claim about all three PR workflows is accurate; dev-pr.yml is push-triggered and correctly out of the count.
  • The codeql.yml if: guard is right to test github.event_namegithub.event.pull_request.draft is null on push/schedule/dispatch, and null == false is false in GitHub expressions, so the bare form the other two workflows use would have silently disabled those triggers.
  • pull-request's --draft path exists and it already declines to change an existing PR's draft state, so "ask it for a draft" and "leave a re-work's draft state as found" are both reachable as written.
  • work-review's allowed-tools already permits the gh pr ready call, and the guardrail's reasoning for why an un-draft is not a read-only violation is sound.
  • No secrets in the diff.

The poll folded every non-fail, non-pending bucket into green, and `skipping`
is exactly what a job-level draft gate reports before the un-draft. `gh pr
ready` fires a fresh run for the same head SHA but that run takes seconds to
register, so a poll inside the window saw three stale `skipping` rows — not
empty, so the "none yet" guard never engaged — and wrote `done` on a head CI
had never analysed.

Filter the buckets by `startedAt` against the moment the head became
reviewable: the stale rows drop out, the list reads empty, and the existing
"none yet is not none at all" rule handles it. A `skipping` that survives the
filter is a job declining on a condition of its own and is a real pass.

The deadline needed the same base, and it could not come from the
`ready_for_review` event alone — that event is written once, so the round
answering a CI failure recomputed it from a flip hours old and escalated
before CI reported, and a PR that never carried the event had no base at all.
Take the later of the un-draft and this head's own commit, and treat an
underivable base as needs-human rather than as an elapsed one, the way an
unreadable round count already is.

Also stop branching on `gh pr checks`' exit status, which is non-zero for
pending and failing runs alike and was blanking the payload it had just read,
and fold a cancelled run into pending so the recipe no longer emits a fifth
reading neither verdict table consumes.

Refs #211
The draft gate reads as "a draft costs nothing to run", and CLAUDE.md went
further and called a draft's check list empty. Neither is what a gate on the
job produces: it still registers a check run and reports it as `skipping`, one
row per gated job, so all three of this repo's workflows leave a full,
non-failing list on every draft.

That is the shape anything reading those checks has to allow for, so state it
where the gate is specified rather than leaving the review side to discover it.

Refs #211
@TitusKirch

Copy link
Copy Markdown
Owner Author

Re-work round 2 — review feedback addressed (head 6e2e48a)

All three findings and the nit are fixed. pnpm verify green in a fresh worktree installed from the head's own lockfile (pnpm install --frozen-lockfile) — 180/180. The PR stays a draft, as the mechanic this PR introduces requires: un-drafting is the review's flip, not the implementer's.

1. A stale skipping list reading as green — fixed

Confirmed on this PR before changing anything: gh pr checks 215 --json name,bucket,startedAt returns three skipping rows, all with a pre-flip startedAt. So the list is non-empty on a draft and the "empty list" guard genuinely never engaged, exactly as reported.

The poll now filters by start time before reading any bucket:

checks=$(gh pr checks "$pr" --json bucket,startedAt 2>/dev/null)
raw=$(printf '%s' "$checks" | jq -r --arg since "$since" \
  '.[]? | select(.startedAt >= $since) | .bucket' 2>/dev/null) || raw=

The stale rows drop out, the list reads empty, and that lands in the none case the "none yet is not none at all" paragraph already answers correctly — so the fix routes the race into an existing, correct branch rather than adding a new one. That paragraph now says it takes a post-filter empty list, not just a literally empty one.

Both halves of your suggestion are in, not just one: the freshness filter and the rule that a bucket only speaks about a head if its run started after that head became reviewable. A skipping that survives the filter is stated separately as a real pass — its job declined on a paths filter or a matrix exclusion, which is the pipeline answering about this head — so the fix does not turn every legitimately-skipped job into a permanent stall.

2. The derived deadline only working for the un-drafting round — fixed

The base is now the later of the newest ready_for_review event and this head's own commit date, recomputed per poll. That covers all three cases you named: the un-drafting round takes the flip (newer than the head it just judged); a re-work round on an already-ready PR takes its new head, so the budget starts when the work did instead of at an ancient flip; a PR that never carried the event has the head alone.

Your point about an unreadable value not being a zero is in as its own rule — an empty base escalates to needs human saying the base could not be read, cross-referencing the round count that already works this way, since reading it as elapsed escalates on an API hiccup and reading it as absent waits forever.

The --paginate per-page trap the round-count section documents applies to the timeline read here too; the sort | tail -1 picks across pages, and the comment says so.

3. The fifth reading — folded into pending

Took the fold option you offered. *cancel*) echo pending with the reason stated: a supersede implies a replacement run, so it is "not a result yet" rather than a verdict; if no replacement reports, the deadline answers it as any other pending does. Both four-row tables now consume every reading the recipe emits.

Nit — fixed

Three mislabelled links, not two — the work.review.timeout config row had the same one. The two that describe this loop's un-draft now read "marking ready, then waiting for CI"; the one on line 78 describing the implement loop's behaviour names The draft gate in work-implement's REFERENCE without a link, matching how this file already cross-references that skill.

Two things found while fixing the above

  • gh pr checks' exit status was blanking the payload. The old raw=$(… ) || raw= cleared the variable on any non-zero exit — but that command exits 8 while runs are pending and 1 on a failure, so *fail* was unreachable and a failing run would have read as none. The status is no longer branched on, and the comment names the codes.
  • The draft gate's own description was inaccurate, in work-implement's REFERENCE and in CLAUDE.md (which called a draft's check list "empty"). A gate on the job — what all three workflows here use — still registers a check run and reports skipping; a gate on the trigger reports nothing. That distinction is the root of finding 1, so it is now stated where the gate is specified rather than left for the review side to rediscover.

Both are corrections to this PR's own prose, not new scope.

@TitusKirch

Copy link
Copy Markdown
Owner Author

AI review — changes requested (round 2 of 3, head 6e2e48a)

The gate. pnpm verify run against the pushed head in a throwaway worktree, installed from the head's own lockfile (pnpm install --frozen-lockfile) — green, exit 0, 180/180 tests including the new the review CI-wait timeout suite. The forge's own checks are still unknown, as the mechanic intends: three skipping rows because the PR is a draft.

Round 1's findings, re-checked. Finding 2 (the deadline base) is properly fixed — max(ready_for_review, head commit) gives every round a base, the un-drafting round takes the flip and a re-work round takes its new head, and an empty base escalates rather than reading as elapsed. Finding 3 is fixed: the recipe now emits exactly none / red / pending / green, and both four-row tables consume all four. The link nit is fixed.

Finding 1 is fixed for the main path only, and the round-2 fix for finding 2 is what reopens it.

1. The head-commit fallback base does not filter the stale skipping rows (blocking)

skipping still folds into green — the *) arm is unchanged. The only thing standing between a stale row and a done is now select(.startedAt >= $since), and that holds only when $since is the flip. The new since is max(ready, head), and the prose deliberately allows ready to be empty (|| ready=, plus max // empty) and fall through to the head commit — "a PR with no ready_for_review event has the head alone, which is a base rather than nothing". The escalation rule only fires when both sources are empty.

Measured on this pull request, right now:

head commit  committedDate  2026-08-02T15:42:31Z
skipping rows  startedAt    2026-08-02T15:43:04Z   (x3, CI / CodeQL / Skill conformance)

The draft-gate rows are registered by the push, so their startedAt is later than the head commit — by 33s here, and structurally always (commit → push → run start). So with since = head commit, all three survive the filter: no fail, no pending, no cancel, and not empty → *)greendone on a head CI never analysed. That is round 1's finding verbatim.

Enumerating the un-drafting round's four propagation states makes the surviving window exact:

ready_for_review visible in the timeline new check runs registered $since reading
yes no flip none → poll on safe
yes yes flip pending → poll on safe
no yes head commit pending (new rows) → poll on safe by luck
no no head commit green false done

So the fix's correctness rests on an unstated race: the flip must be visible in the timeline before the new check runs register. The prose already says that registration "takes seconds" — which is precisely the window the first poll lands in, and $since is recomputed on every poll. A transient timeline read (404 / rate limit / 5xx) puts the run in the bottom row deterministically, and the same paragraph that insists an unreadable round count must not read as zero lets this one read as a permissive base without a word.

Same pattern, second instance, two lines above: draft=$(gh pr view "$pr" --json isDraft --jq '.isDraft') || draft=. A failed read leaves draft empty, [ "" = 'true' ] is false, gh pr ready is silently skipped, the PR stays a draft — and the poll then takes the same bottom row and writes green. Nothing asserts the flip actually happened.

Two ways to close it, either sufficient:

  • Assert the base instead of degrading it. Where this run un-drafted the PR (or the PR is still a draft), the flip moment is required; an unreadable one is the underivable case → needs human, exactly as an empty $since already is. The head commit is a legitimate base only for a PR that was never a draft.
  • Stop folding skipping into green at all — round 1's other option, and race-free. Drop skipping rows from the list rather than counting them: a head that genuinely needs no check then lands in none, which the "none yet is not none at all" paragraph already routes correctly, and where the accept rests on step 5's own verify as the fourth table row already says. That removes the timestamp filter as a single point of failure rather than tuning it.

Non-blocking

  • work.review.timeout's documented meaning and the re-work round's base disagree. The schema says the key bounds "the review unit's wait for CI after it marks a draft pull request ready", but on a round answering a CI failure the base is the head commit — so the budget is already being spent during the reviewer's own worktree create, pnpm install and verify before the first poll. The behaviour is defensible (it measures CI's own elapsed time, and it fails toward needs human), but say so in the key's description, because as written a repo sizing timeout to its pipeline duration will size it short.
  • select(.startedAt >= $since) drops a row whose startedAt is null (jq orders null before any string). A queued-but-not-started check therefore reads as "not yet" rather than as pending. Safe direction, worth one clause.
  • *fail* sits before *pending*, so a red is returned while sibling jobs are still running. Given ci.yml's !cancelled() fan-out that is probably intended — but the feedback then carries only the jobs that had reported, which is worth stating.
  • The adjacent CI-gotchas bullet is factually wrong, and this PR is what makes it load-bearing. CLAUDE.md / AGENTS.md still say "CodeQL only fires on **/*.{js,ts,mjs,cjs} or workflow changes", and the PR description repeats it ("it is path-scoped, so it rarely fires on a prose-only PR"). codeql.yml carries no paths: filter — verified on feat(work): settle the owner decisions behind the needs-human backlog #207, a docs-and-skills-only PR, where both Analyze matrix legs ran and passed. It fires on every PR to main/dev. The gate this PR adds is therefore worth more than the description claims, but the new poll explicitly depends on "read which workflows the base runs before reading their result", and that bullet is the repo's own authority for it. Pre-existing, so not a blocker — but this is the change that turns it into a correctness input.

Checked and fine

  • All three PR workflows confirmed at the head to carry both halves: ready_for_review in types and a job-level if draft gate. codeql.yml's github.event_name != 'pull_request' || … guard is correct — push, schedule and workflow_dispatch keep analysing.
  • gh pr checks --json bucket,startedAt verified to accept both fields and to return ISO-8601 UTC that sorts lexicographically, as the recipe assumes.
  • The --paginate-prints-per-page trap is handled: max // empty per page plus sort | tail -1 across them picks the true maximum.
  • Not branching on gh pr checks' exit status is right — it exits 8 while pending and 1 on failure or with no checks; the old form did blank the payload.
  • case bucket names match gh's actual vocabulary (pass / fail / pending / skipping / cancel); no substring collisions.
  • Folding cancel into pending is sound given cancel-in-progress: true on all three workflows.
  • allowed-tools: Bash covers gh pr ready; the guardrail's argument that an un-draft is not a read-only violation holds.
  • pull-request opens ready by default and declines to change an existing PR's draft state, so "ask it for a draft" and "leave a re-work's draft state as found" are both reachable as written.
  • CLAUDE.md and AGENTS.md are byte-identical (the mirror test passes), and no generated artifact drifted (pnpm skills:check green).
  • No secrets and no attribution lines in the diff.

The CI poll folded every non-fail, non-pending bucket into green, so the
`skipping` rows a job-level draft gate reports could be read as CI having
passed. Timestamping them against the deadline base only narrowed that
window: the base legitimately falls back to the head commit, which predates
the rows the push registers, so one transient timeline read reopened it.

Drop `skipping` outright instead. A job that declined said nothing about
this head — whether it declined on the draft gate or on a `paths` filter of
its own — so the list reads empty and lands in the "none yet is not none at
all" branch that already handles it correctly. The poll no longer needs
`startedAt`, and `$since` now bounds the wait only, never correctness.

An unreadable draft state gets the same treatment as every other unreadable
value here: escalate rather than take the permissive reading, instead of
letting an empty `isDraft` skip the flip while the loop proceeds as though
the pull request had gone ready.

Refs #211
The key's description said it bounds the wait "after it marks a draft pull
request ready", but the base is the later of the un-draft and the head's own
commit — so on a round answering a CI failure, where nothing un-drafts, the
clock starts at the push and is partly spent by the review's own worktree,
install and verify before the first poll. A repo sizing the timeout to its
pipeline duration would size it short.

Refs #211
The CI-gotchas bullet claimed CodeQL only fires on JS/TS or workflow
changes. `codeql.yml` carries no `paths:` filter at all: both matrix legs
run on every pull request to main/dev, verified on a docs-and-skills-only
PR. Pre-existing, but the work loop's CI read now depends on knowing which
workflows a head actually triggers, which makes this bullet an input rather
than trivia.

Refs #211
@TitusKirch

Copy link
Copy Markdown
Owner Author

Re-work round 3 — review feedback addressed (head 4637048)

The blocking finding and all four non-blocking ones are fixed. pnpm verify green in a fresh worktree installed from the head's own lockfile (pnpm install --frozen-lockfile) — exit 0, 180/180. The PR stays a draft: un-drafting is the review's flip, never the implementer's.

1. The head-commit fallback base not filtering the stale skipping rows — fixed by removing the filter

Took the second of the two options, the race-free one: skipping is dropped, never counted.

checks=$(gh pr checks "$pr" --json bucket 2>/dev/null)
# `skipping` is dropped, never counted — it is a job declining to answer, not an answer.
raw=$(printf '%s' "$checks" | jq -r '.[]? | select(.bucket != "skipping") | .bucket' 2>/dev/null) || raw=

Your enumeration of the four propagation states is what settled it. Tuning $since only ever narrows the bottom row of that table; it cannot remove it, because the fallback base is structurally older than the rows the push registers — commit → push → run start, in that order, always. So the timestamp was doing correctness work it can never be trusted with. It is gone from the poll entirely (--json bucket alone now), and $since is stated to bound the wait and nothing else: getting it wrong can make the wait too long or too short, never make an unanalysed head read green.

The honest post-flip skipping — a job declining on its own paths filter or a matrix exclusion — is dropped by the same rule, and that is the correct reading rather than a regression: a job that declined said nothing about this head, so it is not a pass. Where those are the only rows the list falls to none, which the "none yet is not none at all" paragraph already routes, and the fourth table row already rests the accept on step 5's own verify. Nothing stalls, because none is a reading with a verdict rather than a wait.

The second instance you found is fixed too, and asserted rather than degraded. draft= on a failed read no longer falls through a [ "$draft" = 'true' ] test that reads emptiness as "already ready":

case $draft in
true)  gh pr ready "$pr" || echo unreadable ;;
false) : ;;                                     # already ready — a re-work round, nothing to flip
*)     echo unreadable ;;                       # never read as "already ready"
esac

Unreadable → needs human, cross-referenced to the round count and the underivable base, under the one rule the file now states outright: an unreadable value is never the permissive reading. A failed gh pr ready is the same fact one line later and is caught as well.

Non-blocking, all four

  • work.review.timeout's documented meaning now says what it is measured from — the later of the un-draft and the head commit — and that a re-work round therefore spends part of the budget on the reviewer's own worktree, install and verify before the first poll. Both the schema description and the REFERENCE say to size it against that wall-clock span rather than the pipeline's duration, since the error direction is safe (needs human) but avoidable.
  • startedAt being null is moot: the poll no longer reads the field.
  • *fail* before *pending* is now stated as deliberate, with its consequence: the feedback carries only the jobs that had reported, so it must be written as "CI failed, here is what had come in" and say the run was still in flight — otherwise a re-work fixes the one visible failure and rediscovers the rest a round later.
  • The CodeQL gotcha bullet is corrected in CLAUDE.md and AGENTS.md (byte-identical, mirror test green). Verified at the head: codeql.yml carries no paths: filter, so both matrix legs run on every PR to main/dev. The bullet now says so and says why it matters here — the poll depends on reading which workflows a head triggers, and this is the repo's own authority for that. The PR description's "path-scoped, so it rarely fires" claim is gone with it.

Commits

  • fix(work-review): never read a skipping check as a pass — the poll, the draft-state assertion, and the $since reframing across work-review's REFERENCE and SKILL plus work-implement's REFERENCE.
  • docs(work-review): say what the CI-wait budget is measured from — the schema key.
  • docs: correct the CodeQL gotcha — it has no paths filterCLAUDE.md / AGENTS.md.

@TitusKirch

Copy link
Copy Markdown
Owner Author

AI review — accepted (round 3 of 3, head 4637048)

The gate. pnpm verify run against the pushed head in a throwaway worktree, installed from the head's own lockfile (pnpm install --frozen-lockfile) — green, exit 0, 180/180 including the review CI-wait timeout suite. The forge's own checks are unknown, as this mechanic intends: three skipping rows because the PR is a draft. The accept rests on the verify above, which is what step 5's second source exists for.

The round-2 blocker is fixed, and measured rather than taken on trust

Round 2's finding was that skipping still fell into the *) arm, with select(.startedAt >= $since) as the only guard — and that $since legitimately falls back to the head commit, which is structurally older than the rows the push registers. Confirmed on this pull request that the premise still held before this round: head commit 2026-08-02T15:42:31Z, three skipping rows at 15:43:04Z, 33 s later.

The fix removes the failure mode rather than narrowing it. $since is out of the poll entirely (--json bucket alone) and is now stated to bound the wait only; skipping is dropped before any bucket is read. Ran the recipe as written against this PR's live check data:

$ gh pr checks 215 --json bucket
[{"bucket":"skipping"},{"bucket":"skipping"},{"bucket":"skipping"}]

$ jq -r '.[]? | select(.bucket != "skipping") | .bucket'
(empty)   ->  case ''  ->  none

none, not green — on the exact input that produced green two rounds ago. Verified gh emits the literal bucket string skipping (all three rows, live), so the exact-match filter meets the vocabulary the case arms glob against. Re-derived the four propagation states from round 2's table: with the timestamp gone, the bottom row (no flip visible / no new runs / head-commit base) now lands in none like the others, so there is no permissive base left to hit. Post-flip honest skips — a paths filter, a matrix exclusion — drop by the same rule and land in the fourth table row, which is the correct reading: a job that declined said nothing about this head.

The second instance is fixed and asserted rather than degraded: case $draft in true|false|*) replaces the [ "$draft" = 'true' ] test, so an unreadable isDraft and a failed gh pr ready both escalate instead of silently skipping the flip. Stated as one rule — an unreadable value is never the permissive reading — cross-referenced to the round count and the underivable base.

Round 1's finding 2 (max(ready_for_review, head-commit)) and finding 3 (cancel folded into pending; both four-row tables consume every reading) remain fixed and untouched by this round.

Non-blocking items from round 2, all four

  • work.review.timeout's description now says what the budget is measured from and that a re-work round spends part of it on the reviewer's own worktree, install and verify. Schema and REFERENCE agree.
  • startedAt being null is moot — the field is no longer read.
  • *fail* before *pending* is now stated as deliberate, with the obligation to write the feedback as "CI failed, here is what had come in" and name the run as still in flight.
  • The CodeQL gotcha is corrected in CLAUDE.md / AGENTS.md. Verified independently at the head: codeql.yml carries no paths: filter, so both matrix legs analyse every PR to main/dev. The corrected bullet says so.

Checked at the head

  • All three PR workflows confirmed to carry both halves — ready_for_review in types and a job-level if draft gate. codeql.yml's github.event_name != 'pull_request' || … guard is right: push, schedule and workflow_dispatch keep analysing, so nothing on main/dev goes unanalysed.
  • Scope is met against the issue body: implement never un-drafts (SKILL bullet + guardrail + The draft gate in its REFERENCE), review un-drafts only on an otherwise-done verdict, green → done / red → changes-requested, and nothing ever re-drafts. All three open questions are answered — work.review.timeout for the wait, the existing unknown case for "no checks at all", codeql.yml brought in scope.
  • pull-request supports --draft on create and declines to change an existing PR's draft state on update, so both halves of the implement-side rule are reachable as written.
  • allowed-tools: Bash covers gh pr ready; the guardrail's argument that an un-draft is a verdict in the forge's vocabulary rather than a write to the code holds.
  • CLAUDE.md / AGENTS.md byte-identical and no generated artifact drifted (both inside the green pnpm verify).
  • No secrets, no attribution lines, no agent self-naming in the diff.
  • dev carries no branch protection and the repo has no CODEOWNERS, so no human review is mechanically required for these paths.

One note for whoever merges

This PR is still a draft — reviewed under the currently installed work-review, which has no un-draft step, so the flip this change introduces was not applied to the change itself. CI has therefore never analysed this head; the accept rests on the verify run above. It needs marking ready for review before it can merge, and that run is the first real exercise of the mechanic.

# Conflicts:
#	skills/work/work-implement/SKILL.md
#	skills/work/work-review/SKILL.md
#	test/schema.test.ts
@TitusKirch
TitusKirch marked this pull request as ready for review August 3, 2026 13:54
@TitusKirch
TitusKirch merged commit 84f2f55 into dev Aug 3, 2026
8 checks passed
TitusKirch added a commit that referenced this pull request Aug 3, 2026
The gate landed with #215 and binds every workflow added later: a PR workflow
needs both the draft gate and ready_for_review in its trigger types, or the
un-draft fires nothing and the check silently never runs. Until now that
reasoning lived only in CLAUDE.md and work-implement's REFERENCE, so the
decision log read as complete without it.

Scoped deliberately: #217 and #218 stay unrecorded here, their rationale
travelling with their own skill as ADR-0014 decided.
@TitusKirch
TitusKirch deleted the ai/211-draft-pr-until-review-confident branch August 3, 2026 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant