-
-
Notifications
You must be signed in to change notification settings - Fork 108
191 lines (174 loc) · 9.55 KB
/
Copy pathbot-review.yml
File metadata and controls
191 lines (174 loc) · 9.55 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
name: Wheels Bot — Reviewer
# Single-pass reviewer. The former Reviewer A / Reviewer B critique loop
# (bot-review-b.yml + /review-the-review + /respond-to-critique) was retired
# per maintainer decision 2026-06-11: the loop was expensive and flaky, and
# its marginal catch rate no longer justified a second model pass. Reviewer
# B's anti-sycophancy / false-positive mandate now lives inside /review-pr
# itself as an explicit self-adversarial step before posting.
#
# One trigger path: pull_request (PR opens, syncs, or marks ready). The
# Reviewer submits its single substantive review of the diff via /review-pr.
on:
pull_request:
types: [opened, synchronize, ready_for_review]
branches: [develop]
permissions:
contents: read
concurrency:
# The group key retains the legacy "review-a" name: it is shared with
# bot-review-fork.yml so a fork review and an internal review for the same
# PR number can never overlap, and renaming it buys nothing. A PR is either
# fork or internal, so in practice only one of the two workflows matches.
group: wheels-bot-review-a-${{ github.event.pull_request.number }}
cancel-in-progress: false
jobs:
review:
name: Reviewer
runs-on: ubuntu-latest
timeout-minutes: 20
# Reviews bot PRs (even draft) and human ready-for-review PRs.
if: |
vars.WHEELS_BOT_ENABLED == 'true'
&& github.event.pull_request.user.login != 'dependabot[bot]'
&& (github.event.pull_request.user.login == 'wheels-bot[bot]'
|| github.event.pull_request.draft == false)
steps:
- name: Generate App token
id: app-token
uses: actions/create-github-app-token@v2
with:
app-id: ${{ secrets.WHEELS_BOT_APP_ID }}
private-key: ${{ secrets.WHEELS_BOT_PRIVATE_KEY }}
- name: Resolve PR info
id: pr
env:
# Pass event values through env (never interpolate ${{ }} straight
# into the script body). head.sha is captured exactly ONCE here, at
# run start — it is both what the Checkout step pins and the marker
# SHA threaded into the prompt (issue #2848).
PR_NUM: ${{ github.event.pull_request.number }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
set -euo pipefail
if ! [[ "$PR_NUM" =~ ^[0-9]+$ ]]; then
echo "::error::PR number is not numeric: $PR_NUM"
exit 1
fi
if ! [[ "$HEAD_SHA" =~ ^[0-9a-fA-F]{7,40}$ ]]; then
echo "::error::head SHA is not a hex commit id: $HEAD_SHA"
exit 1
fi
echo "pr_num=${PR_NUM}" >> "$GITHUB_OUTPUT"
echo "sha=${HEAD_SHA}" >> "$GITHUB_OUTPUT"
- name: Checkout PR head
uses: actions/checkout@v6
with:
ref: ${{ steps.pr.outputs.sha }}
fetch-depth: 0
- name: Skip check
id: gate
uses: ./.github/actions/wheels-bot-skip-check
with:
target-type: pr
target-number: ${{ steps.pr.outputs.pr_num }}
# No trailing colon: the review marker is
# `wheels-bot:review-a:<pr>:<sha>` with no suffix. Adding a trailing
# `:` made this a no-op gate (issue #2558). The marker keeps its
# legacy `review-a` name so reviews posted before the single-reviewer
# consolidation still satisfy idempotency on already-reviewed SHAs.
marker-pattern: 'wheels-bot:review-a:${{ steps.pr.outputs.pr_num }}:${{ steps.pr.outputs.sha }}'
github-token: ${{ steps.app-token.outputs.token }}
- name: Run Reviewer
if: steps.gate.outputs.skip == 'false'
uses: anthropics/claude-code-action@v1
with:
# Allows the App's bot identity (and github-actions[bot] for
# consistency). The Reviewer runs on the bot's own PRs — a
# legitimately bot-driven trigger.
allowed_bots: 'wheels-bot[bot],github-actions[bot]'
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
github_token: ${{ steps.app-token.outputs.token }}
# Thread the SHA resolved in "Resolve PR info" — the exact commit the
# Checkout step pinned — into the prompt as a second argument. The
# model emits the idempotency marker from this value instead of
# re-deriving it with `gh pr view`, which races with pushes that land
# mid-session and left the marker pointing at the wrong commit
# (issue #2848). The Run Reviewer step's Bash allowlist is gh +
# read-only git (no echo/printenv), so the model can't read a step
# env var — the SHA must travel in the prompt text, the same channel
# the PR number already uses.
prompt: |
/review-pr ${{ steps.pr.outputs.pr_num }} ${{ steps.pr.outputs.sha }}
# Model policy: judging gate = opus (was Fable 5 until its 2026-06 deactivation), coding stages = opus, janitorial = sonnet.
claude_args: |
--model claude-opus-4-8
--max-turns 250
--allowedTools "Bash(gh:*),Bash(git log:*),Bash(git diff:*),Bash(git show:*),Bash(git grep:*),Bash(git status),Read,Grep,Glob"
# Post-submission guard (issue #2558). The Reviewer is trusted to issue a
# single, substantive `gh pr review` per session. When the model misbehaves
# — e.g. probing the CLI with `--body "test body"` before issuing the
# real review — the placeholder leaks out as a public review. This step
# scans wheels-bot reviews on the current SHA after the model exits and
# auto-dismisses any that look like probes (too short, or missing the
# canonical `wheels-bot:review-a` marker). Runs on `always()` so it still
# fires when the Claude step itself failed mid-session.
- name: Validate Reviewer output
if: always() && steps.gate.outputs.skip == 'false'
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
PR_NUMBER: ${{ steps.pr.outputs.pr_num }}
HEAD_SHA: ${{ steps.pr.outputs.sha }}
run: |
set -euo pipefail
# Allow-list APPROVED and CHANGES_REQUESTED only: those are the
# exclusive set of states that (a) GitHub's dismiss API accepts and
# (b) gate merging. Feeding the dismiss API any other state returns
# HTTP 422 — e.g. "Can not dismiss a commented pull request review"
# for COMMENTED, the same shape for PENDING — which would fail this
# step and red-X the check. Originally observed on PR #2795 commit
# 0db188a5 when a COMMENTED placeholder leaked through.
reviews=$(gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews" --paginate \
| jq -c --arg sha "$HEAD_SHA" \
'[.[] | select(.user.login == "wheels-bot[bot]") | select(.commit_id == $sha) | select(.state == "APPROVED" or .state == "CHANGES_REQUESTED")]')
count=$(echo "$reviews" | jq 'length')
if [[ "$count" == "0" ]]; then
echo "::notice::No active wheels-bot reviews on ${HEAD_SHA} to validate"
exit 0
fi
# Dismissal criteria: body must be (a) >= 200 chars AND (b) contain
# `wheels-bot:review-a` as a substring. Both conditions are required —
# a long body without the marker still gets dismissed, because a
# marker-less review breaks downstream idempotency (the skip-check
# action greps for the marker on subsequent runs) and is itself a
# signal that the prompt was not followed. Belt-and-suspenders: the
# guard treats prompt compliance as load-bearing, not advisory.
dismissed=0
while IFS= read -r row; do
id=$(echo "$row" | jq -r '.id')
body=$(echo "$row" | jq -r '.body')
body_len=${#body}
if [[ "$body_len" -lt 200 ]] || ! grep -q 'wheels-bot:review-a' <<<"$body"; then
echo "::warning::Dismissing bogus Reviewer review id=${id} len=${body_len}"
gh api -X PUT \
"repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews/${id}/dismissals" \
-f message="Auto-dismissed by the Reviewer guard: body is shorter than 200 characters or missing the canonical \`wheels-bot:review-a\` marker. See wheels-dev/wheels#2558 for context."
dismissed=$((dismissed + 1))
fi
done < <(echo "$reviews" | jq -c '.[]')
if [[ "$dismissed" -gt 0 ]]; then
# Idempotency check: a manual workflow re-trigger combined with
# new bogus reviews on the same SHA could otherwise produce
# duplicate guard comments. Skip if a guard comment for this
# exact PR + SHA already exists.
guard_marker="wheels-bot:review-a-guard:${PR_NUMBER}:${HEAD_SHA}"
existing=$(gh api "repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments" --paginate \
| jq -r --arg m "$guard_marker" '[.[] | select(.body | contains($m))] | length')
if [[ "$existing" == "0" ]]; then
short_sha=${HEAD_SHA:0:7}
gh pr comment "$PR_NUMBER" --body "## Wheels Bot — Reviewer guard
Detected and dismissed ${dismissed} bogus Reviewer review(s) on commit \`${short_sha}\`. Cause: review body shorter than 200 characters or missing the canonical \`wheels-bot:review-a\` marker. See [wheels-dev/wheels#2558](https://github.com/wheels-dev/wheels/issues/2558) for context.
<!-- ${guard_marker} -->"
else
echo "::notice::Guard comment already present for ${PR_NUMBER}@${HEAD_SHA}; skipping duplicate"
fi
fi