Skip to content

Commit ef5fd1c

Browse files
davegaeddertdavegaeddert
committed
Report the Codex outcome as a review PullApprove can read
Codex cannot be named as an agent directly, whatever the handoff does. It emits no check run at all, and signals a clean result with a reaction rather than a review, so there is nothing on the pull request to read. Declaring it would mean every clean run -- and every run that never happens -- holds human review requests forever. So this watches Codex's native signals and submits a review of its own: findings -> Codex submitted a review after the trigger clean -> a thumbs-up on the trigger comment, and no review never ran -> a usage-limits comment no response -> a 15 minute bound The point is that it reports on every path, including the ones where Codex says nothing. Silence is the common case here, not the edge case: roughly a third of Codex runs never execute at all, and the handoff is already conditional on Claude having approved. An agent that sometimes stays quiet is worse than no agent, because the wait has no bound. Timestamps are recorded before the trigger so a review or quota notice left on an earlier commit is not read as this run's outcome. Not yet declared in CODEREVIEW.toml -- the account to add there is github-actions[bot], once this has been seen posting on a real pull request.
1 parent 2bdafc0 commit ef5fd1c

1 file changed

Lines changed: 82 additions & 1 deletion

File tree

‎.github/workflows/claude-code-review.yml‎

Lines changed: 82 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,21 +79,102 @@ jobs:
7979
# workspace of the commenting account. A github-actions[bot] comment is
8080
# silently dropped.
8181
- name: Trigger Codex review
82+
id: codex
8283
if: success() && steps.pr.outputs.fork == 'false'
8384
env:
8485
GH_TOKEN: ${{ secrets.CODEX_TRIGGER_TOKEN }}
8586
PR: ${{ github.event.pull_request.number || github.event.issue.number || inputs.pr_number }}
8687
REPO: ${{ github.repository }}
8788
run: |
8889
if [ -z "$GH_TOKEN" ]; then
90+
echo "skipped=CODEX_TRIGGER_TOKEN is not set" >> "$GITHUB_OUTPUT"
8991
echo "CODEX_TRIGGER_TOKEN is not set — skipping Codex handoff."
9092
exit 0
9193
fi
9294
# Latest review from a bot account is Claude's, since it just ran.
9395
state=$(gh api --paginate --slurp "repos/$REPO/pulls/$PR/reviews" \
9496
| jq -r 'add | map(select(.user.type == "Bot")) | last | .state // ""')
9597
if [ "$state" != "APPROVED" ]; then
98+
echo "skipped=Claude did not approve (review state: ${state:-none})" >> "$GITHUB_OUTPUT"
9699
echo "Claude review state: ${state:-<none>} — not handing off to Codex."
97100
exit 0
98101
fi
99-
gh pr comment "$PR" -R "$REPO" --body "@codex review"
102+
# Recorded before the trigger so the watcher below only counts what
103+
# Codex does in RESPONSE to it -- a review or a quota notice left on
104+
# an earlier commit must not be read as this run's outcome.
105+
echo "since=$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "$GITHUB_OUTPUT"
106+
comment_id=$(gh api "repos/$REPO/issues/$PR/comments" \
107+
-f body="@codex review" --jq .id)
108+
echo "comment_id=$comment_id" >> "$GITHUB_OUTPUT"
109+
echo "Triggered Codex (comment $comment_id)"
110+
111+
# ADAPTER: turn whatever Codex did into a submitted review.
112+
#
113+
# Codex cannot be named as a PullApprove agent directly. It emits no check
114+
# run at all, and signals a clean result with a 👍 reaction rather than a
115+
# review, so there is nothing for PullApprove to read. This step watches
116+
# those native signals and submits a review under this workflow's own
117+
# identity -- so the account to declare in CODEREVIEW.toml is
118+
# github-actions[bot], not Codex's.
119+
#
120+
# It reports on EVERY path, including "Codex never ran". That is the whole
121+
# point: a declared agent that sometimes stays silent holds human review
122+
# requests forever, which is worse than not declaring it. Roughly a third
123+
# of Codex runs never execute at all (quota), and the handoff above is
124+
# itself conditional -- so silence is the common case, not the edge case.
125+
#
126+
# `!cancelled()` rather than `always()`: a run superseded by a new push
127+
# should leave the reporting to its replacement.
128+
- name: Report the Codex outcome
129+
if: ${{ !cancelled() && steps.pr.outputs.fork == 'false' }}
130+
env:
131+
GH_TOKEN: ${{ github.token }}
132+
PR: ${{ github.event.pull_request.number || github.event.issue.number || inputs.pr_number }}
133+
REPO: ${{ github.repository }}
134+
CODEX: "chatgpt-codex-connector[bot]"
135+
COMMENT_ID: ${{ steps.codex.outputs.comment_id }}
136+
SINCE: ${{ steps.codex.outputs.since }}
137+
SKIPPED: ${{ steps.codex.outputs.skipped }}
138+
TIMEOUT_SECONDS: "900"
139+
run: |
140+
report() {
141+
echo "Reporting: $1"
142+
gh pr review "$PR" -R "$REPO" --comment --body "Codex second opinion — $1"
143+
exit 0
144+
}
145+
146+
[ -n "$SKIPPED" ] && report "not run: $SKIPPED."
147+
[ -z "$COMMENT_ID" ] && report "not run: the review step did not finish."
148+
149+
deadline=$(( $(date +%s) + TIMEOUT_SECONDS ))
150+
while [ "$(date +%s)" -lt "$deadline" ]; do
151+
# Findings: Codex submits a review (always COMMENTED, never a
152+
# verdict), with its findings as inline comments.
153+
if gh api --paginate --slurp "repos/$REPO/pulls/$PR/reviews" \
154+
| jq -e --arg u "$CODEX" --arg t "$SINCE" \
155+
'add | map(select(.user.login == $u and .submitted_at > $t)) | length > 0' \
156+
>/dev/null 2>&1; then
157+
report "reviewed it and left findings."
158+
fi
159+
160+
# Clean: a 👍 on the trigger comment, and no review at all. This is
161+
# the outcome that makes Codex undeclarable on its own.
162+
if gh api "repos/$REPO/issues/comments/$COMMENT_ID/reactions" \
163+
| jq -e 'map(select(.content == "+1")) | length > 0' >/dev/null 2>&1; then
164+
report "reviewed it and found nothing."
165+
fi
166+
167+
# Never ran: it says so in a PR comment. Same channel as the clean
168+
# notice, so the reaction above has to be checked first.
169+
if gh api --paginate --slurp "repos/$REPO/issues/$PR/comments" \
170+
| jq -e --arg u "$CODEX" --arg t "$SINCE" \
171+
'add | map(select(.user.login == $u and .created_at > $t
172+
and (.body | test("usage limits"; "i")))) | length > 0' \
173+
>/dev/null 2>&1; then
174+
report "could not run (Codex usage limits reached)."
175+
fi
176+
177+
sleep 30
178+
done
179+
180+
report "did not respond within $((TIMEOUT_SECONDS / 60)) minutes."

0 commit comments

Comments
 (0)