trying to make the reviewer more deterministic

[derekmisler]

Summary

This PR tightens the PR reviewer agent's prompts to produce more consistent, deterministic outputs. It sets temperature: 0.0 on the Sonnet model, sharpens the drafter and verifier instructions to reduce ambiguity, and updates eval criteria to match the new output format (emoji-labeled assessment labels instead of REQUEST_CHANGES/APPROVE action names).

Changes

• review-pr/agents/pr-review.yaml: Sets temperature: 0.0 on the Sonnet model. Strengthens drafter instructions with a "When in Doubt" section, more precise severity definitions (security findings default to high), and clearer test-file ignore rules. Rewrites verifier instructions to default to LIKELY instead of filtering toward dismissal — DISMISS now requires citing specific mitigation code, with a higher bar for security findings.
• review-pr/agents/evals/security-redirect-uri-{1,2,3}.json and batched-verifier-security-{1,2,3}.json: Updated eval criteria from "The agent decided to REQUEST_CHANGES" to "The review assessment label is '🔴 CRITICAL'" to match the actual output format the agent now produces.
• review-pr/agents/evals/success-{1,2,3}.json: Updated eval criteria from "The agent decided to APPROVE" to checking for '🟢 APPROVE' or '🟡 NEEDS ATTENTION' labels, and replaced the vague "does not request changes" check with "does not label any finding as CRITICAL or block-worthy".

I wasn't able to get a perfect 100% on relevancy, but at least it errs on the over-communication, rather than dismissing an issue and stamping it with '🟢 APPROVE'

How to Test

• Run the eval suite against the updated pr-review.yaml and verify all 9 eval scenarios pass consistently across multiple runs.
• Confirm that security-related PRs (e.g., the redirect URI scenarios) reliably produce a 🔴 CRITICAL label without needing multiple retries.
• Confirm that clean PRs (success scenarios) reliably produce 🟢 APPROVE or 🟡 NEEDS ATTENTION and do not flip to critical on re-runs.

Closes: https://github.com/docker/gordon/issues/200

[derekmisler]

/describe

[docker-agent]

✅ PR description has been generated and updated!

[docker-agent]

Review Summary

Assessment: 🟢 APPROVE

This PR updates evaluation test configurations and agent documentation to align with the new comment-only review policy. The changes are configuration and documentation only, with no production code modifications.

Changes Overview

The PR makes the following changes:

• Evaluation files: Updates expected success criteria from checking specific GitHub review events (REQUEST_CHANGES, APPROVE) to checking assessment labels (🔴 CRITICAL, 🟢 APPROVE, 🟡 NEEDS ATTENTION)
• Agent configuration: Adds temperature: 0.0 to the sonnet model configuration for more deterministic behavior
• Documentation: Expands guidance for the drafter and verifier agents, including clearer severity definitions, verification standards, and handling of test-only code patterns

Findings

No bugs found in the changed code. All modifications are to test/evaluation configurations and documentation, which do not affect runtime behavior.

---

✅ This PR is ready to merge — no issues detected in the changed code.