post-review 422 regression: diff-hunk validation does not cover all edge cases

[fullsend-ai-retro]

What happened

On June 22, 2026, two consecutive review runs for PR #625 failed with GitHub API 422 errors when submitting inline review comments. Run 27981425941 logged 2 inline comment(s) omitted (file not in PR diff) and 1 finding(s) posted as file-level comment(s) (line outside diff hunk) — indicating the existing filtering logic ran — but the API call still returned 422. Run 27982865970 hit the identical failure on retry. The post-review step treated the 422 as fatal, so no review was posted for either run, wasting two full agent invocations.

What could go better

Issue #1067 tracks this failure mode, and issue #1349 claims PR #1348 already fixed it by adding diff-hunk header validation in findingsToReviewComments (internal/cli/postreview.go). However, the June 22 failures prove the fix is incomplete — there are edge cases where inline comments still reference positions GitHub cannot resolve. Likely scenarios: (a) the hunk-header parser does not handle all diff formats (e.g., combined diffs, renamed files, binary files), (b) GitHub's position model differs from what the validator computes (GitHub uses a relative position within the hunk, not an absolute line number), or (c) multi-line comment ranges span hunk boundaries. Confidence: high that this is a regression or gap in PR #1348's fix, based on log evidence showing the filter ran but didn't prevent the 422.

Proposed change

In internal/cli/postreview.go (the findingsToReviewComments function and the review submission path), add a fallback: when CreatePullRequestReview returns 422, retry the submission without inline comments by embedding all findings as markdown in the review body. This ensures the review verdict and findings are always posted even if position validation fails. Additionally, add structured error logging that captures which specific inline comment triggered the 422 (GitHub's error response includes details about invalid parameters) to make future debugging easier. Finally, reopen or update issue #1067 with the June 22 failure evidence, since issue #1349's claim that it was fixed is contradicted by this recurrence.

Validation criteria

After the fix: (1) review runs that encounter 422 errors on inline comment submission should degrade gracefully by posting the review body with findings in markdown, rather than failing the entire run. (2) The next 10 review runs across the org that produce out-of-diff findings should all post reviews successfully (zero 422-induced run failures). (3) Structured logs should capture the specific comment that caused the 422 when it occurs.

---

Generated by retro agent from #625

[fullsend-ai-triage]

🤖 Finished Triage · ✅ Success · Started 3:40 PM UTC · Completed 3:42 PM UTC
Commit: fbaf56c · View workflow run →

[fullsend-ai-triage]

Triage Summary

Severity: High · Category: Bug (regression)

Problem

The diff-hunk validation added in PR #1348 does not cover all edge cases. On June 22, two consecutive review runs on PR #625 failed with GitHub API 422 errors despite the validation filter running and logging omitted comments. The post-review step treats 422 as fatal, so no review was posted — wasting two full agent invocations.

Root Cause

PR #1348 implemented only the validation half of the fix originally proposed in #1067. The validation catches some invalid positions but misses edge cases (likely: combined diffs, renamed files, relative-vs-absolute position model differences, or multi-line ranges spanning hunk boundaries). The fallback half — retrying without inline comments on 422 — was never implemented, so any validation gap remains fatal.

Relationship to Existing Issues

• post-review: handle GitHub API 422 on inline comment submission gracefully #1067 is the original tracking issue (still open) requesting both validation and fallback.
• Close issue #1067 as fixed by PR #1348 to prevent redundant code agent work #1349 noted that PR fix: validate inline review comment lines against diff hunks #1348 supposedly fixed post-review: handle GitHub API 422 on inline comment submission gracefully #1067, but the June 22 failures prove the fix is incomplete.
• This issue extends post-review: handle GitHub API 422 on inline comment submission gracefully #1067 by adding new failure evidence and specifically requesting the fallback path.

Recommended Fix

In internal/cli/postreview.go, add a 422 fallback in the review submission path: when CreatePullRequestReview returns 422 with inline comments attached, retry submission without inline comments, embedding all findings as markdown in the review body. Also add structured error logging capturing which specific comment triggered the 422.

Proposed Test Case

Test: submitFormalReview falls back to body-only review on 422

Setup:
- Mock CreatePullRequestReview to return 422 when called with inline comments
- Provide review findings with file/line positions

Expected behavior:
- First call with inline comments returns 422
- Function retries without inline comments
- All findings appear as markdown in the review body
- Warning log captures which comment caused the 422
- Function returns success

Edge cases:
- Non-422 errors should NOT trigger fallback
- Zero inline comments + 422 should not retry
- Retry preserves review verdict (approve/request-changes)
- All finding metadata (severity, file, line) in body markdown

This issue is well-specified and ready for implementation.

---

Labels: Bug regression in the review agent's post-review CLI with significant impact (wasted agent runs), matching agent/review, type/bug, and priority/high labels.