Skip to content

Re-post review findings when prior REQUEST_CHANGES was dismissed #2918

Description

@fullsend-ai-retro

What happened

On PR #2853, the review agent posted a REQUEST_CHANGES review at 16:17 UTC with 3 medium findings including [submodule-trust] — a legitimate security concern about submodules: true cloning a third-party repo in CI. The author pushed 3 follow-up commits (16:55–17:18), which caused the REQUEST_CHANGES to be dismissed. The final review run (run 28535929661) completed successfully at 17:46 UTC but posted only a status comment ('Finished Review · Success') with no review body or findings. When rh-hemartin approved on Jul 2, the only agent review visible was the dismissed one — effectively invisible in GitHub's review UI. The [submodule-trust] finding was never addressed.

What could go better

The final review run appears to have suppressed its findings, likely because they matched the previous run's findings (deduplication logic). However, since the previous review was DISMISSED, those findings were no longer visible to human reviewers. This created a gap: the review agent correctly identified a security concern, ran successfully on the final commit, but produced no visible output for the human to act on. Confidence: medium-high — I cannot confirm the suppression reason without reading the agent's JSONL trace, but the observable behavior (successful review with no posted body after a dismissed prior review) is clear from the PR timeline. Existing issues #2816 (persistent finding cache) and #2677 (surface unresolved count in status comment) are related but neither addresses this specific interaction: re-review should detect that its prior formal review was DISMISSED and re-post persistent findings rather than treating them as already-visible.

Proposed change

In the review agent's post-processing logic (the post-script or review skill that decides whether to post findings), add a check: before suppressing findings as duplicates of a prior run, verify the prior review's GitHub review state. If the prior review was DISMISSED or if no active (non-dismissed) review from the bot exists on the PR, post findings even if they match a previous run. This ensures findings remain visible when GitHub's review dismissal mechanism removes the original review. The check could use gh api repos/{owner}/{repo}/pulls/{pr}/reviews filtered to the bot user, looking for any review in CHANGES_REQUESTED state that is not DISMISSED. If none exist and the current run has findings, post them.

Validation criteria

On the next 5 PRs where: (a) the review agent posts REQUEST_CHANGES, (b) the author pushes new commits causing the review to be dismissed, and (c) the re-review finds persistent findings — the re-review should post a new review body with findings rather than only a status comment. Verify by checking that no PR reaches human approval with zero visible (non-dismissed) agent reviews when the agent has active findings.


Generated by retro agent from #2853

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/reviewReview agentpriority/highSignificant impact, address soonready-for-triageRetro-filed issue awaiting triage agentready-to-codeTriaged and ready for the code agentsecuritySecurity threat model and related concernstype/bugConfirmed defect in existing behavior

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions