What happened
PR #531 was a Renovate bot dependency update PR in konflux-ci/build-service. Over 30 days (Jun 3 – Jul 3, 2026), the review agent ran 101 times on this single PR, producing 97 status comments and 23 inline review findings. 83 of the 101 reviews were dismissed (superseded by the next Renovate rebase ~4 hours later). Zero humans engaged with any review output. The PR was closed without merge on Jul 3, 2026. Conservative estimate: 100+ review agent invocations × typical token cost = significant waste on a PR no one was reading.
What could go better
The system has no mechanism to detect that reviews are being ignored and stop investing in them. Existing issues address adjacent problems: #1355 proposes skipping re-reviews when the rebase has no semantic change (input-side filter), #1371 proposes skipping bot PRs entirely (too broad — the review agent did find real issues here), and #2842 proposes using a cheaper model (reduces cost per run but not run count). None of these propose a feedback-loop circuit breaker that says: 'after N reviews with zero human reactions (no replies, no emoji, no review dismissal by a human, no label change), stop reviewing this PR until a human signals interest.' This is a high-confidence proposal — the data clearly shows 100+ wasted runs. The only uncertainty is the right threshold (5? 10?) and the right re-engagement signal.
Proposed change
Add a pre-dispatch check in the review workflow (likely in reusable-review.yml or the dispatch logic in fullsend.yaml) that counts the number of prior review-agent reviews on the PR with no intervening human activity. If the count exceeds a configurable threshold (suggested default: 5), skip the review dispatch and log a message explaining why. 'Human activity' should be defined as: a non-bot comment, a non-bot review, a label change by a non-bot user, or a manual /fs-review command. The /fs-review command should always bypass the circuit breaker to allow on-demand reviews. The threshold should be configurable per-repo via .fullsend config or a repo-level setting.
Validation criteria
On the next long-lived Renovate PR with repeated rebases and no human engagement, the review agent should stop after the configured threshold (e.g., 5 runs) rather than continuing indefinitely. Verify by checking that (1) no more than threshold+1 review runs are dispatched for any single PR without intervening human activity, and (2) a /fs-review command successfully triggers a review even after the circuit breaker has engaged. Measure across 3+ bot-authored PRs over a 30-day window.
Generated by retro agent from konflux-ci/build-service#531
What happened
PR #531 was a Renovate bot dependency update PR in konflux-ci/build-service. Over 30 days (Jun 3 – Jul 3, 2026), the review agent ran 101 times on this single PR, producing 97 status comments and 23 inline review findings. 83 of the 101 reviews were dismissed (superseded by the next Renovate rebase ~4 hours later). Zero humans engaged with any review output. The PR was closed without merge on Jul 3, 2026. Conservative estimate: 100+ review agent invocations × typical token cost = significant waste on a PR no one was reading.
What could go better
The system has no mechanism to detect that reviews are being ignored and stop investing in them. Existing issues address adjacent problems: #1355 proposes skipping re-reviews when the rebase has no semantic change (input-side filter), #1371 proposes skipping bot PRs entirely (too broad — the review agent did find real issues here), and #2842 proposes using a cheaper model (reduces cost per run but not run count). None of these propose a feedback-loop circuit breaker that says: 'after N reviews with zero human reactions (no replies, no emoji, no review dismissal by a human, no label change), stop reviewing this PR until a human signals interest.' This is a high-confidence proposal — the data clearly shows 100+ wasted runs. The only uncertainty is the right threshold (5? 10?) and the right re-engagement signal.
Proposed change
Add a pre-dispatch check in the review workflow (likely in
reusable-review.ymlor the dispatch logic infullsend.yaml) that counts the number of prior review-agent reviews on the PR with no intervening human activity. If the count exceeds a configurable threshold (suggested default: 5), skip the review dispatch and log a message explaining why. 'Human activity' should be defined as: a non-bot comment, a non-bot review, a label change by a non-bot user, or a manual/fs-reviewcommand. The/fs-reviewcommand should always bypass the circuit breaker to allow on-demand reviews. The threshold should be configurable per-repo via.fullsendconfig or a repo-level setting.Validation criteria
On the next long-lived Renovate PR with repeated rebases and no human engagement, the review agent should stop after the configured threshold (e.g., 5 runs) rather than continuing indefinitely. Verify by checking that (1) no more than threshold+1 review runs are dispatched for any single PR without intervening human activity, and (2) a
/fs-reviewcommand successfully triggers a review even after the circuit breaker has engaged. Measure across 3+ bot-authored PRs over a 30-day window.Generated by retro agent from konflux-ci/build-service#531