What happened
PR #1481 was a Renovate/MintMaker bot PR created on 2026-02-26 that bumped jsonschema/v5 to v6 in go.mod only — without updating import paths, go.sum, or vendor/. CI failed immediately (Check sources, Go linters, Konflux build all failed). The PR sat open for ~4 months with no human engagement beyond a single /retest on 2026-03-15. During this time, the review agent ran 6 times and the retro agent ran 3 times, all producing identical findings. The PR was eventually autoclosed on 2026-07-01.
What could go better
Existing issue #1980 proposes a CLOSE verdict the review agent can emit during review, which would help at review time. However, this PR illustrates a different failure mode: the bot kept rebasing the branch (triggering new reviews), and no mechanism existed to detect that the PR had been in a failing state with zero human engagement for months. A time-based or event-count-based staleness check — separate from the review agent's per-review verdict — would catch PRs like this earlier and prevent the compounding waste of repeated reviews and retros. Confidence: medium. This is a relatively rare scenario (bot PRs with major version bumps that require code changes), but when it occurs the token waste multiplies over time. If #1980 ships and the review agent can close on first review, this becomes less necessary — but #1980 has been open since before this PR was created, suggesting it may not ship soon.
Proposed change
Add a scheduled or event-driven staleness detector in the fullsend dispatch layer (e.g., in the review or retro workflow) that checks: (1) Is this a bot-authored PR? (2) Has CI been failing continuously for >N days (e.g., 14 days)? (3) Has there been zero human review or comment activity in that window? If all conditions are met, either auto-close the PR with a comment explaining why, or post a comment tagging maintainers to decide. This is complementary to #1980 (which handles closure at review time) — this handles the case where reviews keep happening but no one acts on them. Implementation could be a periodic GitHub Action that queries open bot PRs, or a pre-check in the review dispatch that skips review if staleness criteria are met.
Validation criteria
Bot PRs with consistently failing CI and no human engagement for >14 days should either be auto-closed or flagged within 1 day of meeting the staleness threshold. On the next occurrence of a PR like #1481, the system should not accumulate more than 2 review agent runs before the staleness mechanism intervenes. Measure by tracking the number of review dispatches on bot PRs that are eventually autoclosed without merge — target: ≤2 reviews per such PR.
Generated by retro agent from konflux-ci/integration-service#1481
What happened
PR #1481 was a Renovate/MintMaker bot PR created on 2026-02-26 that bumped
jsonschema/v5tov6ingo.modonly — without updating import paths,go.sum, orvendor/. CI failed immediately (Check sources, Go linters, Konflux build all failed). The PR sat open for ~4 months with no human engagement beyond a single/reteston 2026-03-15. During this time, the review agent ran 6 times and the retro agent ran 3 times, all producing identical findings. The PR was eventually autoclosed on 2026-07-01.What could go better
Existing issue #1980 proposes a CLOSE verdict the review agent can emit during review, which would help at review time. However, this PR illustrates a different failure mode: the bot kept rebasing the branch (triggering new reviews), and no mechanism existed to detect that the PR had been in a failing state with zero human engagement for months. A time-based or event-count-based staleness check — separate from the review agent's per-review verdict — would catch PRs like this earlier and prevent the compounding waste of repeated reviews and retros. Confidence: medium. This is a relatively rare scenario (bot PRs with major version bumps that require code changes), but when it occurs the token waste multiplies over time. If #1980 ships and the review agent can close on first review, this becomes less necessary — but #1980 has been open since before this PR was created, suggesting it may not ship soon.
Proposed change
Add a scheduled or event-driven staleness detector in the fullsend dispatch layer (e.g., in the review or retro workflow) that checks: (1) Is this a bot-authored PR? (2) Has CI been failing continuously for >N days (e.g., 14 days)? (3) Has there been zero human review or comment activity in that window? If all conditions are met, either auto-close the PR with a comment explaining why, or post a comment tagging maintainers to decide. This is complementary to #1980 (which handles closure at review time) — this handles the case where reviews keep happening but no one acts on them. Implementation could be a periodic GitHub Action that queries open bot PRs, or a pre-check in the review dispatch that skips review if staleness criteria are met.
Validation criteria
Bot PRs with consistently failing CI and no human engagement for >14 days should either be auto-closed or flagged within 1 day of meeting the staleness threshold. On the next occurrence of a PR like #1481, the system should not accumulate more than 2 review agent runs before the staleness mechanism intervenes. Measure by tracking the number of review dispatches on bot PRs that are eventually autoclosed without merge — target: ≤2 reviews per such PR.
Generated by retro agent from konflux-ci/integration-service#1481