Skip to content

fix(outcome): forgive candidate regressions when clears outnumber them - #648

Merged
solomonneas merged 1 commit into
mainfrom
fix/629-regression-forgiveness
Aug 1, 2026
Merged

fix(outcome): forgive candidate regressions when clears outnumber them#648
solomonneas merged 1 commit into
mainfrom
fix/629-regression-forgiveness

Conversation

@solomonneas

Copy link
Copy Markdown
Collaborator

Summary

  • Decision for outcome: a regression permanently withholds an unchanged candidate cohort #629: define a bounded forgiveness rule for later verified clears in the same unchanged content-fingerprint cohort (do not withhold forever on any hurt).
  • Candidates withhold only while helped <= hurt; they install when helped >= install_min_helped and clears strictly outnumber regressions.
  • Promoted demotion on hurt remains a separate immediate policy. Append-only evidence is never rewritten. Scorecard effectiveness uses the same helper.

Closes #629.

Verify receipt

20260801-042517-work-verify-aa3a0c

brigade work verify run --target . --command "./scripts/verify" --capture brigade-work

Acceptance criteria → tests

Criterion Test
helped=10 hurt=1 installs (forgiveness) test_decide_installs_candidate_when_later_clears_outnumber_regression
Regressions not outnumbered still withhold test_decide_withholds_candidate_when_regressions_are_not_outnumbered
Content-fingerprint change scores a new cohort test_decide_scores_new_fingerprint_cohort_separately_from_regressed_cohort
Promoted demotion unchanged test_decide_rolls_back_a_promoted_artifact_on_regression

Test plan

  • tests/test_outcome.py + scorecard suites green
  • brigade work verify run --target . --command "./scripts/verify" --capture brigade-work
  • External grader review before merge

Made with Cursor

A single verified hurt permanently withheld an unchanged candidate cohort
even after later clears greatly outnumbered it. Withhold only while
helped <= hurt, and allow install once clears outnumber regressions and
meet install_min_helped. Promoted demotion stays a separate policy.

Co-authored-by: Cursor <cursoragent@cursor.com>
@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • coderabbit-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: escoffier-labs/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 451d2dc6-ee46-45a9-9c61-2303bfa7a644

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@solomonneas solomonneas self-assigned this Aug 1, 2026
@solomonneas
solomonneas merged commit 6e030e3 into main Aug 1, 2026
28 checks passed
@solomonneas
solomonneas deleted the fix/629-regression-forgiveness branch August 1, 2026 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

outcome: a regression permanently withholds an unchanged candidate cohort

1 participant