Skip to content

feat(trials): freeze failure taxonomy, regrade path, and exit codes - #457

Merged
solomonneas merged 3 commits into
mainfrom
feat/trial-failure-taxonomy
Jul 23, 2026
Merged

feat(trials): freeze failure taxonomy, regrade path, and exit codes#457
solomonneas merged 3 commits into
mainfrom
feat/trial-failure-taxonomy

Conversation

@solomonneas

Copy link
Copy Markdown
Collaborator

Summary

  • Freeze the model-trial outcome contract in docs/model-trial-taxonomy.md (Settle the trial failure taxonomy, retry policy, and regrade path before the stable cut #435).
  • Add failure_reason on cells (timeout, transport_drop, provider_5xx), drop exit_code from grader_result.v1, and split measurement_failures / partial_scores in summary.json.
  • Implement three-valued process exit (0/1/3), brigade model trial regrade, resume regrade for grader_error, and project_cell export privacy.

Closes #435.

Test plan

  • brigade work verify run --target . --command "pytest -q tests/ -k 'trial or model_trials or grader or eval'" --capture brigade-work
  • ./scripts/verify (3944 passed, 82.49% coverage)
  • New tests: regrade without seat rerun, timeout failure_reason, process_exit dominance, project_cell privacy, partial score split

Notes

Made with Cursor

@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • coderabbit-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: escoffier-labs/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: a28f8ee4-a2f4-48fb-8881-dba9c34eecfa

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/trial-failure-taxonomy

Comment @coderabbitai help to get the list of available commands.

@solomonneas solomonneas self-assigned this Jul 23, 2026
solomonneas and others added 3 commits July 23, 2026 10:03
Settle #435 with measurement_failures in summary, failure_reason on cells,
three-valued process exit, grader_result exit_code removal, regrade command,
and export privacy projection.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@solomonneas
solomonneas force-pushed the feat/trial-failure-taxonomy branch from 06c8eed to 194a13f Compare July 23, 2026 14:10
@solomonneas
solomonneas merged commit 9b87275 into main Jul 23, 2026
28 checks passed
@solomonneas
solomonneas deleted the feat/trial-failure-taxonomy branch July 23, 2026 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Settle the trial failure taxonomy, retry policy, and regrade path before the stable cut

1 participant