Skip to content

test(trials): harden failure taxonomy contract - #460

Merged
solomonneas merged 1 commit into
mainfrom
codex/verify-trial-taxonomy
Jul 23, 2026
Merged

test(trials): harden failure taxonomy contract#460
solomonneas merged 1 commit into
mainfrom
codex/verify-trial-taxonomy

Conversation

@solomonneas

Copy link
Copy Markdown
Collaborator

Summary

  • add a mixed-battery regression test that exercises timeout and deliberate nonzero failures through the real runner boundary
  • prove measurement failures force process exit 3 even when deterministic rejection is present
  • verify regrade rejects tampered stored output and strengthen export privacy assertions

Why

PR #457 froze the model-trial failure taxonomy. These tests cover the two risky paths that were previously asserted only with test-authored metadata or a synthetic summary.

Verification

  • pytest -q tests/test_model_trials.py (27 passed)
  • ./scripts/verify (4050 passed, 3 skipped, 82.54% coverage)

Co-authored-by: Codex <codex@openai.com>
@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • coderabbit-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: escoffier-labs/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 76e928fe-96a5-4a94-b580-0ff03f95182d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/verify-trial-taxonomy

Comment @coderabbitai help to get the list of available commands.

@solomonneas solomonneas self-assigned this Jul 23, 2026
@solomonneas
solomonneas merged commit 3475d8e into main Jul 23, 2026
28 checks passed
@solomonneas
solomonneas deleted the codex/verify-trial-taxonomy branch July 23, 2026 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant