Conversation
FALSIFY-CGP-061 FAILED: doctor took 2665ms Measured during a full `--lib --tests` sweep of the workspace — the box was running 98,621 tests in parallel. 2665 against a 2500 ms limit is a 6.6% miss that says nothing about `cgp doctor` and everything about the load. It is the LAST of the 37 failures that sweep found, and the one blocking #3249's hard-failing nightly. Raising the limit is the "clever ratio rewrite" that once blocked all nine open PRs. The fix is to stop asserting duration where duration cannot be measured. FIXED ONE, FOUND SIX. The test that failed was the entry point, not the scope: tests/falsify.rs:457 doctor < 2500 ms <- the one that failed tests/falsify.rs:583 profile < 31 s <- same file, simply not unlucky yet tests/integration.rs:39 doctor < 30 s src/doctor.rs:406 checks < 2 s } src/analysis/diff.rs:344 < 100 ms } IN src/ -- these are #[cfg(test)] src/metrics/export.rs:70 < 100 ms } units, so they run in --workspace } --lib: the REQUIRED workspace-test, } on every PR and every merge group A 100 ms assertion on a shared runner is a coin flip, and two of them have been sitting in a required check. `scripts/check_no_timing_in_required.sh` exists for exactly this — its header says "policy is what failed the other eleven times" — but its registry covers `check_*.sh` GUARDS. It has never been able to see a Rust target, so nobody has ever counted these. `timing-gate`, off by default, is that registry for Rust: nothing enables it, so a timing row cannot reach a required check by accident. Arm it where duration is measurable — a dedicated unloaded runner at release time — not in CI's default set and not in #3249's nightly full surface. GATED, NOT DELETED, proved by mutation rather than asserted: limit -> 0, WITH --features timing-gate FAILED (1 failed) the assertion is live limit -> 0, WITHOUT the feature ok compiled out restored ok Also in CGP-061, a second defect in the same row: assert!(stdout.contains("RTX 4090") || stdout.contains("GPU")); That asserts the RUNNER's hardware — the class fixed in #3210 across four sibling rows — and passes here only because this box happens to have a 4090. What CGP-061 owes is that `doctor` reached a VERDICT about the device and printed it, so that is what it now asserts: non-empty output, and a decision among GPU/CUDA/device/no GPU/not available. aprender-cgp: 121 lib + 3 falsify pass, clippy -D warnings clean. Not fixed here, filed separately: the same regex over the workspace finds ~70 duration assertions in `crates/*/src/` and 36 more in `crates/*/tests/`, across 12 crates. This commit fixes the one crate that actually failed. Refs #3239 Pmat-Ticket: PMAT-1098 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
noahgift
enabled auto-merge
September 14, 2026 08:20
|
§13.11 rung 1 — quorum shadow verdict Shadow mode: this records a verdict and merges nothing. A refusal |
Contributor
Author
|
Superseded by #3295, which carries this change unmodified along with eight other Not abandoned and not rejected: the commits are merged into Why: a successful |
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Measured during a full
--lib --testssweep — the box was running 98,621 tests in parallel.2665 against a 2500 ms limit is a 6.6% miss that says nothing about
cgp doctorand everythingabout the load. It is the last of the 37 failures that sweep found, and the one blocking
#3249's hard-failing nightly.
Raising the limit is the "clever ratio rewrite" that once blocked all nine open PRs. The fix is to
stop asserting duration where duration cannot be measured.
Fixed one, found six
The test that failed was the entry point, not the scope:
tests/falsify.rs:457tests/falsify.rs:583tests/integration.rs:39src/doctor.rs:406src/src/analysis/diff.rs:344src/src/metrics/export.rs:70src/Those last three are
#[cfg(test)]units, so they run in--workspace --lib— the requiredworkspace-test, on every PR and every merge group. A 100 ms assertion on a shared runner is acoin flip, and two of them have been sitting in a required check.
scripts/check_no_timing_in_required.shexists for exactly this; its header says "policy is whatfailed the other eleven times." But its registry covers
check_*.shguards. It has never beenable to see a Rust target, so nobody has ever counted these.
timing-gateOff by default, enabled by nothing — the same registry idea, for Rust. Arm it where duration is
measurable: a dedicated unloaded runner at release time. Not in CI's default set, and not in
#3249's nightly full surface.
Gated, not deleted — proved by mutation rather than asserted:
A second defect in the same row
That asserts the runner's hardware — the class fixed in #3210 across four sibling rows — and
passes here only because this box happens to have a 4090. What CGP-061 owes is that
doctorreached a verdict about the device and printed it, so that is what it now asserts: non-empty
output, and a decision among GPU / CUDA / device / no GPU / not available.
Verified:
aprender-cgp121 lib + 3 falsify pass,clippy -D warningsclean.Not fixed here
The same regex over the workspace finds ~70 duration assertions in
crates/*/src/and 36 morein
crates/*/tests/, across 12 crates —aprender-serve9,aprender-core5,aprender-test-lib3, and so on. This PR fixes the one crate that actually failed; the sweep is its own ticket.
Refs #3239
no-close: #3239 stays OPEN — this clears the last of its 37 measured failures, but the tiering
decision it describes is #3249's.
🤖 Generated with Claude Code