Skip to content

Commit 03800e9

Browse files
kwschulzclaude
andauthored
fix(tests): widen timing budget in stays_pending / never_quiet fixtures (#43)
Both ``tests/fixtures/test_browser.json`` cases set ``timeout_s: 0.05`` and assert ``evaluate.call_count >= 2``. The 50ms budget passes in CI's clean Linux container but flakes locally under CPU pressure: the loop's first iteration uses the whole budget on an interpreter under load, exits with call_count == 1, fails ``assert 1 >= 2``. Surfaced — and shipped past — during the v0.8.1 release.py run, which ran the full 1970-test suite. CI's per-job context made it invisible for prior releases; release.py's local full-suite run finally exposed it. Bumping to 0.5 gives 10× headroom, verified across three concurrent stress runs. Co-authored-by: Ken Schulz <kwschulz@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 12cadd3 commit 03800e9

2 files changed

Lines changed: 6 additions & 2 deletions

File tree

CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Fixed
11+
12+
- **Test-fixture timing flake (`stays_pending` / `never_quiet`)**`tests/fixtures/test_browser.json` had `timeout_s: 0.05` for the two timing-loop test cases that assert `evaluate.call_count >= 2`. The 50ms budget passed in CI's clean Linux container but flaked under local CPU pressure (e.g., when `release.py` runs the full 1970-test suite back-to-back), failing with `assert 1 >= 2` because the loop only iterated once. Surfaced — and shipped through — during the v0.8.1 release. Bumping to `timeout_s: 0.5` gives 10× headroom; verified across three concurrent stress runs.
13+
1014
## [0.8.1] - 2026-05-03
1115

1216
### Fixed

tests/fixtures/test_browser.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,13 +49,13 @@
4949
"wait_for_pending_data_cases": [
5050
{"id": "immediate_zero", "pending_values": [0], "timeout_s": 1.0, "expect_timeout": false, "description": "returns immediately when pending is 0"},
5151
{"id": "drains_to_zero", "pending_values": [3, 2, 1, 0], "timeout_s": 1.0, "expect_timeout": false, "description": "waits until pending drains to 0"},
52-
{"id": "stays_pending", "pending_values": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "timeout_s": 0.05, "expect_timeout": true, "description": "times out when requests never complete"},
52+
{"id": "stays_pending", "pending_values": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "timeout_s": 0.5, "expect_timeout": true, "description": "times out when requests never complete (timeout_s=0.5 not 0.05: tighter values flake under local CPU pressure when 1969 other tests compete; CI's clean container hid this for releases)"},
5353
{"id": "evaluate_exception", "pending_values": "exception", "timeout_s": 1.0, "expect_timeout": false, "description": "returns on page.evaluate exception"}
5454
],
5555
"wait_for_quiescence_cases": [
5656
{"id": "already_quiet", "pending_values": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "quiescence_s": 0.01, "timeout_s": 1.0, "expect_timeout": false, "description": "returns when already quiescent"},
5757
{"id": "activity_then_quiet", "pending_values": [2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "quiescence_s": 0.01, "timeout_s": 1.0, "expect_timeout": false, "description": "waits through activity then detects quiescence"},
58-
{"id": "never_quiet", "pending_values": [1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0], "quiescence_s": 5.0, "timeout_s": 0.05, "expect_timeout": true, "description": "times out when activity oscillates"},
58+
{"id": "never_quiet", "pending_values": [1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0], "quiescence_s": 5.0, "timeout_s": 0.5, "expect_timeout": true, "description": "times out when activity oscillates (timeout_s=0.5 not 0.05; same flake mode as stays_pending)"},
5959
{"id": "evaluate_exception", "pending_values": "exception", "quiescence_s": 0.01, "timeout_s": 1.0, "expect_timeout": false, "description": "returns on page.evaluate exception"}
6060
],
6161
"output_path_cases": [

0 commit comments

Comments
 (0)