Skip to content

Commit 9cf9799

Browse files
noahgiftclaude
andcommitted
fix(test): three book rows execute python3 and the clean-room image has none
workspace-test, run 34809941798: 39,613 passed, 2 failed -- aprender-verify-ml::book verification::oracles::test_io_oracle_example aprender-verify-ml::book verification::oracles::test_python_executor_example assertion failed: verdict.is_ok() Third instance tonight of one class: a test that asserts a RUNNER CAPABILITY, gated by nothing, compiled and run because the BSE-17 quick tier reached its crate. The browser falsifiers were the first two. MEASURED, not inferred, on both sides of the seam: docker run --rm localhost:5000/sovereign-ci:stable command -v python3 -> NO_PYTHON3 (and NO_PYTHON; rustc IS present) and the SAME built test binary, run twice: with python3 on PATH 3 passed, 1 ignored with an empty PATH 3 FAILED, 1 ignored So it is the interpreter, not the code, and not the platform. CI reported 2 failures, not 3: `test_verification_with_input_example` did not run in that selection. It fails identically without an interpreter -- the run above is how I know -- so the gate covers the module, not the two rows that happened to be reported. Fixing only what the log named is what left the second browser falsifier armed two commits ago. `python-oracle`, on `mod oracles;`, and nothing enables it. A capability needs a name no sibling reaches by accident; `browser-falsify` in aprender-test-lib is the same shape, and #3246's ratchet is the mechanical version of not letting one leak. NO SKIP. Where `python-oracle` is on, the rows still fail hard without python3 -- they are examples from the book and their whole point is that the executor really executes. Verified both directions: (default) --test book verification:: -> 0 tests, module compiled out --features python-oracle --test book ...::oracles -> 3 passed, 1 ignored Also dark, and not fixed here: `grep -c aprender-verify-ml .github/workflows/ci.yml` is 0, so this target has never run in CI. It surfaced only because this PR's selection reached the crate. And `test_timeout_handling_example` carries a bare `#[ignore]` ("Timeout handling currently hangs - needs executor fix") which the repo bans; it predates this change and needs its own ticket. The close-out is python3 in the clean-room image, then arm the feature. Measure first, assert second. Pmat-Ticket: PMAT-1098 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent e80e8f0 commit 9cf9799

2 files changed

Lines changed: 21 additions & 0 deletions

File tree

crates/aprender-verify-ml/Cargo.toml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,23 @@ tempfile = "3.14"
7373

7474
[features]
7575
default = []
76+
# A RUNNER CAPABILITY, NOT A LIBRARY FEATURE (#3205).
77+
#
78+
# tests/book/verification/oracles.rs runs `PythonExecutor`, which shells out to a
79+
# python3 interpreter. The clean-room CI image has none -- measured:
80+
# docker run --rm localhost:5000/sovereign-ci:stable command -v python3 -> NO_PYTHON3
81+
# (rustc IS present; only the interpreter is missing)
82+
# and the three rows fail there while passing wherever python3 exists. Proved by
83+
# running the SAME built binary both ways: with python3 3 pass, with an empty PATH
84+
# all 3 fail.
85+
#
86+
# The BSE-17 quick tier builds the selected crates' `--lib --tests` in one graph,
87+
# so any PR whose selection reaches this crate ran them on a box without an
88+
# interpreter. A capability needs a name nothing enables by accident; see
89+
# aprender-test-lib's `browser-falsify` for the same shape.
90+
#
91+
# Arm it once the image carries python3, not before.
92+
python-oracle = []
7693
parquet = ["dep:parquet", "dep:arrow"]
7794
tree-sitter = ["dep:tree-sitter", "dep:tree-sitter-python", "dep:tree-sitter-c", "dep:tree-sitter-typescript"]
7895
pest = ["dep:pest", "dep:pest_derive"]
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
11
//! Verification oracle chapter examples
22
3+
// `python-oracle`: these rows execute python3 and the clean-room image has none.
4+
// See the feature's comment in this crate's Cargo.toml. NOT a skip -- where the
5+
// feature is on, they still fail hard without an interpreter.
6+
#[cfg(feature = "python-oracle")]
37
mod oracles;

0 commit comments

Comments
 (0)