Skip to content

bisect run's good/bad judgement is a shell script, so all four run tests fail on Windows #100

Description

@JerryHyun

All four bisect_run_* tests fail on Windows: the good/bad judgement they hand to git bisect run is a shell script, and Windows has no shell to run it.

Visible only now that the Rust suite can start on Windows — see #98.

Where

tests/bisect.rs:

  • bisect_run_converges_via_scripted_good_bad_command (:180)
  • bisect_run_handles_a_skip_exit_code_and_still_converges (:228)
  • bisect_run_cancel_stops_the_loop_before_convergence (:346)
  • bisect_run_start_refuses_a_second_concurrent_call_while_one_is_in_flight

The first two fail by converging on the wrong commit, which is what a judgement command that cannot run looks like from the outside:

assertion `left == right` failed: FIRST-BAD MISMATCH: automated run reported Some("56e1d0c"), expected K 881f5a0
  left:  "56e1d0c"
  right: "881f5a0"

The other two fail on timing instead — the run never gets going, so:

the test command should have started running before the timeout

One of them is timing-sensitive enough to flip between runs, so the suite's total failure count reads 19 on some runs and 20 on others.

Fix

The tests need a judgement command that exists on the platform they run on. Options, roughly in order of how much they change:

  1. Write the judge as a tiny Rust helper binary the test builds and points bisect run at. Platform-independent, no shell involved, and the exit codes (0 good / 1 bad / 125 skip) are the whole contract — but it adds a build artifact.
  2. Keep a script, and pick the interpreter per platform: sh -c on unix, cmd /c on Windows. Smaller change, but now there are two scripts to keep in agreement, and the skip-exit-code case has to be expressed twice.
  3. #[cfg(unix)] the four tests. Honest, and cheap, but it means bisect run — the one bisect path with real control flow in it — has no Windows coverage at all.

(1) looks the most durable; (3) is a reasonable holding position if the others are not worth it yet.

Note

bisect's non-run tests all pass on Windows. This is specifically the automated-run path.


Found while making cargo test runnable on Windows (#98). Measured with cargo test -j 2 --no-fail-fast on Windows 11 / MSVC 14.50.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions