Description
To do:
- issue Doctest merging doesn't work at all on stable #137898
- original PR doctests: fix merging on stable #137899
- beta backport
- stable backport
What happened
Due to a bug in the implementation, merged doctests did not ship on stable. This is caused by the merged “runner” executable using libtest, so the fallback-to-unmerged behavior kicked in and the promised performance improvement didn’t happen.
Proposed remedy
To backport the fix to a point release, before any code accidentally relies on this behavior.
Possible breakage
Merged doctests work by string concatenating doctests into a wrapper binary that looks roughly like this (except that it uses libtest to report progress, and invokes a separate process for every test case).
mod __test_1 { pub fn main() { … } }
mod __test_2 { pub fn main() { … } }
fn main() { __test_1::main(); __test_2::main(); }
If the merged version fails to compile, we silently and automatically fall back to the old way of running them separately.
However, it is possible to write a doctest that successfully compiles as merged, but fails when run. For example, you can examine the stack trace using std::panic::Location. No automatic fallback happens if a doctest fails at runtime.
When we first developed this feature, we tested it on crater and saw that runtime failures introduced by this change were rare #130285 (the one pattern of failure that we found was caused by tests reading their command-line args, and we fixed that). This is why we consider the potential breakage an acceptable risk.