Problem
The cross-version multiplier replays existing P2P tests with one node running an
older release. Some tests assert behavior that only exists in newer versions, so
they cannot pass against an older node and must be skipped.
Right now the only way to skip them is MultiplierExcludes, which names the
multiplier:
MultiplierExcludes: []string{multiplier.CrossVersionOldSource},
That works while there is one version pair, but it says the wrong thing. The test
is not incompatible with the multiplier, it is incompatible with the older
version. As the LTS window grows to the last three minors, every version pair
becomes another multiplier name, and each excluded test has to list all of them.
Miss one and the test runs against a version that cannot support it.
Proposal
Let the test state the version it needs, once:
SupportedFromVersion: "v1.1.0",
The multiplier implements ShouldSkip (already part of testo's
ActionAwareSkipper) and compares that value against the version it is targeting.
Adding a new version pair then needs no change to any excluded test.
Notes
- Deliberately not done as part of the first multiplier. v1.0.0 is currently the
only post-1.0 release, so a version comparison would have exactly one value to
compare against, and MultiplierExcludes is the pattern the codebase already
uses (signed_docs).
- Worth doing when the second supported version pair appears, whichever comes
first with the LTS window widening.
- Needs a small version comparison helper. Release tags are semver, so this
should stay simple.
MultiplierExcludes should keep working. This is for the version-specific
case, not a replacement.
Part of #5066.
Problem
The cross-version multiplier replays existing P2P tests with one node running an
older release. Some tests assert behavior that only exists in newer versions, so
they cannot pass against an older node and must be skipped.
Right now the only way to skip them is
MultiplierExcludes, which names themultiplier:
That works while there is one version pair, but it says the wrong thing. The test
is not incompatible with the multiplier, it is incompatible with the older
version. As the LTS window grows to the last three minors, every version pair
becomes another multiplier name, and each excluded test has to list all of them.
Miss one and the test runs against a version that cannot support it.
Proposal
Let the test state the version it needs, once:
The multiplier implements
ShouldSkip(already part of testo'sActionAwareSkipper) and compares that value against the version it is targeting.Adding a new version pair then needs no change to any excluded test.
Notes
only post-1.0 release, so a version comparison would have exactly one value to
compare against, and
MultiplierExcludesis the pattern the codebase alreadyuses (
signed_docs).first with the LTS window widening.
should stay simple.
MultiplierExcludesshould keep working. This is for the version-specificcase, not a replacement.
Part of #5066.