Represent non-encodable test argument values in Test.Case.ID #1000
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This expands
Test.Case.ID
to represent combinations of arguments which aren't fully encodable, and uniquely distinguishes test cases associated with the same parameterized test function which have otherwise identical IDs.Motivation:
It is possible to declare a parameterized test function with a collection of arguments that includes the same element more than once. As a trivial example:
This can happen in more realistic scenarios if you dynamically construct a collection of arguments that accidentally or intentionally includes the same value more than once.
The testing library attempts to form a unique identifier for each argument passed to a parameterized test. It does so by checking whether the value conforms to
Encodable
, or one of the other protocols mentioned in the documentation (see Run selected test cases).One problem with the current implementation is that if the value doesn't conform to one of those known protocols, the testing library gives up and doesn't produce a unique identifier for that test case at all. Specifically, in that situation the
argumentIDs
property ofTest.Case.ID
will have a value ofnil
.Another problem is that if an argument is passed more than once, the derived identifiers for each argument's test case will be the same and the result reporting will be ambiguous at best; or worse, the lack of a unique identifier could cause an integrated tool to misbehave or crash. (Current versions of Xcode 16 experience this issue non-deterministically for projects which have test parallelization enabled.) To solve this, there needs to be a way to deterministically distinguish test cases which, from the testing library's perspective, appear identical—either because they actually are the same value, or because they encode to the same representation.
Modifications:
isStable
boolean property toTest.Case.Argument.ID
representing whether or not the testing library was able to encode a stable representation of the argument value it identifies.discriminator
toTest.Case
which distinguishes test cases associated with the same test function whose arguments are identical.Test.Case.ID
with the same name,discriminator
.Test.Case.ID.argumentIDs
property to non-Optional, so that it always has a value even if one or more argument IDs is non-stable.isStable
toTest.Case.ID
whose value istrue
iff all of its argument IDs are stable.Hashable
conformance toTest.Case
, since the reason for avoiding such conformance has been resolved.Checklist:
Resolves #995
Resolves rdar://119522099