Revise design of untestable assertions#1440
Merged
Merged
Conversation
jugglinmike
changed the title
> Instead of implementing requirement 4 as written in #1352, I propose > that we keep the calculation of percentage passing assertions > unchanged from its prior formula: > > > (count of passing p1 assertions + count of passing p2 assertions) / (count of p1 assertions + count of p2 assertions)
> when untestable assertions are present, after the Summary of Negative > Side Effects and before the results for the first test, add a section > with: > > 1. A heading with content: "Summary of Untestable Assertions (N must, > N should, N may)" > 2. A table with the same columns and type of content as the summary of > failing assertions: > 1. Test Name > 2. Command > 3. Assertion Priority > 4. Assertion > 5. AT_NAME Response > 3. The table should have an accessible name provided by the heading.
jugglinmike
force-pushed
the
gh-1429-revise-untestable
branch
from
June 26, 2025 03:15
5edd041 to
2aa994b
Compare
jugglinmike
force-pushed
the
gh-1429-revise-untestable
branch
from
June 26, 2025 03:27
2aa994b to
cae9ff7
Compare
jugglinmike
marked this pull request as ready for review
June 26, 2025 03:40
Contributor
Author
|
@stalgiag & @ChrisC: this is ready for review! I've rebased this branch over the latest commit in This changeset includes a revision to an assumption we made in an earlier patch. It would be cleaner to address that revision in a separate pull request, but I've taken it on here so as not to further delay this work. (As a middle ground, I've structured those changes into a dedicated commit.) |
stalgiag
approved these changes
Jun 26, 2025
| ...processPriorityAssertionResults('MUST'), | ||
| ...processPriorityAssertionResults('SHOULD') | ||
| ]; | ||
| const assertionResults = scenarioResult.untestable |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves gh-1429.
I'll leave this in "draft" until we receive confirmation about the assumption I've made in this implementation.