-
-
Notifications
You must be signed in to change notification settings - Fork 335
Moving infrequently failing tests into the usual exclude files #6782
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
This excludes any tests that could interrupt a release, and also helps keep our test runs green at the expense of nonblocker test executions. Plus it reduces complexity by reducing the number of exclude files, which makes maintenance easier. Addendum: All "unreliable" tests excluded in this commit have been identified as not blocking a release (nonblocker) in at least one previous release cycle. Signed-off-by: Adam Farley <[email protected]>
|
I am looking at the issue links under which these exclusions are being done, and we should not be referring to release triage issues such as #6421 |
smlambert
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be referencing actual issues that articulate the intermittent failure (look in aqa-tests, infrastructure, and JBS existing issues that describe the problem, if an issue does not exist that describes the actual type of failure, then we should create one, some testcases may be failing in the same way, and can be grouped under the same issue)
Co-authored-by: Shelley Lambert <[email protected]>
Co-authored-by: Shelley Lambert <[email protected]>
Fair enough. Will screen them out. |
|
It seems many of the triage issues have expired job links, so I'm going through each of the failing unit tests and identifying the actual problem for each based on more recent test runs. I'll keep an eye out for failures sharing symptoms (with each other and existing issues). Plan: For each unreliable test:
Grinders: JDK11:
JDK17: |
This excludes any tests that could interrupt a release, and also helps keep our test runs green at the expense of nonblocker test executions. Plus it reduces complexity by reducing the number of exclude files, which makes maintenance easier.
Note 1: All "unreliable" tests excluded in this commit have been identified as not blocking a release (nonblocker) in at least one previous release cycle.
Note 2: The issues related to these infrequently failing tests will be updated to include a title change and an additional tag to indicate they they are infrequent failure.
@sophia-guo - Do you think this PR can go in in parallel to that work, or do you feel we should hold fire until the related issues are updated?
Resolves #6593