Skip to content

Conversation

@adamfarley
Copy link
Contributor

@adamfarley adamfarley commented Dec 10, 2025

This excludes any tests that could interrupt a release, and also helps keep our test runs green at the expense of nonblocker test executions. Plus it reduces complexity by reducing the number of exclude files, which makes maintenance easier.

Note 1: All "unreliable" tests excluded in this commit have been identified as not blocking a release (nonblocker) in at least one previous release cycle.

Note 2: The issues related to these infrequently failing tests will be updated to include a title change and an additional tag to indicate they they are infrequent failure.

@sophia-guo - Do you think this PR can go in in parallel to that work, or do you feel we should hold fire until the related issues are updated?

Resolves #6593

This excludes any tests that could interrupt a release, and also
helps keep our test runs green at the expense of nonblocker
test executions. Plus it reduces complexity by reducing the number
of exclude files, which makes maintenance easier.

Addendum: All "unreliable" tests excluded in this commit have
been identified as not blocking a release (nonblocker) in at least
one previous release cycle.

Signed-off-by: Adam Farley <[email protected]>
@smlambert
Copy link
Contributor

I am looking at the issue links under which these exclusions are being done, and we should not be referring to release triage issues such as #6421

Copy link
Contributor

@smlambert smlambert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be referencing actual issues that articulate the intermittent failure (look in aqa-tests, infrastructure, and JBS existing issues that describe the problem, if an issue does not exist that describes the actual type of failure, then we should create one, some testcases may be failing in the same way, and can be grouped under the same issue)

@adamfarley
Copy link
Contributor Author

I am looking at the issue links under which these exclusions are being done, and we should not be referring to release triage issues such as #6421

Fair enough. Will screen them out.

@adamfarley
Copy link
Contributor Author

adamfarley commented Dec 11, 2025

It seems many of the triage issues have expired job links, so I'm going through each of the failing unit tests and identifying the actual problem for each based on more recent test runs. I'll keep an eye out for failures sharing symptoms (with each other and existing issues).

Plan: For each unreliable test:

  • If a test is linked to a triage issue, comment out the line.
  • Uncomment the line after one of these:
    • Finding an issue for the failure
    • Raising an issue for the failure
    • Reproducing the failure and then raising an issue

Grinders:

JDK11:

JDK17:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Exclude unreliables in the usual exclude files

3 participants