JCA conformance harness: jtreg run + exclusions - #557
Open
WillChilds-Klein wants to merge 1 commit into
Open
Conversation
This was referenced Jul 31, 2026
Second branch of issue #550. Turns the scaffolding into a working conformance run and switches CI to it. Harness (run-jtreg.sh --full): - Builds jtreg from source at a pinned openjdk/jtreg tag (jtreg-version.txt). No third-party prebuilt binaries. - Sparse-checks-out the OpenJDK test content for the JDK under test from a pinned per-version GA tag (jdk-tags.txt), using a partial + sparse + shallow clone so only the crypto test subtrees and the jtreg support dirs (test/lib, test/jtreg-ext) are pulled, not the whole JDK source tree. Per-version pins because the test set differs by JDK version. - Installs ACCP at provider priority 1 by emitting a renumbered java.security override (ACCP at 1, JDK defaults shifted to 2..N+1). A bare security.provider.1=ACCP would OVERWRITE the SUN provider and break JDK-internal SecureRandom bootstrap (InternalError: SHA-1 not available, since ACCP doesn't register SUN's legacy SHA/SHA1 alias); renumbering keeps every default present so those aliases fall through. - Runs jtreg over the configured roots minus the exclusion lists and exits nonzero on any non-excluded failure. Config: - jdk-tags.txt: openjdk/jdk17u@jdk-17.0.13-ga, openjdk/jdk21u@jdk-21.0.5-ga - jtreg-version.txt: jtreg-7.5.1+1 Seeded exclusions (all triaged as non-ACCP-conformance issues): - common.txt: DefaultProviderList, Versions, SecurityProviderModularTest -- meta-tests asserting the JDK's exact default provider set, which ACCP intentionally perturbs by installing at priority 1. - jdk21.txt: CryptoPermissions/InconsistentEntries -- writes a policy file into the read-only JDK install dir (AccessDeniedException); environmental, would fail the same against SunJCE. Local results against ACCP: - JDK17: 533 passed, 3 excluded, exit 0. - JDK21: 567 passed, 4 excluded, exit 0. Workflow: jca-conformance.yml now runs --full on the JDK 17/21 matrix and uploads the jtreg report as an artifact. Tracking: #550
WillChilds-Klein
force-pushed
the
jca-conformance-jtreg-run
branch
from
July 31, 2026 17:15
91dd324 to
09eb4ff
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## jca-conformance-scaffolding #557 +/- ##
=================================================================
- Coverage 78.56% 78.56% -0.01%
+ Complexity 1218 1217 -1
=================================================================
Files 127 127
Lines 7760 7759 -1
Branches 983 982 -1
=================================================================
- Hits 6097 6096 -1
- Misses 1132 1133 +1
+ Partials 531 530 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Second of two PRs for #550, stacked on #556. Turns the scaffolding into a working conformance run against ACCP and switches CI to it.
Harness (
run-jtreg.sh --full)jtreg-version.txt(openjdk/jtreg). No third-party prebuilt binaries.jdk-tags.txt), via partial (--filter=blob:none) + sparse + shallow clone — only the crypto subtrees + jtreg support dirs (test/lib,test/jtreg-ext), not the whole JDK source (~226 MB vs multi-GB). Per-version pins because the test set differs by JDK version.java.securityoverride (ACCP at 1, JDK defaults shifted to 2..N+1).Why renumber rather than
security.provider.1=ACCPA bare
security.provider.1=ACCPoverwrites slot 1 (theSUNprovider), evicting it — which breaks JDK-internalSecureRandombootstrap withInternalError: SHA-1 not available(ACCP doesn't register SUN's legacySHA/SHA1alias) and fails nearly every test. Renumbering keeps every default present so legacy aliases fall through to SUN. This was the main harness bug found and fixed while bringing the run up.Seeded exclusions
All triaged as not ACCP algorithm-conformance issues:
common.txt:DefaultProviderList,Versions,SecurityProviderModularTest— meta-tests asserting the JDK's exact default provider set, which ACCP intentionally perturbs by installing at priority 1.jdk21.txt:CryptoPermissions/InconsistentEntries— writes a policy file into the read-only JDK install dir (AccessDeniedException); environmental, would fail the same against SunJCE.Each entry carries a rationale comment per the exclusion model.
Results against ACCP (local)
Workflow
jca-conformance.ymlnow runs--fullon the JDK 17/21 matrix and uploads the jtreg report as a CI artifact.Test plan
shellcheckcleanTracking: #550