Skip to content

JCA conformance harness: scaffolding - #556

Open
WillChilds-Klein wants to merge 1 commit into
mainfrom
jca-conformance-scaffolding
Open

JCA conformance harness: scaffolding#556
WillChilds-Klein wants to merge 1 commit into
mainfrom
jca-conformance-scaffolding

Conversation

@WillChilds-Klein

Copy link
Copy Markdown
Contributor

Summary

First of two PRs implementing #550. Lands the scaffolding for running the JDK's own jtreg crypto conformance tests against ACCP, to catch behavioral divergence from the JDK-default providers (SunJCE, SunEC, SunRsaSign).

This PR is deliberately inert with respect to the test suite itself — it wires the harness end-to-end in --smoke mode and leaves the actual OpenJDK test-tree checkout + run to the follow-up.

What's here

  • tests/jca-conformance/run-jtreg.sh — driver script.
    • --smoke (default): builds/loads ACCP, verifies it installs as the highest-priority provider under the JDK under test, serves an overridden SPI (SHA-256), and that the config files parse. No OpenJDK checkout, fast, no external test dependency.
    • --full: placeholder; implemented in PR 2/2.
  • tests/jca-conformance/jtreg-test-roots.txt — JCA-relevant test/jdk subtrees (ciphers, Mac, Signature, KeyFactory, EC/RSA, provider tests), excluding areas ACCP doesn't implement (TLS/JSSE, PKCS#11, Kerberos).
  • tests/jca-conformance/exclusions/{common,jdk17,jdk21}.txt — per-JDK exclusion lists, empty to start. A failing test gets an entry here with a required rationale comment (linking DIFFERENCES.md or a tracking issue), keeping CI green while making the divergence backlog explicit in source.
  • tests/jca-conformance/README.md — design, exclusion model, and the JCK-access note (JCK can't be fetched anonymously; tracked separately).
  • .github/workflows/jca-conformance.yml — GHA job, matrix JDK 17 / 21, builds ACCP and runs the harness in --smoke mode.

Provider installation

ACCP is installed as provider #1 via a generated java.security.properties override. The native library self-extracts from the jar via ACCP's Loader, so only the jar needs to be on the classpath (no java.library.path).

Test plan

  • shellcheck tests/jca-conformance/run-jtreg.sh clean
  • --smoke passes locally on JDK 17 (ACCP installs as provider Initial commit #1, serves SHA-256)
  • CI: smoke job green on JDK 17 and 21

Follow-up (PR 2/2)

Switches the run to --full: checks out a pinned OpenJDK test tree, runs jtreg over the configured roots, enumerates failures, and adds an exclusion entry per failing test so CI is green with an explicit, reviewable list of divergences to work through.

Tracking: #550

First branch of issue #550. Wires up a harness for running the JDK's
own jtreg crypto conformance tests against ACCP, to catch behavioral
divergence from the JDK-default providers (SunJCE, SunEC, SunRsaSign).

This commit lands the scaffolding only:

- tests/jca-conformance/run-jtreg.sh: driver script. Runs in --smoke
  mode by default, validating that ACCP installs as the highest-priority
  provider under the JDK under test and that the config files parse. The
  --full mode (real OpenJDK test-tree checkout + jtreg run) is a
  placeholder delivered in branch 2.

- tests/jca-conformance/jtreg-test-roots.txt: the JCA-relevant test/jdk
  subtrees to run (ciphers, Mac, Signature, KeyFactory, EC/RSA, etc.),
  excluding areas ACCP does not implement (TLS/JSSE, PKCS#11, Kerberos).

- tests/jca-conformance/exclusions/{common,jdk17,jdk21}.txt: per-JDK
  exclusion lists, empty to start. A failing test gets an entry here
  (with a required rationale comment linking DIFFERENCES.md or an issue)
  so CI stays green while the divergence backlog stays explicit.

- tests/jca-conformance/README.md: design, exclusion model, JCK note.

- .github/workflows/jca-conformance.yml: GHA job, matrix JDK 17/21,
  builds ACCP and runs the harness in --smoke mode (non-gating wiring
  until branch 2 enables --full).

Tracking: #550
@WillChilds-Klein
WillChilds-Klein force-pushed the jca-conformance-scaffolding branch from c0a72ab to 6822ebc Compare July 31, 2026 17:15
@codecov

codecov Bot commented Jul 31, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.56%. Comparing base (7813506) to head (6822ebc).

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #556      +/-   ##
============================================
+ Coverage     78.55%   78.56%   +0.01%     
- Complexity     1217     1218       +1     
============================================
  Files           127      127              
  Lines          7759     7760       +1     
  Branches        982      983       +1     
============================================
+ Hits           6095     6097       +2     
+ Misses         1133     1132       -1     
  Partials        531      531              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant