Add ML-KEM PQC support to hsmCompatVerify tools

[ladycfu]

Adds comprehensive ML-KEM (FIPS 203) support to hsmCompatVerify verification tools:
Depends on #5363 (CryptoUtil ML-KEM bits fix)

PQC Mode:

• --pqc flag enables ML-DSA for CA and ML-KEM for transport/storage
• --pqc-ca-algorithm: ml-dsa-44, ml-dsa-65, ml-dsa-87 (default: ml-dsa-65)
• --pqc-kem-algorithm: ml-kem-512, ml-kem-768, ml-kem-1024 (default: ml-kem-768)
• Uses NIST FIPS 203/204 standard hyphenated algorithm names

User Key Type Support:

• --user-key-type: RSA, EC, or ML-KEM (default: ML-KEM if --pqc, RSA otherwise)
• Allows verifying ML-KEM transport with different user key types
• Isolates ML-KEM transport functionality from user key operations

PKCS#12 Modes:

• --pkcs12-mode: kwp (AES-KWP), cbc (AES-256-CBC PBKDF2), legacy (3DES-CBC)
• Default: kwp
• Supports ML-KEM key export to PKCS#12

CryptoUtil Integration:

• Uses CryptoUtil.getMLDSAStrength() for algorithm name mapping
• Uses CryptoUtil.getMLDSASignatureAlgorithm() for signature algorithms
• Uses CryptoUtil.getMLKEMStrength() for KEM strength mapping
• Uses CryptoUtil.encapsulateMLKEM() for key wrapping
• Uses CryptoUtil.decapsulateMLKEM() for key unwrapping (3 occurrences)
• Replaces ~100 lines of switch statements and KEM boilerplate with helper calls

Bug Fixes:

• Fix --p12-output path handling to respect relative paths from CWD
• Add ML-KEM recognition to CryptoUtil.unwrap()

Verification Improvements:

• --test-decrypt-user flag for debugging wrapped key data
• Enhanced configuration output for PQC-specific options
• Hides non-applicable options in PQC mode

This enables end-to-end verification of ML-KEM key encapsulation for KRA transport and storage certificates, validating the PQC migration path.

IDM-6295, IDM-5815, IDM-5475
Assisted-by: Claude

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 424d6b44-8735-45fb-8ac1-07f7e9e3850f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Code Review

This pull request implements Post-Quantum Cryptography (PQC) support in the HSM compatibility verification tools, adding ML-DSA for CA signing and ML-KEM for KRA transport and storage. The changes include new command-line options, refactored initialization logic, and updated archival/verification workflows. Review feedback identifies a potential deadlock in process execution, security concerns regarding temporary file permissions and default charsets, and recommends using temporary HSM objects to avoid persistent artifacts.

[gemini-code-assist]

Calling p.waitFor() before consuming the process output can lead to a deadlock if the process produces enough output to fill the OS pipe buffer. The output should be read while the process is running or before waiting for it to complete to ensure the process can finish execution.

                StringBuilder output = new StringBuilder();
                try (java.io.BufferedReader reader = new java.io.BufferedReader(
                        new java.io.InputStreamReader(p.getInputStream()))) {
                    String line;
                    while ((line = reader.readLine()) != null) {
                        output.append(line).append("\n");
                    }
                }
                int exitCode = p.waitFor();
                if (exitCode != 0) {
                    log("    certutil output:\n" + output.toString());
                    throw new Exception("Failed to create NSS database (exit code: " + exitCode + ")");
                }

[gemini-code-assist]

Using File.createTempFile and String.getBytes() without an explicit charset can lead to security and portability issues. File.createTempFile may create files with default permissions that are too permissive for sensitive data like passwords. It is recommended to use java.nio.file.Files.createTempFile (which typically uses more restrictive permissions on POSIX systems) and to specify StandardCharsets.UTF_8 when writing the password.

Suggested change

	java.io.File tmpPwdFile = java.io.File.createTempFile("nsspwd", ".txt");
	tmpPwdFile.deleteOnExit();
	java.nio.file.Files.write(tmpPwdFile.toPath(), pkiservPasswd.getBytes());
	java.nio.file.Path tmpPwdPath = java.nio.file.Files.createTempFile("nsspwd", ".txt");
	java.io.File tmpPwdFile = tmpPwdPath.toFile();
	tmpPwdFile.deleteOnExit();
	java.nio.file.Files.write(tmpPwdPath, pkiservPasswd.getBytes(java.nio.charset.StandardCharsets.UTF_8));

[gemini-code-assist]

Specify an explicit charset (e.g., StandardCharsets.UTF_8) when converting the PEM string to bytes to ensure consistent behavior across different platforms.

Suggested change

	java.nio.file.Files.write(java.nio.file.Paths.get(transportCertFile), transportCertPEM.getBytes());
	java.nio.file.Files.write(java.nio.file.Paths.get(transportCertFile), transportCertPEM.getBytes(java.nio.charset.StandardCharsets.UTF_8));

[gemini-code-assist]

Specify an explicit charset (e.g., StandardCharsets.UTF_8) when converting the PEM string to bytes to ensure consistent behavior across different platforms.

Suggested change

	java.nio.file.Files.write(java.nio.file.Paths.get(storageCertFile), storageCertPEM.getBytes());
	java.nio.file.Files.write(java.nio.file.Paths.get(storageCertFile), storageCertPEM.getBytes(java.nio.charset.StandardCharsets.UTF_8));

[gemini-code-assist]

Unwrapping the user private key as a permanent object (false) on the HSM may fail if the token is read-only or lacks appropriate permissions, and it leaves persistent objects on the token. Since this is a verification tool, consider using true (temporary) instead, as is done elsewhere in the code (e.g., line 2757). If a permanent key is required for specific operations, it should be explicitly deleted after use.

Suggested change

	PrivateKey unwrappedUserPrivate = CryptoUtil.unwrap(
	hsmTokenObj,
	userPublicKey,
	false, // permanent - try permanent keys for ML-KEM PKCS#12 export
	PrivateKey unwrappedUserPrivate = CryptoUtil.unwrap(
	hsmTokenObj,
	userPublicKey,
	true, // temporary

[sonarqubecloud]

Quality Gate failed

Failed conditions
2 Security Hotspots
D Security Rating on New Code (required ≥ A)
D Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE