OpenSSL compat with ML-DSA

[kojo1]

Description

wolfSSL_PEM_read_bio_PrivateKey
wolfSSL_PEM_read_PrivateKey
wolfSSL_d2i_PrivateKey, wolfSSL_d2i_PublicKey
wolfSSL_X509_sign
wolfSSL_X509_REQ_sign
wolfSSL_X509_set_pubkey

Fixes zd#22151

Testing

Added test_wolfSSL_PEM_PrivateKey_mldsa
Added ML-DSA cases in test_wolfSSL_X509_set_pubkey

Checklist

• added tests
• updated/added doxygen
• updated appropriate READMEs
• Updated manual and documentation

[github-actions]

MemBrowse Memory Report

linuxkm-standard

• Data: __patchable_function_entries +8 B (+0.0%, 48,744 B)
  No memory changes detected for:
• gcc-arm-cortex-m0plus
• gcc-arm-cortex-m3
• gcc-arm-cortex-m4
• gcc-arm-cortex-m4-baremetal
• gcc-arm-cortex-m4-crypto-only
• gcc-arm-cortex-m4-dtls13
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-openssl-compat
• gcc-arm-cortex-m4-pkcs7
• gcc-arm-cortex-m4-pq
• gcc-arm-cortex-m4-rsa-only
• gcc-arm-cortex-m4-sp-math
• gcc-arm-cortex-m4-tls12
• gcc-arm-cortex-m4-tls13
• gcc-arm-cortex-m7
• gcc-arm-cortex-m7-pq
• gcc-arm-cortex-m7-tls13
• linuxkm-pie
• stm32-sim-stm32h753

[Frauschi]

🐺 Skoll Code Review

Overall recommendation: APPROVE
Findings: 4 total — 3 posted, 2 skipped

Posted findings

• [High] ML-DSA private key decode in X509 sign/set_pubkey missing PRIVATE_KEY_UNLOCK bracket (fails under FIPS) — src/x509.c:12827, src/x509.c:16508
• [Medium] WC_MAX_X509_GEN increase puts 20 KB on the stack in wolfSSL_X509_REQ_sign for all key types — src/x509.c:12953-12962, 16922-16923
• [Medium] wolfSSL_X509_REQ_sign changes lack ML-DSA test coverage — src/x509.c:16918-16950

Skipped findings

• [Low] fnames[] can become a zero-length array if all ML-DSA levels are disabled
• [Medium] WC_MAX_X509_GEN increase puts 20 KB on the stack in wolfSSL_X509_REQ_sign for all key types

Review generated by Skoll via Claude/Codex

[Frauschi]

🟡 [Medium] wolfSSL_X509_REQ_sign changes lack ML-DSA test coverage
💡 SUGGEST test

The PR modifies wolfSSL_X509_REQ_sign (buffer sizing) and the wolfSSL_X509_resign_cert ML-DSA signing path that it shares with wolfSSL_X509_sign. The new tests exercise wolfSSL_X509_sign (in test_wolfSSL_X509_set_pubkey) and PEM/d2i loading, but there is no test that signs a certificate REQUEST with an ML-DSA key. A grep of tests/api/test_x509.c (which holds the REQ_sign tests) shows no ML-DSA/dilithium references. The REQ path has its own buffer-declaration mechanism (WC_DECLARE_VAR stack vs. XMALLOC heap) and argument ordering, so it is worth covering separately.

Suggestion: Add a test that builds an X509_REQ, calls wolfSSL_X509_set_pubkey/X509_REQ_sign with an ML-DSA key, and verifies the resulting request.

[kojo1]

lack ML-DSA test coverage

Add ML-DSA test coverage for X509_REQ_sign

missing PRIVATE_KEY_UNLOCK bracket

PRIVATE_KEY_UNLOCK to X509 sign/set_pubkey

WC_MAX_X509_GEN increase puts 20 KB on the stack

Dynamic buff size based on expected cert size