Add OuterExtensions encoding for TLS ECH client

[sebastian-carpenter]

Summary

Adds client-side ECH ech_outer_extensions encoding (RFC 9849 §5.1). The inner ClientHello now encodes duplicate extensions so they can be expanded on the server-side later. This can shrink the HPKE-sealed payload significantly.

Enabled by default. Opt-out:

• wolfSSL_CTX_SetEchEncodeOE(ctx, enable)
• wolfSSL_SetEchEncodeOE(ssl, enable)

Changes

• TLSX_ECH_IsEncodeable — gate list of non-compressible extensions (SNI, ECH, ALPN, PSK, early_data).
• TLSX_ECH_BuildOuterExtensions — walks ssl->extensions then ssl->ctx->extensions and emits the ech_outer_extensions block. Capped at 127 extensions.
• TLSX_GetSizeWithEch / TLSX_WriteWithEch — size and write the encoded inner as [ech_outer_extensions][non-encodables] (what is sent on the wire), and the expanded inner as [encodables][non-encodables] (what is hashed for the transcript).
• TLSX_ECH_ExpandOuterExtensions — fix extension-length rewrite when the outer carries a session id.
• SendTls13ClientHello — computes both inner sizes, hashes the expanded form via EchHashHelloInner, then rewrites the buffer in encoded form for HPKE seal. ech->innerCount is set explicitly after TLSX_FinalizeEch rather than as a side effect of hashing.
• EchHashHelloInner — always receives the expanded body; no longer strips paddingLen + Nt.
• Options.disableEchEncodeOE / WOLFSSL_CTX.disableEchEncodeOE plus WOLFSSL_ECH.writeEncoded flag to select expanded vs. encoded within a single TLSX_WriteRequest path.
• CI (openssl-ech.sh / openssl-ech.yml) — adds --pqc argument and runs with SecP384r1MLKEM1024 so the inner exceeds openssl's ECH limit (if OuterExtensions was not used).

Testing

• test_wolfSSL_Tls13_ECH_outer_extensions covers CTX- and SSL-level disable.
• OpenSSL-server ↔ wolfSSL-client interop with an ML-KEM hybrid key share (CI). Previously would fail from the ECH extension being too big for openSSL.

Checklist

• added tests
• updated/added doxygen
• updated appropriate READMEs
• Updated manual and documentation

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds client-side ECH ech_outer_extensions encoding support (RFC 9849 §5.1) to shrink the sealed ClientHelloInner, with new APIs to enable/disable at CTX/SSL scope and CI interop updates.

Changes:

• Implement OuterExtensions encoding/expansion in the TLS extensions write/size paths and ClientHelloInner hashing/sealing.
• Add CTX/SSL options + public setters to opt out of the encoding (enabled by default).
• Extend CI OpenSSL ECH interop to run with an ML-KEM hybrid group and new --pqc plumbing.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
wolfssl/ssl.h	Exposes new public APIs to toggle ECH OuterExtensions encoding.
wolfssl/internal.h	Adds new ECH/writeEncoded flag and CTX/SSL option bits for encoding control.
src/ssl_ech.c	Implements the new CTX/SSL setters controlling the encoding behavior.
src/internal.c	Propagates CTX encoding-disable setting into per-SSL options at init.
src/tls.c	Adds OuterExtensions encoding selection and fixes expansion length rewrite for session-id cases.
src/tls13.c	Separates “expanded” vs “encoded” inner sizes and hashes expanded form before sealing encoded form.
tests/api.c	Adds coverage for disabling encoding via CTX and SSL APIs.
.github/workflows/openssl-ech.yml	Enables ML-KEM and runs OpenSSL↔wolfSSL interop using a PQC hybrid group.
.github/scripts/openssl-ech.sh	Adds --pqc CLI support and forwards it to the wolfSSL client.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

MemBrowse Memory Report

No memory changes detected for:

• gcc-arm-cortex-m4
• gcc-arm-cortex-m4-baremetal
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-tls12

[dgarske]

Do we really need to gate the OE (Outer Extensions) separate that the normal ECH? Is that because of a compatibility issue? I don't want to add more new public API's unless its 100% required. Also the OE is not intuitive on its own... Consider the "enable" argument as -> "enOutExt" or something more useful.

[sebastian-carpenter]

Oh! Thank you for raising this point. I took another look at the RFC and it seems the server has to implement OuterExtensions parsing. This removes the need for a flag on the client, so I'll remove it and have OE always be done.

[sebastian-carpenter]

Rebase + changes:

• Removed ability to turn off OuterExtensions
• Added some extra downgrade guards for the ECH code
• Found interop issue during HRR. Ended up forcing ECH to be printed last in the HRR since OpenSSL relies on it.
• Added more interop tests to openssl-ech.yml

[sebastian-carpenter]

Jenkins retest this please.