plan-lms: living plan for LMS and mbedtls-v4 work

Ongoing planning document tracking LMS support and the Mbed TLS 4.x
PSA porting effort. Updated in place as work progresses; will be
converted into proper documentation once the implementation lands.

Signed-off-by: David Brown <david.brown@linaro.org>

Author: d3zd3z

plan-lms.md

@@ -0,0 +1,366 @@
+# Plan to implement LMS in MCUboot
+
+The LMS digital signature algorithm {insert RFC and FIPS references with links} is a post-quantum
+digital signature algorithm.  It has some unusual features that make it a reasonable choice for a
+bootloader, even though these are limitations for other applications:
+
+- Keys are very small.  The public key is a single SHA, for SHA-256 (recommended by CNSA-2.0),
+  that's 32 bytes. The private key is generally 32-bytes plus a small counter.
+- Signatures are moderate. Larger than ECDSA or RSA signatures, but smaller than ML-DSA signature.
+  The algorithm is parameterizable with the main tradeoff being between signature size, and speed of
+  verification.
+- Verification is fast, typically on the order of 10s of ms, and consist entirely of the repeated
+  application of the given hash function.
+- Importantly: it is based on a **one time signature** algorithm, LM-OTS.  If the same underlying
+  key is ever used to sign more than one message, security is broken.  LMS builds a Merkle tree
+  around this, allowing at key generation time, this balance to be determined.
+
+At key generation time, there are a small set of parameters.  For the LM-OTS part:
+
+- N - size of hash function, 32 for SHA-256.
+- W - the width (in bits) of the Winternitz coefficients, this is the primary tradeoff value
+
+| w | p (chains) | Chain length (2^w) | Hash ops to verify (worst case) | Signature size (p×n bytes) |
+|---|---|---|---|---|
+| 1 | 265 | 2 | 265 | 8,480 bytes |
+| 2 | 133 | 4 | 399 | 4,256 bytes |
+| 4 | 67 | 16 | 1,072 | 2,144 bytes |
+| 8 | 34 | 256 | 8,704 | 1,088 bytes |
+
+For LMS, there are two parameters:
+
+- m - The number of bytes with each node, also 32, the size of the hash function.
+- h - The height of the merkel tree.
+
+The height determines the number of total signatures that can be made over the lifetime of the key.
+Larger values allow for more signatures to be made, but also increases the size of the message.
+
+| h | Total signatures | Additional signature bytes |
+|---|---|---|
+| 5 | 32 | 168 bytes |
+| 10 | 1,024 | 328 bytes |
+| 15 | 32,768 | 488 bytes |
+| 20 | 1,048,576 | 648 bytes |
+| 25 | 33,554,432 | 808 bytes |
+
+## Mbed TLS 4.1 constraints on LMS
+
+Mbed TLS 4.1.0 ships an LMS implementation in
+`ext/mbedtls-4.1.0/tf-psa-crypto/extras/lms.c` (header
+`tf-psa-crypto/include/mbedtls/lms.h`). The relevant facts that shape
+this plan:
+
+- The implementation **supports only one parameter set**:
+  `MBEDTLS_LMS_SHA256_M32_H10` + `MBEDTLS_LMOTS_SHA256_N32_W8`. The w
+  and h tradeoff tables above become theoretical — in practice we get
+  h=10 (1024 signatures) and w=8 (1088-byte signature, ~8,700 SHA-256
+  ops to verify) and nothing else.
+- The library advertises only LMS *verification* by default
+  (`MBEDTLS_LMS_C`). LMS *signing* requires the separate
+  `MBEDTLS_LMS_PRIVATE`, and since mcuboot only verifies, we do not
+  need to enable it. Signing is done externally by imgtool using a
+  non-Mbed-TLS LMS library.
+- The public key is **56 bytes**, not 32: 4 bytes LMS type + 4 bytes
+  LMOTS type + 16 bytes I (key id) + 32 bytes M-node. A future TLV
+  design must allocate for 56 bytes.
+
+## Current state
+
+The work below was done on the `add-lms` branch. Commits in order:
+
+- `ext/mbedtls: rename submodule to ext/mbedtls-3.6.0`
+- `ext/mbedtls-4.1.0: add as second Mbed TLS submodule`
+- `ptest: ignore local .log output files`
+- `ext/tinycrypt: hmac.c: avoid Clang 19+ const-VLA diagnostic`
+- `sim: build Mbed TLS 4.1.0 via upstream CMake for mbedtls-v4`
+- `plan-lms: reflect CMake-driven Mbed TLS 4.1 build`
+- `sim: expand CI matrix to cover mbedtls-v4 combinations`
+- `boot: encrypted_psa: Mbed TLS 4.x compat and -Werror fixes`
+- `sim: enable genuine PSA encryption for sig-ecdsa-psa+enc-ec256+mbedtls-v4`
+- `sim: extend mbedtls-v4 PSA encryption to enc-aes256-ec256`
+
+### Dual-submodule arrangement
+
+Rather than replacing Mbed TLS outright, the submodule was renamed to
+`ext/mbedtls-3.6.0` and a second submodule `ext/mbedtls-4.1.0` was
+added alongside. This preserves the LTS-backed build paths for the
+legacy crypto simulator features (`sig-rsa`, `sig-ecdsa-mbedtls`,
+`enc-rsa`, `enc-ec256-mbedtls`, `enc-x25519-mbedtls`), whose APIs are
+demoted to `private/` headers in 4.x and would require substantial
+rework to port. New code and PSA-based features migrate to 4.1
+incrementally via the `mbedtls-v4` Cargo feature.
+
+The Zephyr port is unaffected — it consumes Zephyr's own Mbed TLS
+module, not either of our submodules.
+
+### PSA features ported (CMake-driven)
+
+The 4.x build is driven through upstream's own CMake, matching the
+approach Zephyr adopted when it moved to Mbed TLS 4.1: stop shadowing
+the upstream build and let it own the file layout, generator
+plumbing, and config-adjustment logic. Mechanism:
+
+- `sim/mcuboot-sys/Cargo.toml`: `mbedtls-v4` feature and
+  `cmake = "0.1"` build-dep.
+- `sim/mcuboot-sys/csupport/config-ec-psa-v4.h`: TF-PSA-Crypto 1.1.0
+  config using `PSA_WANT_*` macros. The 4.x
+  `crypto_adjust_config_enable_builtins.h` infrastructure
+  auto-enables the corresponding `MBEDTLS_*_C` internals. Gated
+  blocks (`#if defined(MCUBOOT_SIGN_EC384)`,
+  `#if defined(MCUBOOT_ENCRYPT_EC256)`, etc.) layer in algorithms
+  for each enabled Cargo feature.
+- `sim/mcuboot-sys/build.rs`: new `add_mbedtls_v4_psa_ecdsa()`
+  branch invokes the `cmake` crate on
+  `ext/mbedtls-4.1.0/tf-psa-crypto` with `TF_PSA_CRYPTO_CONFIG_FILE`,
+  `ENABLE_{PROGRAMS,TESTING}=OFF`,
+  `USE_STATIC_TF_PSA_CRYPTO_LIBRARY=ON`,
+  `TF_PSA_CRYPTO_FATAL_WARNINGS=OFF`, builds the `tfpsacrypto`
+  target, and links the resulting `libtfpsacrypto.a`. CMake's own
+  `GEN_FILES` path invokes the upstream Python generators for
+  `psa_crypto_driver_wrappers*` and `tf_psa_crypto_config_check_*.h`
+  — we don't replicate that plumbing.
+- `sim/mcuboot-sys/csupport/psa_rng_stub_v4.c`: self-contained stub
+  for `mbedtls_psa_external_get_random` plus empty
+  enable/disable-insecure-RNG toggles. The equivalent upstream test
+  shim (`framework/tests/src/fake_external_rng_for_test.c`) drags in
+  a large transitive header tree (`test_common.h` →
+  `tf_psa_crypto_common.h` → `<test/build_info.h>`), which
+  verification-only tests do not need.
+- `scripts/requirements.txt`: `jinja2`, `jsonschema` — required by
+  upstream's generators. 4.x does not ship pre-generated files;
+  CMake regenerates them on every configure.
+
+Feature combinations that build and pass 25/25 tests on `mbedtls-v4`:
+
+- `sig-ecdsa-psa` (base — signature verification only)
+- `sig-ecdsa-psa sig-p384`
+- `sig-ecdsa-psa` crossed with each of: `swap-move`, `swap-offset`,
+  `bootstrap`, `max-align-16`, `validate-primary-slot`,
+  `overwrite-only`, `multiimage`, `ram-load`, `direct-xip`,
+  `overwrite-only downgrade-prevention`,
+  `hw-rollback-protection multiimage`.
+- `sig-ecdsa-psa enc-ec256` — first real-PSA encryption path (vs.
+  the 3.6 stub). Uses `encrypted_psa.c` for primitives + `encrypted.c`
+  for the high-level `boot_enc_*` interface, with
+  `CONFIG_BOOT_ECDSA_PSA` defined to skip `encrypted.c`'s duplicated
+  legacy ASN.1/ECDH block.
+- `sig-ecdsa-psa enc-aes256-ec256` — same machinery, with
+  `MCUBOOT_AES_256` defined. PSA_KEY_TYPE_AES covers all AES key
+  sizes, so no config delta.
+
+### Build-time prerequisites for `mbedtls-v4`
+
+- `cmake` in PATH (most Rust developers already have it; the `cmake`
+  crate requires it).
+- `python3` with `jinja2` and `jsonschema` available to the
+  interpreter CMake finds via `find_package(Python3)`.
+
+### Gotchas / lessons learned
+
+- **Any `MCUBOOT_*` macro that gates `PSA_WANT_*` entries in
+  `config-ec-psa-v4.h` must be forwarded to CMake**, not just to the
+  `cc::Build`. Otherwise the library compiles without that algorithm
+  enabled while the boot code uses it, yielding silent
+  PSA_ERROR_NOT_SUPPORTED failures (or, in the P-384 case we hit,
+  seven upgrade tests failing under `sig-p384 mbedtls-v4` because
+  `PSA_WANT_ECC_SECP_R1_384` wasn't switched on in the library build).
+  `add_mbedtls_v4_psa_ecdsa()` handles this via
+  `cmake_conf.cflag("-DMCUBOOT_SIGN_EC384")` /
+  `"-DMCUBOOT_ENCRYPT_EC256"`; any future gate macro needs the same.
+- **`encrypted.c` guards its legacy ASN.1/ECDH block with Zephyr
+  Kconfig symbols** (`CONFIG_BOOT_ECDSA_PSA`,
+  `CONFIG_BOOT_ED25519_PSA`) rather than `MCUBOOT_USE_PSA_CRYPTO`.
+  Our sim build therefore defines `CONFIG_BOOT_ECDSA_PSA` — it's
+  load-bearing, not cosmetic. This is a small OS-leakage that
+  arguably ought to be cleaned up upstream (the guard should key on
+  `MCUBOOT_USE_PSA_CRYPTO`, not a Zephyr-namespace symbol).
+- **`encrypted_psa.c`'s `bootutil_aes_ctr_{encrypt,decrypt}` ignore
+  `blk_off`**. The tinycrypt and mbedtls versions both thread it
+  through as the CTR sub-block offset. This may be fine for paths
+  that stay block-aligned, but is almost certainly the cause of the
+  x25519 failure below — the PSA-side comment claiming "PSA handles
+  CTR block alignment internally" is not accurate.
+
+### Local carries
+
+- `ext/tinycrypt/lib/source/hmac.c` — Clang 19+ const-VLA diagnostic.
+  See `TASKS.md`. Vendored tinycrypt, lives forever.
+- `boot/bootutil/src/encrypted_psa.c` — three carry patches for
+  Mbed TLS 4.x + `-Werror -Wall -Wextra`: define
+  `MBEDTLS_OID_EC_ALG_UNRESTRICTED` / `_EC_GRP_SECP256R1` when
+  unavailable (they moved to a private header in 4.x), cast the
+  `"MCUBoot_ECIES_v1"` string literal to `const uint8_t *`, mark
+  `blk_off` as intentionally unused. Should flow back upstream as a
+  proper 4.x compatibility fix; drop this carry when it does.
+
+### Known gaps
+
+- The main `~/ai/pythons` venv needs `jinja2` + `jsonschema` for
+  `cargo test` to work without a PATH override. Update the uv-managed
+  `pyproject.toml` alongside this plan's completion.
+- `sig-ecdsa-psa enc-x25519 mbedtls-v4` (and the aes256 variant) is
+  wired but fails 7/25 upgrade-path tests. Private key parses, HKDF
+  and MAC verification both succeed, but image-payload AES-CTR
+  produces wrong bytes. Most likely cause is the `blk_off`-is-ignored
+  issue in `encrypted_psa.c` noted above; full triage notes and a
+  concrete resume step are in `TASKS.md`. The combo works on the 3.6
+  stub path, so this blocks only the 4.x port of those two features.
+
+## Plan
+
+- [x] Add Mbed TLS 4.1.0 alongside the 3.6 LTS as a second
+  submodule (`ext/mbedtls-4.1.0`).
+- [x] Port one PSA-only simulator feature (`sig-ecdsa-psa`) to build
+  against 4.1 via the `mbedtls-v4` Cargo feature.
+- [x] Resolve hand-picked-vs-CMake question: adopted CMake via the
+  `cmake` crate for the 4.x build path, per the Zephyr precedent.
+- [x] Port remaining PSA-based simulator features to `mbedtls-v4`
+  (done except for the x25519 variants):
+  - [x] `sig-p384` — verified; uncovered and fixed a bug where
+    `MCUBOOT_SIGN_EC384` wasn't being forwarded to the CMake build,
+    so the library lacked P-384 support.
+  - [x] `sig-ecdsa-psa swap-move bootstrap max-align-16` plus a
+    broader set of orthogonal feature crosses (see above).
+  - [x] `sig-ecdsa-psa enc-ec256` / `enc-aes256-ec256` — moved from
+    the `psa_crypto_init_stub.c` path to genuine PSA-based
+    encryption via `encrypted_psa.c`.
+  - [ ] `sig-ecdsa-psa enc-x25519` / `enc-aes256-x25519` — tests
+    fail, probably `blk_off` handling. See Known gaps and
+    `TASKS.md`.
+- [x] Add and document a new image-trailer TLV type for LMS signatures
+  in `boot/bootutil/include/bootutil/image.h` and `docs/design.md`.
+  Allocated `IMAGE_TLV_LMS = 0x26` (next free slot after
+  `IMAGE_TLV_SIG_PURE`). Added to the unprotected-TLV allow list in
+  `image_validate.c`. `docs/design.md` now has a short subsection
+  spelling out the RFC 8554 wire formats so the 56-byte public key
+  (`u32 lms_type | u32 lmots_type | 16-byte I | 32-byte T[1]`) and
+  1452-byte signature (for the single `LMS_SHA256_M32_H10` +
+  `LMOTS_SHA256_N32_W8` set Mbed TLS 4.x supports) are documented at
+  the level a future reader will want. No new TLV is needed for the
+  public key itself — the existing `IMAGE_TLV_PUBKEY` / `KEYHASH`
+  pair covers it.
+- [x] Add LMS key generation and image signing to imgtool. Landed via
+  the `pyhsslms` Pure-Python package (Russ Housley, RFC 8554 author;
+  BSD-3-Clause). No pip-installable Python wrapper around a C LMS
+  exists today — `pyca/cryptography` does not implement LMS, and
+  `liboqs-python` requires a system `liboqs` install which would
+  break a vanilla `pip install` of imgtool from PyPI.
+
+  Implementation notes:
+
+  - `scripts/imgtool/keys/lms.py` wraps `pyhsslms.LmsPrivateKey` /
+    `LmsPublicKey`. Constructor takes `(lms_type, lmots_type)` so
+    additional parameter sets are a one-line addition to
+    `LMS_PARAM_SETS`. Initially exposes `lms-sha256-h10-w8` (the
+    only set Mbed TLS 4.x verifies) and `lms-sha256-h5-w8` (much
+    faster, useful for round-trip tests).
+  - Private-key file is a custom PEM-style envelope (`-----BEGIN
+    MCUBOOT LMS PRIVATE KEY-----`) holding the 60-byte serialized
+    state `lms_type || lmots_type || SEED || I || q`.
+    `keys/__init__.py:load()` sniffs the `BEGIN MCUBOOT LMS` marker
+    before falling through to the cryptography PEM loader.
+  - `sign_digest()` advances `q` in-memory then atomically rewrites
+    the key file (tmpfile + fsync + rename) before returning the
+    signature. If the disk write fails, no signature is produced; if
+    a crash happens after the write but before the caller persists
+    the signed image, an LMS index is wasted but never reused. The
+    signing-policy hazard (re-signing from a stale backup) lives in
+    the `lms.py` module docstring.
+  - Public key is exported in the 56-byte RFC 8554 form (raw or
+    PEM-wrapped). `image.py` uses it directly for both
+    `IMAGE_TLV_PUBKEY` and `IMAGE_TLV_KEYHASH` (SHA-256 over the
+    same 56 bytes).
+  - `image.py:TLV_VALUES['LMS'] = 0x26` and a corresponding
+    `ALLOWED_KEY_SHA[LMS] = ['256']` entry; the rest of the verify
+    path picks up `verify_digest()` automatically since the LMS
+    wrapper deliberately omits a `verify` method.
+
+  Timing on a 2026 development laptop (CPython 3.13, single-thread):
+
+  - h=5 keygen: 0.12 s; load (rebuilds the tree): 0.12 s.
+  - h=10 keygen: 3.9 s; load: 3.9 s. Each `imgtool sign` invocation
+    pays the load cost because pyhsslms only persists the 60-byte
+    state, not the 1024-leaf precomputed tree. A long-lived signing
+    daemon would amortize this; for a CLI invocation per build, 4 s
+    on top of the existing build is tolerable but worth flagging.
+  - Sign and verify (post-load) are both ~2 ms.
+  - Signature size is 1452 bytes for h=10/w=8, matching
+    `docs/design.md`.
+
+  Tests in `scripts/tests/test_keys.py` parameterize over
+  `keygens.keys()`; the LMS variants pass `test_keygen`,
+  `test_getpub`, `test_getpubhash`, and `test_sign_verify`.
+  `test_keygen_type` (uses `openssl pkey`) and `test_getpriv`
+  (PKCS8/openssl formats) skip for LMS since LMS keys are not
+  PEM/PKCS8.
+- [ ] Add simulator support for LMS signatures.
+  - [x] Wire-format compatibility tests between imgtool's pyhsslms
+    wrapper and the `lms-signature` crate the simulator will use
+    (`sim/tests/lms_compat.rs`). Round-trips a fresh keypair in
+    both directions (Python→Rust and Rust→Python verification),
+    no committed key material; uses the H5/W8 set for keygen
+    speed. `lms-signature` is now a regular `[dependencies]` of
+    bootsim alongside `getrandom`, `rand_core`, `hybrid-array`,
+    `signature`.
+  - [x] Hook `lms-signature` into `TlvGen` so the simulator can
+    sign LMS test images and feed them through the bootloader's
+    verifier. The tests using this path will fail until the
+    bootloader-side LMS verification below lands.
+
+    Implementation notes:
+
+    - `sim-lms` Cargo feature added in `sim/Cargo.toml` and
+      `sim/mcuboot-sys/Cargo.toml`. No C code is gated by it yet —
+      the bootloader-side verifier doesn't exist.
+    - `TlvKinds::LMS = 0x26` matches `IMAGE_TLV_LMS` from the
+      bootutil header.
+    - `TlvGen::new_lms()` generates a fresh `SigningKey<LmsScheme>`
+      per call. `LmsScheme` is `LmsSha256M32H5<LmsOtsSha256N32W8>`
+      for now: H5 keygen runs in tens of ms, vs. seconds for H10.
+      When bootloader-side LMS verification (Mbed TLS 4.x) is
+      wired up, the alias must move to H10 because that is the
+      only scheme Mbed TLS 4.x supports — both the alias and
+      `LMS_SIG_LEN` (currently 1292; H10 = 1452) need updating
+      together.
+    - `make_tlv()` now takes `mut self: Box<Self>` so it can
+      `take()` the LMS signing key. It emits a `KEYHASH` TLV
+      (SHA-256 over the 56-byte serialized public key) followed
+      by an `LMS` TLV containing the 1292-byte signature.
+    - `image.rs:make_tlv()` short-circuits to `TlvGen::new_lms()`
+      when `cfg!(feature = "sig-lms")` is set, since there is no
+      `Caps::Lms` bit yet (no bootloader code to publish it).
+    - `TlvGen` no longer derives `Debug` because
+      `LmsSigningKey` does not implement it. Nothing in the tree
+      printed `TlvGen` via `{:?}`.
+
+    With `--features sig-lms`, all tests fail at the upgrade step
+    ("Unable to perform basic upgrade") because the bootloader
+    rejects the image. That confirms the simulator-side path is
+    sound and isolates the remaining work to the C side.
+- [ ] Generalize the signing/verification interface as needed to
+  support LMS (follow `plan-refactor.md`). For verification the
+  interface should look much like the existing ones — no state to
+  maintain at verify time.
+- [ ] Add LMS signature verification support to the bootloader:
+  - Config knob (`MCUBOOT_SIGN_LMS` or similar)
+  - Zephyr Kconfig support
+  - Integration into the boot path (`image_lms.c` alongside the
+    existing `image_ecdsa.c`, `image_rsa.c`, `image_ed25519.c`)
+- [ ] Expand `.github/workflows/sim.yaml` matrix to cover LMS
+  (probably `sig-lms` feature, combined with the usual swap/validate/
+  enc matrix, though see constraint note below).
+- [ ] Retire `ext/mbedtls-3.6.0` once all features have been ported
+  to 4.x (or explicitly dropped). 3.6 has LTS coverage until 2027, so
+  no rush; this is a longer-term cleanup.
+
+## Notes for future work
+
+- LMS is verification-only in this plan; signing stays in imgtool.
+  The stateful-private-key hazard is the single biggest operational
+  risk — a signing-policy document may be warranted before this lands
+  in production (not part of this branch's scope).
+- ML-DSA (Dilithium) source is already sitting in the 4.1 submodule
+  at `ext/mbedtls-4.1.0/tf-psa-crypto/drivers/pqcp/mldsa-native/`.
+  With the CMake path in place, adding a second PQC algorithm is
+  primarily a matter of extending the config header and declaring
+  the TLV/bootloader glue.