Expand comparative benchmarks to three-way hpke-ng vs hpke-rs vs rust-hpke suite

[danieldia-dev]

This PR substantially rewrites benches/comparative.rs to add rust-hpke as a third comparison point, fixes systematic PRNG overhead that was biasing every timed iteration, and fills in missing benchmark categories. It also adds a CONTRIBUTORS.md file to the repository.

See full benchmark suite output for numerical details: benchmark_results.txt (updated after review)

Changes

1. rust-hpke added as a third library. Where it lacks a feature (no K256, Vec allocation in the timed path biases the comparison, etc.) this is explicitly noted inline. Rows where it has no bare encap/decap equivalent carry a _via_setup_* suffix and a comparability warning (those include the full key schedule on top of the KEM).

2. PRNG unified to ChaCha20Rng. The previous code constructed a new OsRng on almost every b.iter() call, charging syscall overhead to every hpke-ng and rust-hpke measurement. hpke-rs was also calling seed() inside b.iter() (~58 ns/call). All three now use a single ChaCha20Rng::from_seed([0x42; 32]) per benchmark function, with seed() hoisted into setup code. The SEED buffer is also changed from a const &[u8] (which baked 64 MiB into the binary's rodata segment, inflating it to ~72 MB) to a static LazyLock<Vec<u8>> heap-allocated at first use, keeping the binary ~5 MB while retaining the full 64 MiB at runtime to prevent InsufficientRandomness panics on fast ops that Criterion runs 10^5-10^6 times per window.

3. New benchmark functions: bench_setup_x25519_aes256, bench_setup_p256_aes256, bench_open_x25519_aes128_payload_sweep, bench_context_open_x25519_chacha.

4. Criterion budgets justified per group. KEM/setup groups use the 60-sample/3s default. Seal/open payload sweeps use 40/2s (24 benchmarks per group; trimming keeps total wall-clock under ~2 min). Context seal/open uses 50/2s (cheaper per-iteration, fewer benchmarks). Round-trip uses 50/3s (most expensive per-iteration; two full KEM+key-schedule cycles).

Context open fix via a "lockstep" design

The naive approach to benchmarking context open fails (silently): a receiver context can only open message N when its sequence counter is at N, so pre-sealing one ciphertext and re-opening it breaks after the first iteration. The fix is "lockstep": establish both sender and receiver contexts once outside b.iter(), then seal-then-open inside each iteration. Both counters advance together, no KEM cost is included, and the reported time is seal + open combined. To get pure open latency, subtract the context_seal measurement. This design is applied to all three libraries.

Interesting benchmark observations

1. hpke-ng wins the headline round-trip at 205.35 µs vs hpke-rs 229.85 µs vs rust-hpke 246.03 µs (X25519/ChaCha20, 1 KiB). All three now have tight CIs ([203.14-208.26 µs], [229.50-230.21 µs], [245.60-246.47 µs]); the wide rust-hpke CI seen in earlier runs was a benchmark artifact from setup_receiver being called inside b.iter(), resolved by the lockstep fix.

2. P-256 rust-hpke encap is ~50% slower than hpke-rs (565.10 µs vs 377.46 µs, both measured via setup_sender); despite all three sharing the same RustCrypto p256 crate. Since key generation is identical across all three (~185 µs), the overhead is in rust-hpke's wrapper, not the field arithmetic. Worth investigating upstream.

3. ML-KEM derive_key_pair: the previously reported 8–12% gap has essentially closed. hpke-ng now measures 51.185 µs vs hpke-rs 51.894 µs for ML-KEM-768 (hpke-ng ~1.4% faster), and 82.678 µs vs 81.287 µs for ML-KEM-1024 (hpke-ng ~1.7% slower, within noise). The gap visible in the prior run was almost certainly a benchmark artifact from PRNG overhead and unhoisted seed() calls contaminating the timed path; both are now fixed. generate is similarly within noise across all sizes.

4. Context seal: three-way tie at medium-to-large payloads, with a notable exception at 1 KiB. At 16 KiB all three libraries are essentially identical (hpke-ng 21.461 µs, hpke-rs 21.315 µs, rust-hpke 21.341 µs). At 64 KiB, rust-hpke trails by ~3% (87.646 µs vs 85.138/85.551 µs). However, at 1 KiB, hpke-ng's context_seal (2.1414 µs) is ~17% slower than hpke-rs (1.8238 µs) and rust-hpke (1.8272 µs). This is an isolated mid-size regression worth investigating in hpke-ng's AEAD framing path.

5. hpke-ng context open at 16 KiB: prior anomaly was a benchmark artifact, now resolved. The previous run showed hpke-ng CI [47.201-56.225 µs] vs [42.5-43.0 µs] for the other two — that variance arose because the old benchmark called setup_receiver (including a full decap) inside b.iter(), making timing highly sensitive to KEM latency fluctuations. With the lockstep fix, all three now show tight CIs ([42.582-42.754 µs], [42.645-42.816 µs], [42.332-42.550 µs]) and essentially identical derived open latencies: hpke-ng 42.676 - 21.461 = 21.215 µs, hpke-rs 42.722 - 21.315 = 21.407 µs, rust-hpke 42.435 - 21.341 = 21.094 µs.

6. X-Wing generate/derive_key_pair and ML-KEM generate are within noise between hpke-ng and hpke-rs. This is absolutely not surprising, since the KEM implementations are effectively equivalent on those paths.

[nadimkobeissi]

That's a big one. Some notes:

The 64 MiB SEED blows up the bench binary

Building --release --bench comparative produces a 72 MB binary, and ~64 MB of that is just the zero-filled rodata segment from const SEED. Probably a fix, please try:

use std::sync::LazyLock;
static SEED: LazyLock<Vec<u8>> = LazyLock::new(|| vec![0x5Eu8; 8 * 1024 * 1024]);

That drops the binary back to ~5 MB and keeps the headroom you want.

The export exclusion comment is wrong

rust-hpke's export(label, &mut buf) isn't incompatible: let mut buf = vec![0; len]; ctx.export(b"…", &mut buf)? is a one-liner. I can think of different reasons: "different allocation pattern, biases the comparison"? Or maybe "wrapper overhead would dominate at these lengths."? Also small grammar bug: "making its export API is incompatible", drop the "is".

rust-hpke absent in some benchmarks without explanation

• setup_sender_psk skips rust-hpke without saying why (it supports OpModeS::Psk fine).
• kem/xwing and kem/mlkem768/1024 omit rust-hpke (no PQ KEMs there), but only kem/k256 has the explicit "rust-hpke omitted: …" explaining why.
• The doc header promises setup_receiver "across multiple ciphersuites" but it only exists for x25519/chacha20 + the three PQ suites?
• Kem as RustHpkeKem — the alias is never used by name. use hpke::Kem as _; says what you mean.

[danieldia-dev]

@nadimkobeissi Thanks for the detailed review! All four points are addressed: const SEED is replaced with static SEED: LazyLock<Vec<u8>> (binary back to ~5 MB), the export comment now gives the real reason (Vec allocation in the timed path biases the comparison), rust-hpke is added to setup_sender_psk, all absent-without-explanation groups now have detailed omission comments citing the relevant commit or reason, the doc header is corrected, and Kem as RustHpkeKem is changed to Kem as _. I also ran a full fresh benchmark suite after all fixes (attached above).