Background
A perf profile of the 3,000,000-request pi_basic workload attributes approximately 24.06% inclusive CPU to CommittableTx::serialise(). Across the full profile, memory copy/set leaves account for 5.06% and allocation/refcount leaves for 16.25%; within transaction serialization alone these categories account for approximately 1.08% and 2.02% of total CPU respectively.
The current private-domain serialization path creates and copies through several buffers:
GenericSerialiseWrapper builds separate public and private writer buffers.
serialise_domains() allocates the final ledger entry.
- It allocates a
serialised_hdr vector and an encrypted_private_domain vector.
KeyAesGcm_OpenSSL::encrypt() allocates another local ciphertext vector, fills it, and move-assigns it over the caller-provided vector.
- The GCM header, public domain, and encrypted private domain are copied into the final ledger entry.
This work is separate from reusing initialised AES-GCM contexts (#8168) and should be implemented/measured independently.
Proposed implementation
- Add a span/caller-owned-output encryption API alongside or beneath the existing vector API.
- Precompute the complete serialised-entry layout and allocate the final ledger entry once.
- Write the entry header, public-domain size, and public-domain bytes directly to their final positions.
- Encrypt the private domain directly into its final ciphertext span in the ledger entry, while continuing to authenticate the public domain as AAD.
- Serialize the fixed-size GCM header/tag directly into the reserved header region once encryption completes.
- Remove the local
ciphertext allocation in KeyAesGcm_OpenSSL::encrypt() when the caller already provides correctly sized output storage.
- Consider moving/taking writer buffers where direct output is not possible, but keep that secondary to the single-allocation final layout.
Compatibility and correctness coverage
- Preserve the ledger entry format byte-for-byte; this must not require a compatibility-version change.
- Verify public-only, private-only, mixed-domain, and empty-private-domain transactions.
- Verify snapshots and historical-secret serialization/decryption.
- Verify ledger replay and recovery against entries produced before and after the change.
- Ensure encryption failure does not expose or replicate a partially constructed entry.
- Preserve the existing rule that public-domain bytes are GCM additional authenticated data.
- Retain vector-based public APIs where required, implemented via the new output-span primitive if practical.
Measurement
Add focused crypto/serialization benchmarks for small transaction payloads and compare allocation counts, bytes copied, and throughput. Run an A/B pi_basic benchmark/profile independently from #8168 so context-reuse and reduced-copy gains remain attributable.
Background
A
perfprofile of the 3,000,000-requestpi_basicworkload attributes approximately 24.06% inclusive CPU toCommittableTx::serialise(). Across the full profile, memory copy/set leaves account for 5.06% and allocation/refcount leaves for 16.25%; within transaction serialization alone these categories account for approximately 1.08% and 2.02% of total CPU respectively.The current private-domain serialization path creates and copies through several buffers:
GenericSerialiseWrapperbuilds separate public and private writer buffers.serialise_domains()allocates the final ledger entry.serialised_hdrvector and anencrypted_private_domainvector.KeyAesGcm_OpenSSL::encrypt()allocates another local ciphertext vector, fills it, and move-assigns it over the caller-provided vector.This work is separate from reusing initialised AES-GCM contexts (#8168) and should be implemented/measured independently.
Proposed implementation
ciphertextallocation inKeyAesGcm_OpenSSL::encrypt()when the caller already provides correctly sized output storage.Compatibility and correctness coverage
Measurement
Add focused crypto/serialization benchmarks for small transaction payloads and compare allocation counts, bytes copied, and throughput. Run an A/B
pi_basicbenchmark/profile independently from #8168 so context-reuse and reduced-copy gains remain attributable.