Add a Memory Usage section to SHA2/SHA3/HashDRBG docs

alpha_0.1.2_release_notes.md

@@ -6,35 +6,12 @@
     * Check the crate release checklist and run claude against the style guide (maybe Francis could cross-check me)
     * Run Crucible testing
     * Add factories for ML-DSA and ML-KEM (if we are keeping factories, see below)
-    * After merging the Signer/Verifier, Encrypter/Decrypter split, check if the keygen_from_rng() is still on the right
-      trait.
-* Split the Signature trait into a Signer and a Verifier so that, for example, we can implement the verifier for MTC in
-  a different struct from the signer; or so that you can get FIPS compliance on old algorithms that are currently only
-  FIPS-allowed for verification of existing signatures but not for creation of new ones.
-* Check out Megan's email May 13 about KeyMaterial: "I was wondering if there might be scope for a closure based
-  approach that could guarantee encapsulation of the state change from safe to hazardous back to safe again."
-* Go back to previous algs and apply memory optimization tricks like internal functions. And add a docs section "Memory
-  Usage" that measures with valgrind.
 * Ensure that all crates have `#![forbid(missing_docs)]`
 * Apply Secret trait consistently across the library --> study the `Zeroize` trait in RustCrypto
-* Change all "[u8;0]" to "[]" throughout the code and docs ... or better yet, change the APIs to take an Option<>
-* Change all `-> Vec<u8>` to `-> [u8; CONST_LEN]`, and the `output: &mut [u8]` to `output: &mut [u8; CONST_LEN]` where
-  appropriate.
-* Probably it makes sense to leave Hex and Base64 as requiring std; ... or maybe add a no_std version that uses
-  fixed-sized blocks?
-* Make this build on the stable compiler. IE Remove the rust-toolchain.toml file that builds with nightly. Will require
-  some refactoring.
-* Create a cargo feature #[cfg(feature='rng')] and put it around things like keygen that takes an rng so that the build
-  dependency on bouncycastle_rng is optional.
-* Factories ... Are they worth it? Michael Richardson says Very Yes. If we are keeping them, then we need a serious
-  re-engineering of them because I really dislike that currently they make it hard for the underlying primitive to have
-  static one-shot APIs.
 * Deal with as many of the inline TODOs as possible
 * Close all open github issues and document them in this file.
 * After everything is merged, circle back to crucible, and make sure that the harness still works (and maybe remove the
   nightly build toolchain)
-* Search for all the uses of .unwrap() in non-test code and replace each with either a comment or an expect with a
-  meaningful error string.
 
 # 0.1.2 Features / Changelog
 
@@ -46,7 +23,7 @@
 * All public `*_out(.., out: &mut [u8])` functions now begin by zeroizing the entire output buffer with `.fill(0)`,
   preventing exposure of stale data in oversized output buffers or on early error returns.
 * Reworked the way KeyMaterial hazardous operations work; instead of a stateful .allow_hazardous_operations() /
-  .drop_hazardous_operations(), it now uses a closure-based do_hazardous_operations(). Github issue #39.
+  .drop_hazardous_operations(), it now uses a closure-based do_hazardous_operations(). Github issue #39.me
 * Renamed KeyMaterial::KeyType's and deleted KeyMaterial::concatenate in order to give a better intuition and
   FIPS-alignment.
 * Github issues resolved:

crypto/rng/src/lib.rs

@@ -26,6 +26,15 @@
 //! This crate contains the [Sp80090ADrbg] trait, which is intentionally defined here and not in [bouncycastle_core::traits]
 //! since misuse of [Sp80090ADrbg::instantiate] can completely undermine the security of your entire
 //! cryptographic application.
+//!
+//! # Memory Footprint
+//!
+//! The following table lists the size of the struct in memory when using the stateful API between multiple calls to `.do_update()`.
+//!
+//! | Key Object | Struct size in memory (bytes) |
+//! |------------|-------------------------------|
+//! | HashDRBG_SHA256   | 144                    |
+//! | HashDRBG_SHA512   | 144                    |
 
 #![forbid(unsafe_code)]
 

crypto/sha2/src/lib.rs

@@ -33,6 +33,17 @@
 //!
 //! let output: Vec<u8> = sha2.do_final();
 //! ```
+//!
+//! # Memory Footprint
+//!
+//! The following table lists the size of the struct in memory when using the stateful API between multiple calls to `.do_update()`.
+//!
+//! | Key Object | Struct size in memory (bytes) |
+//! |------------|-------------------------------|
+//! | SHA224     | 112                           |
+//! | SHA256     | 112                           |
+//! | SHA384     | 208                           |
+//! | SHA512     | 208                           |
 
 #![forbid(unsafe_code)]
 #![allow(private_bounds)]

crypto/sha3/src/lib.rs

@@ -103,6 +103,19 @@
 //! This would also be the case even if the input had type
 //! [KeyType::CryptographicRandom] since the input [KeyMaterial] is 16 bytes but [SHA3_256] needs at least 32 bytes of
 //! full-entropy input key material in order to be able to produce full entropy output key material.
+//!
+//! # Memory Footprint
+//!
+//! The following table lists the size of the struct in memory when using the stateful API between multiple calls to `.do_update()`.
+//!
+//! | Key Object | Struct size in memory (bytes) |
+//! |------------|-------------------------------|
+//! | SHA3_224   | 440                           |
+//! | SHA3_256   | 440                           |
+//! | SHA3_384   | 440                           |
+//! | SHA3_512   | 440                           |
+//! | SHAKE128   | 440                           |
+//! | SHAKE256   | 440                           |
 
 #![forbid(unsafe_code)]
 #![allow(private_bounds)]

mem_usage_benches/Cargo.toml

@@ -13,4 +13,16 @@ path = "bench_mldsa_mem_usage.rs"
 
 [[bin]]
 name = "bench_mlkem_mem_usage"
-path = "bench_mlkem_mem_usage.rs"
+path = "bench_mlkem_mem_usage.rs"
+
+[[bin]]
+name = "bench_sha2_mem_usage"
+path = "bench_sha2_mem_usage.rs"
+
+[[bin]]
+name = "bench_sha3_mem_usage"
+path = "bench_sha3_mem_usage.rs"
+
+[[bin]]
+name = "bench_hashdrbg_mem_usage"
+path = "bench_hashdrbg_mem_usage.rs"

mem_usage_benches/bench_hashdrbg_mem_usage.rs

@@ -0,0 +1,83 @@
+//! The purpose of this binary is to perform a single run of the primitive under test so that
+//! its peak memory usage can be measured with:
+//!
+//!     valgrind --tool=massif --heap=no --stacks=yes -- target/release/bench_hashdrbg_mem_usage > /dev/null
+//!
+//!     ms_print massif.out.835000
+//!
+//! or, shoved all into one line:
+//!
+//!     clear; clear; valgrind --tool=massif --heap=no --stacks=yes -- target/release/bench_hashdrbg_mem_usage > /dev/null; ms_print massif.out.*; rm massif.out.*
+//!
+//! Make sure you build in release mode!
+//!
+//! Note: I'm using print!() to force the compiler not to optimize away the actual code.
+//! I'm printing the important stuff for benchmarking to stderr so that I can pipe the junk to /dev/null
+//! (I'm not doing it the other way because /usr/bin/time prints its useful stuff to stderr as well)
+//!
+//! This exercises the HashDRBG (SP 800-90A) implementation from the `rng` crate. We seed from
+//! fixed dummy entropy (NOT from the OS) so the run is deterministic and reproducible under massif.
+//!
+//! Main is at the bottom, controls which this was actually run.
+
+#![allow(dead_code)]
+#![allow(unused_imports)]
+
+use bouncycastle::core::key_material::{KeyMaterial0, KeyMaterial256, KeyMaterial512, KeyType};
+use bouncycastle::core::traits::{RNG, SecurityStrength};
+use bouncycastle::rng::{HashDRBG_SHA256, HashDRBG_SHA512, Sp80090ADrbg};
+
+/// Fixed dummy entropy for seeding. HashDRBG-SHA256 needs 32 bytes (128-bit strength);
+/// HashDRBG-SHA512 needs 64 bytes (256-bit strength). We take the prefix we need.
+const DUMMY_SEED: [u8; 64] = [
+    0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F,
+    0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F,
+    0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x2A, 0x2B, 0x2C, 0x2D, 0x2E, 0x2F,
+    0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x3A, 0x3B, 0x3C, 0x3D, 0x3E, 0x3F,
+];
+
+/// This exists so I can use /usr/bin/time to measure the base memory footprint of the harness.
+fn bench_do_nothing() {
+    eprintln!("DoNothing");
+
+    print!("{}", 1 + 1);
+}
+
+/// This prints the in-memory size of the DRBG state structs.
+fn print_struct_sizes() {
+    use core::mem::size_of;
+
+    println!("size_of<HashDRBG_SHA256>: {}", size_of::<HashDRBG_SHA256>());
+    println!("size_of<HashDRBG_SHA512>: {}", size_of::<HashDRBG_SHA512>());
+}
+
+fn bench_hashdrbg_sha256_generate() {
+    eprintln!("HashDRBG-SHA256/Generate");
+
+    let seed = KeyMaterial256::from_bytes_as_type(&DUMMY_SEED[..32], KeyType::Seed).unwrap();
+    let mut rng = HashDRBG_SHA256::new_unititialized();
+    rng.instantiate(false, seed, &KeyMaterial0::new(), &[], SecurityStrength::_128bit).unwrap();
+
+    let mut out = [0u8; 32];
+    rng.generate_out(&[], &mut out).unwrap();
+    print!("{:x?}", out);
+}
+
+fn bench_hashdrbg_sha512_generate() {
+    eprintln!("HashDRBG-SHA512/Generate");
+
+    let seed = KeyMaterial512::from_bytes_as_type(&DUMMY_SEED, KeyType::Seed).unwrap();
+    let mut rng = HashDRBG_SHA512::new_unititialized();
+    rng.instantiate(false, seed, &KeyMaterial0::new(), &[], SecurityStrength::_256bit).unwrap();
+
+    let mut out = [0u8; 32];
+    rng.generate_out(&[], &mut out).unwrap();
+    print!("{:x?}", out);
+}
+
+fn main() {
+    // bench_do_nothing()
+    print_struct_sizes()
+    // bench_hashdrbg_sha256_generate()
+    // bench_hashdrbg_sha512_generate()
+}

mem_usage_benches/bench_sha2_mem_usage.rs

@@ -0,0 +1,141 @@
+//! The purpose of this binary is to perform a single run of the primitive under test so that
+//! its peak memory usage can be measured with:
+//!
+//!     valgrind --tool=massif --heap=no --stacks=yes -- target/release/bench_sha2_mem_usage > /dev/null
+//!
+//!     ms_print massif.out.835000
+//!
+//! or, shoved all into one line:
+//!
+//!     clear; clear; valgrind --tool=massif --heap=no --stacks=yes -- target/release/bench_sha2_mem_usage > /dev/null; ms_print massif.out.*; rm massif.out.*
+//!
+//! Make sure you build in release mode!
+//!
+//! Note: I'm using print!() to force the compiler not to optimize away the actual code.
+//! I'm printing the important stuff for benchmarking to stderr so that I can pipe the junk to /dev/null
+//! (I'm not doing it the other way because /usr/bin/time prints its useful stuff to stderr as well)
+//!
+//! Main is at the bottom, controls which this was actually run.
+
+#![allow(dead_code)]
+#![allow(unused_imports)]
+
+use bouncycastle::core::traits::Hash;
+use bouncycastle::sha2::{SHA224, SHA256, SHA384, SHA512};
+
+/// A fixed input larger than one block for every SHA-2 variant, so a single hash call
+/// exercises multiple compression rounds.
+const INPUT: [u8; 1024] = [0xAB; 1024];
+
+/// Number of `do_update` calls in the streaming benches (total absorbed = 16 KiB).
+const STREAMING_ITERS: usize = 16;
+
+/// This exists so I can use /usr/bin/time to measure the base memory footprint of the harness.
+fn bench_do_nothing() {
+    eprintln!("DoNothing");
+
+    print!("{}", 1 + 1);
+}
+
+/// This prints the in-memory size of all the SHA-2 state structs.
+fn print_struct_sizes() {
+    use core::mem::size_of;
+
+    println!("size_of<SHA224>: {}", size_of::<SHA224>());
+    println!("size_of<SHA256>: {}", size_of::<SHA256>());
+    println!("size_of<SHA384>: {}", size_of::<SHA384>());
+    println!("size_of<SHA512>: {}", size_of::<SHA512>());
+}
+
+fn bench_sha224_oneshot() {
+    eprintln!("SHA224/OneShot");
+
+    let mut digest = [0u8; 28];
+    SHA224::new().hash_out(&INPUT, &mut digest);
+    print!("{:x?}", digest);
+}
+
+fn bench_sha224_streaming() {
+    eprintln!("SHA224/Streaming");
+
+    let mut digest = [0u8; 28];
+    let mut md = SHA224::new();
+    for _ in 0..STREAMING_ITERS {
+        md.do_update(&INPUT);
+    }
+    md.do_final_out(&mut digest);
+    print!("{:x?}", digest);
+}
+
+fn bench_sha256_oneshot() {
+    eprintln!("SHA256/OneShot");
+
+    let mut digest = [0u8; 32];
+    SHA256::new().hash_out(&INPUT, &mut digest);
+    print!("{:x?}", digest);
+}
+
+fn bench_sha256_streaming() {
+    eprintln!("SHA256/Streaming");
+
+    let mut digest = [0u8; 32];
+    let mut md = SHA256::new();
+    for _ in 0..STREAMING_ITERS {
+        md.do_update(&INPUT);
+    }
+    md.do_final_out(&mut digest);
+    print!("{:x?}", digest);
+}
+
+fn bench_sha384_oneshot() {
+    eprintln!("SHA384/OneShot");
+
+    let mut digest = [0u8; 48];
+    SHA384::new().hash_out(&INPUT, &mut digest);
+    print!("{:x?}", digest);
+}
+
+fn bench_sha384_streaming() {
+    eprintln!("SHA384/Streaming");
+
+    let mut digest = [0u8; 48];
+    let mut md = SHA384::new();
+    for _ in 0..STREAMING_ITERS {
+        md.do_update(&INPUT);
+    }
+    md.do_final_out(&mut digest);
+    print!("{:x?}", digest);
+}
+
+fn bench_sha512_oneshot() {
+    eprintln!("SHA512/OneShot");
+
+    let mut digest = [0u8; 64];
+    SHA512::new().hash_out(&INPUT, &mut digest);
+    print!("{:x?}", digest);
+}
+
+fn bench_sha512_streaming() {
+    eprintln!("SHA512/Streaming");
+
+    let mut digest = [0u8; 64];
+    let mut md = SHA512::new();
+    for _ in 0..STREAMING_ITERS {
+        md.do_update(&INPUT);
+    }
+    md.do_final_out(&mut digest);
+    print!("{:x?}", digest);
+}
+
+fn main() {
+    // print_struct_sizes()
+    // bench_do_nothing()
+    // bench_sha224_oneshot()
+    // bench_sha224_streaming()
+    // bench_sha256_oneshot()
+    // bench_sha256_streaming()
+    // bench_sha384_oneshot()
+    // bench_sha384_streaming()
+    bench_sha512_oneshot()
+    // bench_sha512_streaming()
+}