perf(alp-rd): optimise RDEncoder dictionary search and split()

[jrmoynihan]

Summary

Four targeted improvements to the ALP-RD hot path, all contained in src/alp_rd/mod.rs. No public API changes — RDEncoder::new(), RDEncoder::split(), Split::decode(), and Split::into_parts() all preserve their existing signatures.

Direct-addressed frequency array in build_left_parts_dictionary

Replace HashMap<u16, usize> with a &mut [u32; 65536] passed in from the caller. Left-bit patterns are u16 values (0–65535), so the value itself is a valid array index — no hashing, no collision, O(1) insert. The buffer is allocated once per find_best_dictionary call and reset with fill(0) per cut-point iteration, avoiding 15 redundant heap alloc/free cycles across the 16-iteration search.

Input subsampling in find_best_dictionary

When sample.len() > MAX_SAMPLE (4096), stride through the input with step_by(). Dictionary search then costs O(MAX_SAMPLE × 16) rather than O(N × 16). The dominant left-bit patterns are stable across large inputs; 4096 samples is sufficient to identify the top-8 dictionary entries with negligible quality loss.

Reverse lookup table in RDEncoder

Pre-compute a Box<[u8; 65536]> in new(): left_raw_u16 → code + 1 (0 = not in dictionary). The +1 sentinel avoids a separate Option, keeping the table at 64 KB (fits in L2 cache). This replaces the codes.iter().position() O(dict_size) linear scan called once per element in split().

Single-pass split() and inline dict in Split

Fuse the two loops in split() into one: compute bits once per element, extract left_raw, do a single table lookup, push code or exception in the same iteration.

Change Split.left_dict from Vec<u16> to [u16; 8] + left_dict_len: u8. Eliminates one Vec heap allocation per split() call (one per 1024-element chunk). into_parts() materialises a Vec<u16> on demand; decode() uses the inline slice directly.

Benchmark results

Measured on 30 M f32 values with a log-normal distribution (high dynamic range, many distinct left-bit patterns — a stress case for the dictionary search):

Operation	Before	After	Speedup
RDEncoder::new()	~3.2 s	~720 us	~4500x
split() encode (30 M values)	~187 ms	~66 ms	~2.8x
Full encode (new + split)	~3.4 s	~67 ms	~51x
Decode	~20 ms	~22 ms	unchanged

Test plan

• cargo test passes (all 4 existing tests)
• cargo clippy --all-targets clean on nightly-2025-02-24
• Roundtrip correctness verified for f32 and f64

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}