perf: bound watcher hashing memory with reusable scratch

[Mohamed Mansour (mohamedmansour)]

Summary

Reuse one 8 KiB content buffer per watcher instead of allocating a whole-file buffer for each hash. Retain the existing standard-hasher digest, unchanged-save suppression, and retry-after-error behavior.

Regular-file metadata is checked before and after opening. Incremental reads accept files up to 8 MiB and probe one additional byte to detect growth after metadata. Short/interrupted reads are handled; failures and oversized input do not cache partial hashes and still trigger normal invalidation.

Add focused regression coverage, a five-case Criterion benchmark, benchmark-registry coverage, and specification/benchmark documentation. Criterion is an existing workspace dev dependency; no production dependency, public API, CLI processing, streaming writer, or pool behavior changes are included.

This is a bounded-memory improvement with mixed latency results, not a claim that all watcher workloads become faster. The 1 MiB file and small-file burst regressions are retained below.

Source-derived allocation/storage change

These figures follow the implementation and are not measured process-RSS deltas.

Property	Before	After
Normal nonempty-file content storage	One file-sized Vec allocation per hash	No per-file content-buffer heap allocation
Content storage at 8 MiB cap	8 MiB	8 KiB reused buffer: 1024 times smaller
32 x 256 KiB burst	32 content allocations; 8 MiB cumulative content allocation	Reuse the same 8 KiB scratch
Live content buffer between events	None	8 KiB retained per watcher

Path, event, and hash-map allocations are unchanged. An allocator can retain freed buffers, so source-level storage and cumulative allocation must not be equated with RSS.

Recorded release benchmarks

This PR is based on newer main 2dab6fc6; these timings were not rerun against that newer base. The intervening main commits did not change watcher hashing. The original complete evidence remains associated with #527.

Method: release Criterion, 50 samples, 1 second warmup, 3 second target measurement, hot page cache. File opening and metadata are timed; fixture creation is not. Shared Intel Core Ultra 9 285K / Linux x86_64 WSL2 environment, Rust/Cargo 1.98.0. Host load/frequency/cache variation limits interpretation; these are not end-to-end rebuild measurements.

Values are extracted median/P50 estimates and median 95% confidence intervals, in microseconds. They are not Criterion's printed regression/mean estimates.

Case	Before median us [95% CI]	After median us [95% CI]	Median change
1 KiB file	2.8852 [2.7815, 3.0209]	2.7888 [2.7253, 2.9230]	-3.34%
1 MiB file	189.3235 [187.3793, 194.0550]	215.5880 [209.4294, 223.4490]	+13.87%
8 MiB file	3261.7713 [2892.7735, 3534.1622]	1992.3002 [1961.0281, 2176.1765]	-38.92%
32 x 1 KiB burst	101.7087 [99.9075, 109.5873]	122.4937 [113.3522, 131.4069]	+20.44%
32 x 256 KiB burst	2327.7129 [2198.5900, 2433.0210]	1983.9213 [1936.6086, 2088.3039]	-14.77%

Additional bounded read calls are a real potential latency/CPU cost. The justification is predictable content-buffer memory and removal of repeated file-sized allocations, with the mixed timing tradeoff explicitly accepted for this dev-server path. No allocator-count trace or process-RSS improvement is claimed.

Reproduction

Run the same newly added harness against the original and changed hashing implementation:

cargo bench -p microsoft-webui-dev-server --bench watch_hash_bench -- \
  --sample-size 50 --warm-up-time 1 --measurement-time 3 --save-baseline before
cargo bench -p microsoft-webui-dev-server --bench watch_hash_bench -- \
  --sample-size 50 --warm-up-time 1 --measurement-time 3 --baseline before

Read each Criterion case's median.point_estimate and median.confidence_interval from estimates.json, converting nanoseconds to microseconds. cargo xtask bench all now includes this harness.

Coverage and validation

Tests cover digest equivalence across varied short-read boundaries and repeated interruptions, empty/multiple-buffer/exact-cap input, bounded overflow reads, deterministic growth after metadata, failures before/after partial reads, buffer reuse after failure, missing/non-regular files, and retry/recreation behavior.

cargo test -p microsoft-webui-dev-server, the focused xtask benchmark-registry selection, and complete cargo xtask check passed on this independent branch. The full gate completed in 77.4 seconds, including benchmark smoke, docs, and workspace/WASM/example builds.

No streaming hard limit from #527 is included or required. This PR can merge independently of the CLI improvement in #529.