feat(tools/e2e): add limits harness Shipper fan-in axis + instrument proof (#382)

[Minipada]

Summary

• Adds tools/e2e/scripts/run_limits_shipper_fanin.py, the orchestrator that wires Limits harness: measure where the pipeline saturates, on four axes #323's four already-shipped deep modules together for real: ramp_controller.find_knee() drives synthetic connection count (via load_driver.py) against Limits harness: two-tier topology composer (aggregating Shipper stack) #381's two-tier topology's aggregating Shipper, saturation_probe.evaluate() judges each level from a window of ack-latency/unacked-window-depth/disk-buffer observations, and curve_reporter turns the result into a stable report plus the PRD's required closing sentence. None of the four modules are modified.
• Zero loss is asserted at every sustainable level, not once at the end, by hooking into the ramp controller's own driver/probe seam: only when a level's verdict stays clear does the probe also run verify_zero_loss.py ("the existing verifier," reused unmodified) against a snapshot of the ledger-checked real stack's still-running output. A violation hard-fails the whole ramp immediately.
• Adds tools/e2e/scripts/run_limits_shipper_fanin.sh, a sibling scenario script (own topology bring-up, own gates) reusing Limits harness: two-tier topology composer (aggregating Shipper stack) #381's topology files verbatim. It runs the ramp twice — unconstrained (the flagship number), then against a deliberately CPU/memory-constrained aggregating Shipper — and hard-fails unless the constrained ceiling is substantially lower (<= 0.5x by default), proving the harness detects an induced limit per the PRD.

Verified against real infrastructure

Ran the full scenario against a real dc-e2e image on real podman infrastructure multiple times (not just unit-level reasoning against fakes):

• A coarse-ramp run correctly failed the instrument-proof gate when both phases happened to saturate at the same tested level — the harness refused to fabricate a pass.
• A finer-grained run drove real synthetic load through a real aggregating Shipper, found a real saturation knee, and asserted real zero-loss at clear levels.
• Found and root-caused a real bug in the process: the live-volume snapshot verify_zero_loss_at_level() reads raced dc_mcap_writer's 15s rotation window (entrypoint.sh --max-duration-secs 15), surfacing as a false "ZERO-LOSS VERIFICATION FAILED" for Records that were actually safe on disk but not yet in a finished, readable .mcap file. Fixed the retry to cover any failure shape (not just parse crashes) with an 18s pause, and re-verified the fix against the same live containers that had failed under the original logic.

Full details in progress.txt's #382 entry.

Closes #382

Test plan

• prek run --all-files --skip build-doc passes (ruff, shellcheck, codespell, clang-format, etc.)
• python3 -m py_compile / CLI --help smoke test on the orchestrator
• In-process smoke test against a fake ingest server exercising the full ramp/probe/zero-loss-hook/curve-report/closing-sentence flow
• Multiple real end-to-end runs against a real dc-e2e image + real podman containers (Postgres/RustFS/aggregating Shipper/real DC stack), including a run that correctly failed the instrument-proof gate and a run that found a real saturation knee with real zero-loss assertions
• A full unattended pass of both phases end-to-end in one run, outside this sandbox's per-command time constraints — not run by ci.yaml, same as every other narrow scenario script in this harness

🤖 Generated with Claude Code

https://claude.ai/code/session_01Mh8vrBQibc4B5JqJn5f45Z