feat(tools/e2e): add limits harness one-command capacity baseline (#386)

Adds scripts/run_limits_capacity_baseline.sh, the final piece of #323's
limits-harness epic: sequences the four axis scripts (Shipper fan-in,
single-robot ceiling, Uploader concurrency, drain rate) and combines their
curve_reporter.py reports into one stable/diffable artifact via the new
scripts/limits_baseline.py, so a contributor can check the effect of a
change with one command and a release can record a capacity baseline.

curve_reporter.py gains from_json() (the inverse of to_json(), to read a
per-axis report back in) and a generic closing_sentence() so every axis
gets #323's PRD "single defensible sentence" — composition and
measured/extrapolated conditions stated — without a bespoke
implementation per axis; run_limits_shipper_fanin.py keeps its own
existing wording for the PRD's literal flagship example.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JsH1x5McUTKyuXScdaf7ZN
Signed-off-by: David Bensoussan <d.bensoussan@proton.me>

Author: Minipada

tools/e2e/README.md

@@ -569,6 +569,47 @@ testing decision.
 
 Not wired into `ci.yaml`, same as the scenarios above.
 
+## One-command capacity baseline (#386)
+
+The final piece of #323's epic: a single entry point that runs all four axes above and
+combines their reports into one artifact, so a contributor can check the effect of a
+change with one command instead of assembling four scenarios by hand, and a release can
+record a capacity baseline instead of repeating the previous release's claim.
+
+```sh
+./tools/e2e/scripts/run_limits_capacity_baseline.sh
+```
+
+Runs `run_limits_shipper_fanin.sh`, `run_limits_single_robot_ceiling.sh`,
+`run_limits_upload_concurrency.sh`, and `run_limits_drain_rate.sh` in sequence — each is
+its own full podman bring-up/teardown, and none of the four are meant to run
+concurrently, same as every other pair of sibling scenario scripts in this harness —
+then hands the four already-written `curve_reporter.py` report JSON files (only the
+Shipper fan-in axis's **unconstrained** phase; its constrained phase is that axis's own
+internal instrument proof, not a fifth axis here) to `scripts/limits_baseline.py`.
+`merge_reports()` combines them into one `CurveReport` sorted by axis name — the same
+stability guarantee `curve_reporter.build_report()` gives within one report, extended
+across reports, so the combined artifact is diffable between runs of an unchanged
+system exactly as each axis's own report already is. `render_baseline()` then renders
+`curve_reporter.render_summary()`'s per-axis detail followed by every axis's one-sentence
+capacity claim from the new `curve_reporter.closing_sentence()` — a generic form of
+`run_limits_shipper_fanin.py`'s own bespoke "single defensible sentence" (which keeps its
+particular wording, since it's the PRD's literal flagship example), built entirely from
+an already-computed `AxisReport` with no axis-specific knowledge, so every axis gets one
+without a fifth bespoke implementation. Both `merge_reports()` and `render_baseline()`
+are pure functions over already-written reports, unit-tested with synthetic data
+(`test_limits_baseline.py`), matching this epic's testing philosophy: this script and the
+four axis scripts it sequences are proven by running them, not by a unit test.
+
+Because `set -euo pipefail` combines with each axis script's own hard-failing gates, any
+axis failing aborts the whole baseline immediately — a combined artifact is only ever
+written from four axes that actually passed. Every env var an individual axis script
+accepts (`DC_E2E_FANIN_*`, `DC_E2E_CEILING_*`, `DC_E2E_UPLOAD_*`, `DC_E2E_DRAIN_*`,
+`DC_E2E_IMAGE`/`DC_WORKSPACE_IMAGE`, `DC_E2E_KEEP`) still applies unchanged, since this
+script does no parameter translation of its own.
+
+Not wired into `ci.yaml`, same as the scenarios above.
+
 ## Layout
 
 - `Containerfile` — builds the full DC workspace (every `dc_*` package, all C++ since
@@ -703,6 +744,13 @@ Not wired into `ci.yaml`, same as the scenarios above.
   `find_drain_rate_curve()` are pure and unit-tested with fakes
   (`test_drain_rate_axis.py`); the podman/load_driver.py orchestration is exercised by
   running the scenario script.
+- `scripts/limits_baseline.py` / `scripts/run_limits_capacity_baseline.sh` — the
+  one-command capacity baseline (#386, the final piece of #323's epic) described above:
+  sequences the four axis scripts, then combines their `curve_reporter.py` reports into
+  one stable artifact with every axis's closing sentence. `merge_reports()`/
+  `render_baseline()` are pure and unit-tested with synthetic reports
+  (`test_limits_baseline.py`); the four axes it sequences are each exercised by running
+  their own scenario script, not re-tested here.
 
 ## `.dockerignore`
 

tools/e2e/scripts/curve_reporter.py

@@ -231,6 +231,49 @@ def to_json(report: CurveReport) -> str:
     return json.dumps(dataclasses.asdict(report), indent=2, sort_keys=True)
 
 
+def _resource_sample_from_dict(d: dict | None) -> ResourceSample | None:
+    return None if d is None else ResourceSample(**d)
+
+
+def _binding_constraint_from_dict(d: dict) -> BindingConstraint:
+    return BindingConstraint(
+        resource=None if d["resource"] is None else ResourceKind(d["resource"]),
+        percent=d["percent"],
+        reason=d["reason"],
+    )
+
+
+def _level_result_from_dict(d: dict) -> LevelResult:
+    return LevelResult(
+        level=d["level"],
+        saturated=d["saturated"],
+        kind=PointKind(d["kind"]),
+        composition=RunComposition(**d["composition"]),
+        resources=_resource_sample_from_dict(d["resources"]),
+    )
+
+
+def axis_report_from_dict(d: dict) -> AxisReport:
+    """The inverse of `dataclasses.asdict()` for one axis, used to read back a
+    per-axis report a scenario script already wrote via `to_json()` — the shape #386's
+    one-command entry point combines across axes."""
+    return AxisReport(
+        axis=d["axis"],
+        unit=d["unit"],
+        outcome=RunOutcome(d["outcome"]),
+        highest_clear_level=d["highest_clear_level"],
+        tripped_level=d["tripped_level"],
+        binding_constraint=_binding_constraint_from_dict(d["binding_constraint"]),
+        points=tuple(_level_result_from_dict(p) for p in d["points"]),
+    )
+
+
+def from_json(text: str) -> CurveReport:
+    """The inverse of `to_json()`."""
+    data = json.loads(text)
+    return CurveReport(axes=tuple(axis_report_from_dict(a) for a in data["axes"]))
+
+
 def render_summary(report: CurveReport) -> str:
     """A human-readable summary naming, per axis, the outcome and the binding
     constraint at saturation, with every point labelled measured or extrapolated and
@@ -271,3 +314,60 @@ def render_summary(report: CurveReport) -> str:
         lines.append("")
 
     return "\n".join(lines).rstrip() + "\n"
+
+
+def _point_at(axis: AxisReport, level: float) -> LevelResult:
+    for point in axis.points:
+        if point.level == level:
+            return point
+    raise ValueError(f"axis {axis.axis!r} has no point at level {level}")
+
+
+def closing_sentence(axis: AxisReport) -> str:
+    """#323's PRD's single defensible sentence, generalised across axes: 'sustains N
+    <unit> before applying backpressure, with its composition and measured/extrapolated
+    conditions stated' — never a fabricated ceiling. `run_limits_shipper_fanin.py`
+    carries its own bespoke wording for the PRD's literal flagship example; this is the
+    generic form #386's one-command entry point uses for every axis, built entirely from
+    the already-computed `AxisReport`, with no axis-specific knowledge."""
+    unit = axis.unit
+    if axis.outcome == RunOutcome.BOUND_NOT_FOUND:
+        highest = axis.highest_clear_level
+        if highest is None:
+            return (
+                f"{axis.axis}: bound not found — no level was tested without saturating, "
+                f"so no ceiling is reported (never fabricated)."
+            )
+        point = _point_at(axis, highest)
+        comp = point.composition
+        return (
+            f"{axis.axis} sustains at least {highest} {unit} ({comp.real_stacks} real + "
+            f"{comp.synthetic_senders} synthetic, {point.kind.value}) without applying "
+            f"backpressure — the ramp never saturated within the tested range, so no "
+            f"ceiling is reported beyond {highest} {unit} (bound not found; never "
+            f"fabricated)."
+        )
+
+    tripped = axis.tripped_level
+    assert tripped is not None  # KNEE_FOUND always carries a tripped_level
+    tripped_point = _point_at(axis, tripped)
+    extrapolated_note = (
+        "not extrapolated — the knee itself was measured, not projected"
+        if tripped_point.kind == PointKind.MEASURED
+        else "extrapolated beyond what was directly measured"
+    )
+    highest = axis.highest_clear_level
+    if highest is None:
+        return (
+            f"{axis.axis} saturated at the first level tested, {tripped} {unit} "
+            f"({tripped_point.composition.real_stacks} real + "
+            f"{tripped_point.composition.synthetic_senders} synthetic); no lower level "
+            f"was sustained ({extrapolated_note})."
+        )
+    sustain_point = _point_at(axis, highest)
+    comp = sustain_point.composition
+    return (
+        f"{axis.axis} sustains {highest} {unit} ({comp.real_stacks} real + "
+        f"{comp.synthetic_senders} synthetic, {sustain_point.kind.value}) before applying "
+        f"backpressure; saturation observed at {tripped} {unit} ({extrapolated_note})."
+    )

tools/e2e/scripts/limits_baseline.py

@@ -0,0 +1,85 @@
+# SPDX-FileCopyrightText: 2022-2026 David Bensoussan
+# SPDX-License-Identifier: MPL-2.0
+
+"""Limits harness: one-command capacity baseline (#386, part of #323's epic).
+
+Combines the four axes' already-written `curve_reporter.CurveReport` JSON files
+(Shipper fan-in, single-robot ceiling, Uploader concurrency, drain rate — each written
+by its own axis script via `curve_reporter.to_json()`) into one combined, diffable
+report plus a human-readable summary carrying every axis's closing sentence
+(`curve_reporter.closing_sentence()`). `merge_reports()`/`render_baseline()` are pure
+functions of the per-axis reports already on disk — no containers, no ramping, no
+axis-specific knowledge — so they're unit-tested directly against synthetic
+`CurveReport`s, matching this epic's other reporting code
+(`curve_reporter.build_report()`'s own testing precedent). Reading the four report files
+and writing the combined ones is the only I/O here; `run_limits_capacity_baseline.sh`
+is what actually runs the four axes first.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from collections.abc import Sequence
+from pathlib import Path
+
+from curve_reporter import CurveReport, closing_sentence, from_json, render_summary, to_json
+
+
+def merge_reports(reports: Sequence[CurveReport]) -> CurveReport:
+    """One combined report from N single-axis (or multi-axis) reports. Axis order is
+    resolved by name alone, independent of the order the reports were handed in or of
+    each axis's own internal ordering — the same stability guarantee
+    `curve_reporter.build_report()` gives within one report, extended across reports."""
+    axes = tuple(sorted((axis for report in reports for axis in report.axes), key=lambda a: a.axis))
+    names = [axis.axis for axis in axes]
+    if len(names) != len(set(names)):
+        duplicates = sorted({name for name in names if names.count(name) > 1})
+        raise ValueError(f"duplicate axis name(s) across the reports being combined: {duplicates}")
+    return CurveReport(axes=axes)
+
+
+def render_baseline(report: CurveReport) -> str:
+    """The combined artifact's human-readable form: `curve_reporter.render_summary()`'s
+    per-axis detail, followed by every axis's one-sentence capacity claim in one place —
+    #386's acceptance criterion that each axis's closing sentence, with its composition
+    and measured/extrapolated conditions, is present in the combined output."""
+    lines = [render_summary(report).rstrip(), "", "Closing statements:", ""]
+    for axis in report.axes:
+        lines.append(f"- {closing_sentence(axis)}")
+    return "\n".join(lines).rstrip() + "\n"
+
+
+def _parse_args(argv: list[str]) -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument(
+        "--report",
+        dest="reports",
+        action="append",
+        required=True,
+        metavar="PATH",
+        help="a per-axis curve_reporter report.json; repeat once per axis",
+    )
+    parser.add_argument("--output-json", required=True)
+    parser.add_argument("--output-txt", required=True)
+    return parser.parse_args(argv)
+
+
+def main(argv: list[str] | None = None) -> int:
+    args = _parse_args(argv if argv is not None else sys.argv[1:])
+
+    reports = [from_json(Path(p).read_text()) for p in args.reports]
+    combined = merge_reports(reports)
+
+    Path(args.output_json).write_text(to_json(combined))
+    summary = render_baseline(combined)
+    Path(args.output_txt).write_text(summary)
+    print(summary)
+
+    print(json.dumps({"axes": [axis.axis for axis in combined.axes]}))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tools/e2e/scripts/run_limits_capacity_baseline.sh

@@ -0,0 +1,73 @@
+#!/bin/bash
+# SPDX-FileCopyrightText: 2022-2026 David Bensoussan
+# SPDX-License-Identifier: MPL-2.0
+
+# Limits harness: one-command capacity baseline (#386, the final piece of #323's epic).
+# From the repo root:
+#
+#   ./tools/e2e/scripts/run_limits_capacity_baseline.sh
+#
+# Runs all four limits-harness axes — Shipper fan-in (#382), single-robot ceiling
+# (#383), Uploader concurrency (#384), drain rate (#385) — one after another (each is
+# its own full podman bring-up/teardown, and no two of them are meant to run
+# concurrently, same as every other pair of sibling scenario scripts in this harness),
+# then combines the four already-written curve_reporter.py reports into one artifact via
+# scripts/limits_baseline.py: a single combined_report.json (stable/diffable — a sort by
+# axis name on top of each axis's own stable report, nothing more) and a
+# combined_summary.txt carrying every axis's closing sentence
+# (curve_reporter.closing_sentence()) alongside its per-axis detail. That's the artifact
+# a contributor diffs to check the effect of a change, and the one a release records as
+# its capacity baseline instead of repeating the previous release's claim.
+#
+# Only the Shipper fan-in axis's UNCONSTRAINED phase is folded into the combined
+# report — its own deliberately-constrained second phase is that axis's internal
+# instrument proof (#382's own gate, run and asserted here as part of running that
+# axis), not a fifth axis of this baseline.
+#
+# Each axis script already fails loudly and aborts on its own gate (a hard-fail process
+# exit, not a skip) — combined with this script's own `set -euo pipefail`, any axis
+# failing aborts the whole baseline immediately: a combined artifact is only ever
+# written from four axes that actually passed.
+#
+# Every env var each individual axis script accepts (DC_E2E_FANIN_*, DC_E2E_CEILING_*,
+# DC_E2E_UPLOAD_*, DC_E2E_DRAIN_*, DC_E2E_IMAGE / DC_WORKSPACE_IMAGE, DC_E2E_KEEP) still
+# applies here unchanged — this script does no parameter translation of its own, it only
+# sequences the four scripts and combines what they already produce. See each axis
+# script's own header for its env vars.
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+E2E_DIR="$(dirname "$SCRIPT_DIR")"
+RUN_DIR="$E2E_DIR/.run/limits_capacity_baseline"
+
+mkdir -p "$RUN_DIR"
+cd "$E2E_DIR"
+
+log() { echo "[e2e-baseline $(date -u +%H:%M:%S)] $*"; }
+
+log "=== axis 1/4: Shipper fan-in (#382) ==="
+"$SCRIPT_DIR/run_limits_shipper_fanin.sh"
+
+log "=== axis 2/4: single-robot ceiling (#383) ==="
+"$SCRIPT_DIR/run_limits_single_robot_ceiling.sh"
+
+log "=== axis 3/4: Uploader concurrency (#384) ==="
+"$SCRIPT_DIR/run_limits_upload_concurrency.sh"
+
+log "=== axis 4/4: drain rate (#385) ==="
+"$SCRIPT_DIR/run_limits_drain_rate.sh"
+
+log "combining the four axes' reports into one capacity baseline"
+uv run --frozen python3 "$SCRIPT_DIR/limits_baseline.py" \
+  --report "$E2E_DIR/.run/limits_shipper_fanin/unconstrained_report.json" \
+  --report "$E2E_DIR/.run/limits_single_robot_ceiling/curve_report.json" \
+  --report "$E2E_DIR/.run/upload_concurrency/curve_report.json" \
+  --report "$E2E_DIR/.run/limits_drain_rate/curve_report.json" \
+  --output-json "$RUN_DIR/combined_report.json" \
+  --output-txt "$RUN_DIR/combined_summary.txt"
+
+echo
+echo "=== #323's capacity baseline (#386) ==="
+cat "$RUN_DIR/combined_summary.txt"
+
+log "PASS: limits harness one-command capacity baseline (#386) — $RUN_DIR/combined_report.json"