Commit 0c0a3ae
[iris] benchmark_controller: production-mix scenario over real RPC (#5775)
## Summary
- Reworks `lib/iris/scripts/benchmark_controller.py` so every benchmark
exercises the controller through its real Connect/HTTP surface
(serialization, ASGI dispatch, `@on_loop` vs threadpool, real network)
rather than calling service/transitions helpers directly in-process.
- Adds an out-of-process controller harness so the benchmark process's
client threads don't share a GIL or asyncio loop with the server.
- Adds a `scenario` abstraction (`RpcLoad` / `Scenario` /
`ScenarioRunner`) that drives a *mix* of RPCs at
independently-configurable target rates, with a `PRODUCTION_MIX_RPS`
preset derived from the controller stats page (SetTaskStatusText
dominates at 316 rps, every other RPC < 8 rps). `--scale` multiplies all
rates for overload runs.
- Also bumps `_wait_until_job_drained`'s backoff in `service.py` so long
replacement drains don't hammer the heartbeat path.
## What's new
| component | purpose |
|---|---|
| `RpcHarness` | Real `Controller(dry_run=True)` in-process;
production-cadence loops, destructive writes gated. |
| `SubprocessRpcHarness` | Spawns the controller in a child process via
the new `serve` subcommand. Parent owns clients, child owns the event
loop. |
| `RpcLoad` / `Scenario` / `ScenarioRunner` | Multi-RPC mix runner.
`n_clients_min` knob so e.g. SetTaskStatusText runs from 200 connection
threads, matching the production fleet's connection count. |
| 17 `load_*` factories | One per RPC in the production distribution.
`benchmark_rpcs` now drives its per-call microbenches through the same
factories. |
| `benchmark_scenario` group | Production-shape mix run via `... run
--only scenario --scale 1.0 --scenario-duration 60`. |
| `benchmark_set_task_status_text` group | Head-to-head `@on_loop` vs
threadpool comparison + blast-radius probe (1000 RPS SetTaskStatusText
storm vs victim UpdateTaskStatus). |
CLI is now a click.group:
- `... run [--only X] [--scale Y] [--scenario-duration Z]` — benchmark
groups
- `... serve --db-path X --state-dir Y` — controller subprocess used by
`SubprocessRpcHarness`
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>1 parent 1bfeb6b commit 0c0a3ae
2 files changed
Lines changed: 1317 additions & 153 deletions
0 commit comments