Skip to content

Commit 0c0a3ae

Browse files
rjpowerclaude
andauthored
[iris] benchmark_controller: production-mix scenario over real RPC (#5775)
## Summary - Reworks `lib/iris/scripts/benchmark_controller.py` so every benchmark exercises the controller through its real Connect/HTTP surface (serialization, ASGI dispatch, `@on_loop` vs threadpool, real network) rather than calling service/transitions helpers directly in-process. - Adds an out-of-process controller harness so the benchmark process's client threads don't share a GIL or asyncio loop with the server. - Adds a `scenario` abstraction (`RpcLoad` / `Scenario` / `ScenarioRunner`) that drives a *mix* of RPCs at independently-configurable target rates, with a `PRODUCTION_MIX_RPS` preset derived from the controller stats page (SetTaskStatusText dominates at 316 rps, every other RPC < 8 rps). `--scale` multiplies all rates for overload runs. - Also bumps `_wait_until_job_drained`'s backoff in `service.py` so long replacement drains don't hammer the heartbeat path. ## What's new | component | purpose | |---|---| | `RpcHarness` | Real `Controller(dry_run=True)` in-process; production-cadence loops, destructive writes gated. | | `SubprocessRpcHarness` | Spawns the controller in a child process via the new `serve` subcommand. Parent owns clients, child owns the event loop. | | `RpcLoad` / `Scenario` / `ScenarioRunner` | Multi-RPC mix runner. `n_clients_min` knob so e.g. SetTaskStatusText runs from 200 connection threads, matching the production fleet's connection count. | | 17 `load_*` factories | One per RPC in the production distribution. `benchmark_rpcs` now drives its per-call microbenches through the same factories. | | `benchmark_scenario` group | Production-shape mix run via `... run --only scenario --scale 1.0 --scenario-duration 60`. | | `benchmark_set_task_status_text` group | Head-to-head `@on_loop` vs threadpool comparison + blast-radius probe (1000 RPS SetTaskStatusText storm vs victim UpdateTaskStatus). | CLI is now a click.group: - `... run [--only X] [--scale Y] [--scenario-duration Z]` — benchmark groups - `... serve --db-path X --state-dir Y` — controller subprocess used by `SubprocessRpcHarness` --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 1bfeb6b commit 0c0a3ae

2 files changed

Lines changed: 1317 additions & 153 deletions

File tree

0 commit comments

Comments
 (0)