feat: aiod tune — cheapest-per-token benchmark sweep (v0.5.0)

[jhammant]

Designed + implemented + adversarially reviewed via multi-agent workflows; I verified the diff and the cost-safety tests before pushing.

aiod tune <model|profile> rents a box, sweeps it, prints a $/1M-tokens leaderboard, and recommends the cheapest serving config that meets a latency bar — the payoff for the v0.4.0 optimization registry.

Cost-safety is the spine (this rents real GPUs)

• --max-cost is REQUIRED (or an explicit --yes-i-know), enforced before every launch and around every bench point via a CostGuard.
• Teardown is GUARANTEED on success, exception, AND Ctrl-C — it destroys inst OR state.load(), so a box rented during the multi-minute boot is killed even before tune binds the handle.
• Additive startup_grace kwarg on engine.launch (default None == today's exact 1200s loop) lets tune early-abort a bad node.

What it sweeps

• Default: one box, one model load, a concurrency ladder (cheap — vLLM batches).
• Opt-combo sweep (relaunch per combo) is opt-in.
• Projected (~) rows order the run + feed the estimate but are never the recommendation — the winner is always a measured point.

Surface

aiod tune Qwen/Qwen3-32B --max-cost 5 --concurrency 1,4,8 [--sweep-opt kv-cache-fp8] [--save-profile fast]

Tests

39 new (teardown-always incl. KeyboardInterrupt + boot-window leak, cost caps, ranking, projected-never-recommended, startup_grace back-compat) + a @pytest.mark.live smoke. 205 passing, ruff clean. Additive only; bumps v0.5.0.

Known nits (follow-ups, no billing leak — box always destroyed)

• Cost cap is between-boundary: a slow weight pull inside engine.launch can bill past --max-minutes before the next check (bounded by the 1800s ready-timeout).
• A rare unexpected-exception-after-rent path skips one box's cost from the running total (cap precision, not a leak).