Derive client timeouts from the token budget, not a latency percentile — and enforce the bound in CI

[OriNachum]

The finding

examples/worker_seam.py shipped REQUEST_TIMEOUT = 300.0 in front of a
cortex that generates at ~23 tok/s and thinks for 4–11.5 k tokens per turn.
Every generation past ~6,975 tokens was cut mid-flight, re-run from the top,
and after four attempts discarded — while recording truncated: false. It cost
two scored answers and censored the completion-length distribution of every
measurement taken through that seam. Full account in
#41; fixed for that
harness under pre-registration amendment 1 and deviation d4.

This issue is about the general rule, because the specific fix does not
prevent the next instance.

Why "300 was too small, use 400" is the wrong lesson

The operator asked the right question: why 300 and not 330 or 360, in case we
miss? The answer is that no value chosen that way is safe, and the reason is
worth stating precisely.

At the measured ~23 tok/s:

timeout	tokens it buys
300 s	~6,975
330 s	~7,670
360 s	~8,370
600 s	~13,900

The first call recorded after the fix was 11,458 completion tokens in
460 s — a perfectly ordinary successful turn. A 360 s timeout would have cut
it. The largest completion ever recorded through this seam before the fix was
6,372 tokens, and that number is not a fact about the model: it is a fact about
the timeout.

Which is the trap. Sizing a timeout as "observed tail × margin" requires
knowing the tail — but the timeout censors the tail. The distribution you would
size the margin from is the one the defect has already truncated, so the
estimate is biased low by exactly the mechanism you are trying to protect
against, and the bias is invisible: nothing in the record says "a longer turn
happened here and was deleted". A margin over a censored percentile is not a
safety factor, it is a restatement of the bug.

The same circularity bit our own pre-registration. §14 computed a 1.47×
margin over a 204.3 s measured tail and flagged it as thin — good instinct,
right axis, wrong quantity. That 204.3 s came from an easier task, so the
margin was over a percentile that did not describe the workload. It correctly
predicted the failure and still could not size the fix.

The rule proposed

A client timeout in front of a token-budgeted model is derived from the
budget, not from a latency percentile:

REQUEST_TIMEOUT  >=  max_tokens / slowest_measured_generation_rate

Two properties the percentile approach does not have:

1. It is not circular. max_tokens is a hard ceiling the model cannot
   exceed, so time-to-exhaust-budget is a true worst case that can be computed
   without observing a tail — including a tail the current timeout is hiding.
2. It moves the binding constraint onto the one that reports itself. Above
   the bound, a turn that runs long ends as finish_reason == "length":
   visible, gradeable, and already handled as an instrument event. Below it,
   the identical turn ends as a transport retry, and nothing downstream can
   tell it happened. This is the same defect shape as
   #37 (ModelResponse
   carries no finish_reason) and the same reasoning that produced d16.

The rate belongs to the rig and should be cited, not guessed. Measured here on
2026-08-01, ten calls: 21.5 tok/s slowest, 23.2 mean
(docs/live-test-results/orchestrator-worker-preregistration.md §18).

Repo-wide audit — where every timeout sits today

Script: timeout-audit.py (to be committed with the fix, so it is re-runnable
rather than a number in an issue). Bound computed at 21.5 tok/s against the
largest token budget in each file.

file	constant	secs	budget	need	verdict
examples/worker_seam.py	REQUEST_TIMEOUT	300	16000	744	BELOW BOUND (0.40×) — fixed to 1200
examples/league_commander.py	REQUEST_TIMEOUT	900	16000	744	OK, 1.21× — the narrowest passing
examples/league_h2h.py	REQUEST_TIMEOUT	1800	16000	744	OK, 2.42×
examples/devague_legs.py	REQUEST_TIMEOUT_S	600	4000	186	OK, 3.23×
examples/muse_arms.py	REQUEST_TIMEOUT_S	600	3000	140	OK, 4.30×
examples/worker_throughput.py	BATCH_WAIT_TIMEOUT_SECONDS	300	1200	56	OK, 5.38×

Only one harness was below the bound, and it is the one that failed — which is
mild evidence the bound is measuring the right thing rather than flagging
everything.

league_commander.py at 1.21× deserves a look. Its margin over a full
budget is 156 s. That harness produced the 2.4–4.4× hierarchy cost figure this
repo cites regularly, and if any of its turns were cut the same way, its cost
numbers are inflated and its correctness numbers are understated. Whether that
happened is checkable: its committed records carry retries per call.

Proposed work

1. Commit the audit as a test, not a script. A test_timeout_bounds.py
   that walks the harnesses, computes max_tokens / RATE per file, and fails
   on any timeout below it. The value of the M3 discipline is that the number
   is recomputed from a committed input — a timeout that drifts below its bound
   should break CI, not wait for a series to notice.
2. Re-examine league_commander's committed records for cut turns —
   retries > 0 on any call, and any gap between wall clock and
   tokens / rate. If they are clean, say so and the 1.21× stands; if not, its
   published figures need the same treatment C1-E got.
3. Decide whether the bound belongs in embodiment/ rather than per-harness.
   Six harnesses have each invented a timeout; the rule is the same for all of
   them and none of them is where it belongs. This needs a design decision
   about whether the loop layer owns transport policy at all — it is a genuine
   question and out of scope for a defect fix.
4. Carry the rule to colleague. They dial the same cortex through their own
   transport and would hit this identically. Filed separately as part of the
   t19 findings post.

What this is not

Not a claim that 1200 s is right for every rig. The bound is
max_tokens / rate, and both terms are local: a faster card or a smaller
budget moves it. The transferable part is the derivation and the reason —
put the binding constraint on the quantity that reports itself.

• embodiment (Claude)

[OriNachum]

Correcting this issue's own audit — it was wrong in both directions

The table above was produced by a script that matched *TIMEOUT* named constants only. Two independent errors followed, and both are more interesting than the original finding.

Error 1 — it missed every inline literal

Most model dials in this repo do not name their timeout. The real inventory is 18 dial sites, not 6:

examples/challenge_subset.py:239     urlopen(request, timeout=600)
examples/challenge_register.py:306   urlopen(request, timeout=600)
examples/challenge_entropic.py:440   urlopen(request, timeout=600)
examples/proof.py:560                urlopen(request, timeout=600)
examples/challenge_coding.py:1599    urlopen(request, timeout=900)

Four of those are genuinely below the bound. challenge_subset, challenge_register, challenge_entropic and proof each pair a literal timeout=600 with a literal max_tokens=16000 in the same file. At 21.5 tok/s the bound is 744 s, so they cut anything past ~12,900 tokens. No inference involved — both numbers are literals at the call site.

Error 2 — the rewrite then over-reported, confidently

The second version matched inline literals but resolved bare timeout=timeout parameters by taking the smallest default in the file. It reported league_seat.py at 300 s and echo_probe.py at 15 s, and both are wrong:

• league_seat.py dials at --turn-timeout, default 900.0 (line 1906, passed at 1765). The 300.0 at line 972 is a signature default the CLI overrides; the 300.0 at line 1905 is the league binary subprocess timeout. Three unrelated 300s in one file, and the script picked the wrong one with total confidence.
• echo_probe.py's 15 s is a health check. Its dial default is 300 s.

So an audit that only checked the tidy cases certified the untidy ones, and the fix for that over-reported by guessing. This is why work item 1 asks for a test rather than a script: a test resolves what the CLI actually passes, and a false alarm fails loudly instead of sitting in a table looking authoritative.

The finding this actually produced

league_seat clearing the bound settles something load-bearing: t24's 0 of 58 truncated at 16000 was not censored, so P1's basis in the orchestrator-worker pre-registration stands. An earlier claim of mine that it was an artifact is withdrawn.

And the four harnesses that are under the bound were correctly sized when written. live-suite-and-challenges.md records challenge_register reaching finish_reason: length at a full 16000/16000 inside its 600 s timeout — which requires ≥26.7 tok/s. Today's cortex measures 21.5–25.4 tok/s.

The model upgrade made the cortex slower per token, and that is what moved four harnesses under their bound. Nobody misconfigured anything.

That sharpens the rule this issue proposes:

A timeout is not a property of a harness. It is a property of a harness paired with a model — and a model swap silently re-sizes every timeout in the repository at once, with no diff, no error, and no test failure.

Which makes the CI check the whole point rather than a tidiness measure: it is the only thing that would notice a model swap invalidating a constant nobody edited.

• embodiment (Claude)

[OriNachum]

The league_commander re-exam came back CLEAN — and the audit table has an error

Task t4 of the follow-up cycle (#46) did the check this issue asked for in proposed work 2. Committed as docs/live-test-results/corrections.md §9, with a re-runnable script and 13 pinning tests.

Verdict: clean. Across 384 calls (12 matches / 120 calls primary, 6 matches / 264 calls escalation): retries > 0 in 0 calls, transport errors 0, wall clock ≥ REQUEST_TIMEOUT in 0 — slowest single call 193.6 s, 21.5% of the 900 s clock — finish_reason tool_calls 384/384 with zero length, and 0 orphan transcripts. The 2.4–4.4× hierarchy figures stand, independently recomputed from the raw ledger (2.379 and 4.369).

Two structural facts make that stronger than a row of zeroes usually is. gateway_seam starts its stopwatch before the first attempt and never resets it, so any call that timed out even once would record ≥ 930 s — arithmetically impossible at a 193.6 s maximum. And run_series wraps play_match in no try, so a crashed match would leave transcript lines with no ledger row; there are none. (Correction to this issue's arithmetic: that harness sleeps 30 s between attempts, not 20 s, so its exhausted identity is 4 × 900 + 3 × 30 = 3690 s.)

The clock was never in a position to bind because the budget never was: the largest completion was 2,253 tokens — 14.1% of the 16000 budget.

The audit table is wrong about this row, in a way the rule should have caught

| examples/league_commander.py | REQUEST_TIMEOUT | 900 | 16000 | 744 | OK, 1.21×, the narrowest passing |

That row derives the bound at the cortex rate. But league_commander dials Gemma 4 31B through the same constant. At Gemma's measured 12.1 tok/s the bound is 1,322 s — so 900 s is 0.68×, BELOW BOUND, not the narrowest passing row.

It never mattered in practice (Gemma's largest completion was 236 tokens, 1.5% of budget), and the caveats are real: Gemma's rate is fitted from 31–236-token completions, so applying it to 16000 is a ~68× extrapolation, and Gemma is proxied on this rig. But the rule stated here needs a clause it does not currently have:

derive the bound at the slowest model the constant fronts — not at the rate of whichever model you had in mind when you wrote it.

A single timeout in front of two models has two bounds, and only the larger one is safe. This is the same shape as the finding itself: the quantity that binds is not the one you were thinking about.

Second advisory, also not covered by the clean verdict. The bound is generation-only. 900 s over a 744 s bound leaves 155.8 s for everything that is not generation — and one measured Qwen call spent 179.3 s not generating (193.6 s wall for 365 completion tokens). A full-budget completion queued behind that would total ~923 s and be cut. The margin is thin rather than comfortable, and derive_timeout should either carry an explicit non-generation allowance or state plainly that it excludes queue time.

Both are now instructions on task t2 of the plan, which lands test_timeout_bounds.py.

One honest limit recorded with the verdict: committed records cannot exclude a run whose transcript file and arena root were deleted and restarted from empty. Nothing suggests it — all four record files landed in a single commit and were never amended — but the check cannot see that case, so it is stated rather than implied.

• embodiment (Claude)

[OriNachum]

A third clock category this issue's audit does not cover: the retry backoff

The audit here enumerates client timeouts. The follow-up cycle added wait deadlines (the fan-out's 60 s, which bounded a whole unit drive and is now derived). t8's probe surfaces a third, and it is the one that has actually corrupted measurements:

examples/worker_seam.py   RETRY_SLEEP_SECONDS = 20.0

It is timed inside the call's own stopwatch. WorkerSeam starts started = time.monotonic() before the attempt loop and never resets it, so a retried call reports its backoff as latency — and every rate derived from that latency is wrong by however long the sleep was.

20 s is 20× the median healthy scoped call. For a call whose whole job takes ~1 s, the backoff does not perturb the measurement, it replaces it.

Three separate instances in one session, all the same shape:

1. worker-throughput.jsonl — 9 calls (32% of width 14) carried retries: 1, producing the "slowest measured rate" of 12.921 tok/s that both the rate config and pre-registration amendment 2 derive from. Retry-clean floor: 30.558. Written up in corrections.md §10.
2. t8's own pilot — 1 retry in 141 calls would have published a lean speedup of 0.25×, i.e. concurrency making calls slower, purely as a sleep-constant artefact.
3. The document that reported all of this as clean: worker-throughput.md's stability section claims "zero transport retries" against records holding nine, because ok: true is set after a retry succeeds.

The rule this issue states — put the binding constraint on the quantity that reports itself — has an unstated companion, and these three instances are it:

A constant that lands inside a measurement must be derived against that measurement's own scale, or it silently becomes the measurement.

20 s was presumably chosen against a request that takes tens of seconds. Against a 1 s call it is not a backoff, it is the result. Same defect as sizing a timeout against the wrong denominator — just pointing at the instrument rather than at the model.

Now an instruction on t2 alongside the two findings from the league_commander re-exam: bring backoff constants under the same derive-or-justify rule, or state explicitly why a fixed backoff is exempt. Either is fine; silence is not.

• embodiment (Claude)

[OriNachum]

The rule as stated is necessary but not sufficient — a live counter-example

This issue's rule derives the client timeout. That is correct and it is not enough, and we now have a measured instance rather than an argument.

The pre-registered series stopped this morning with cell C2-E UNFINISHED at 2 of 6 attempts, its rung 3678 s past cap. Two of its calls look like this:

{"finish_reason": "transport_failure", "completion_tokens": 0, "seconds": 2460.384, "retries": 4}
{"finish_reason": "transport_failure", "completion_tokens": 0, "seconds": 2460.412, "retries": 4}

Our retry identity is 4 × REQUEST_TIMEOUT + 3 × 20 s. Solving for the timeout that actually fired:

client timeout	predicted	matches 2460.4 s?
300 s	1260 s	no
600 s	2460 s	yes
1200 s	4860 s	no

Our client was at 1200 s — amendment 1's derived value, this issue's rule correctly applied. The bound that fired was 600 s: GATEWAY_READ_TIMEOUT, a default in the lobes gateway, in a different process, with a different owner. At this cortex's measured 21.5 tok/s a 16000-token budget needs 744 s, so that hop sits at 0.81× — below bound — and no client-side value can raise it.

Filed as lobes-cli#169.

The clause this issue needs

Every hop between the caller and the model has its own bound, and the smallest one binds. Deriving your own and stopping there just means the censoring happens somewhere you cannot see.

Note what survived and what did not. The derivation held up perfectly — 744 s is right, 1200 s is a sound client value, and amendment 1 was correct. What failed is the assumption that the client bound is the outer bound. It is only the outer bound if nothing shorter sits between you and the model.

And the failure mode is the same one this issue was opened about, with the same signature: zero completion tokens, four retries, ~41 minutes of GPU time per call, and nothing in the record saying a healthy generation was cut. The client cannot distinguish "the gateway gave up on a working generation" from "the backend died", so it retried the whole thing four times. 82 minutes of measured work, discarded, and the rung cap blown partly as a result.

The running tally

This is the fourth distinct clock this cycle has found sized against the wrong quantity:

#	clock	was	problem
1	worker_seam.REQUEST_TIMEOUT	300 s	censored the completion-length distribution (this issue)
2	DEFAULT_FANOUT_TIMEOUT	60 s	bounded a whole unit drive at 1/248th of it (amendment 2)
3	RETRY_SLEEP_SECONDS	20 s	timed inside the call it retries, 20× the median scoped call
4	GATEWAY_READ_TIMEOUT	600 s	not ours, not visible, and it binds

Three of the four were invisible in the record at the moment they fired. That is the actual through-line, and it is worth stating as a design rule rather than four separate fixes: a bound that cannot report when it fires will be discovered by its damage.

The structural fix pointing at all four is the streaming ask (lobes-cli#168): a streaming client bounds idle time between chunks rather than total request time, which makes generation length irrelevant to every clock in the chain — dissolving the class rather than resizing its members one at a time.

The immediate one is smaller: the effective read timeout should be discoverable from /capabilities, so a consumer can check its derived bound against every hop at startup and fail loudly, instead of finding out four retries later.

• embodiment (Claude)

[OriNachum]

Closing result — 8 of 8 constants clear their derived bound, enforced in CI

The cycle this issue seeded is on PR #49.
Full accounting in docs/live-test-results/timeouts-bee-hive-accountability.md §1.

The count moved from 6 to 8, and that is the result worth reporting

stage	count	how it grew
this issue's audit	6	hand-listed client timeouts
c16	7	found the fan-out wait deadline the audit had not classified as a timeout
the shipped test	8	closes the category by AST; the eighth turned itself in

The eighth is worker_scoped_overhead.BATCH_WAIT_TIMEOUT_SECONDS, which landed after
this audit was written and inherited a sibling's value. It passes (bound 231.0 s
against a shipped 300.0). The point is that nothing had checked, four days after an
audit whose entire subject was underived clocks.

A list somebody maintains goes stale the first time it is not maintained; a category
that closes itself does not. If one thing from this cycle is worth carrying to
another repo, it is the AST walk rather than any of the numbers.

Every constant, by value

constant	kind	shipped	derived bound	margin
worker_seam.REQUEST_TIMEOUT	client	1300.0	1238.29	1.05×
league_commander.REQUEST_TIMEOUT	client	1600.0	1501.29	1.07×
league_h2h.REQUEST_TIMEOUT	client	1800.0	1501.29	1.20×
devague_legs.REQUEST_TIMEOUT_S	client	600.0	365.72	1.64×
muse_arms.REQUEST_TIMEOUT_S	client	600.0	427.14	1.40×
worker_throughput.BATCH_WAIT_TIMEOUT_SECONDS	wait	300.0	272.13	1.10×
orchestrator_tools.DEFAULT_FANOUT_TIMEOUT	wait	14860.0	14859.53	1.00×
worker_scoped_overhead.BATCH_WAIT_TIMEOUT_SECONDS	wait	300.0	230.97	1.30×

Plus 3 streaming clocks and 4 retry backoffs, each with a stated exemption.
tests/test_timeout_bounds.py — 120 passed, 3 skipped — recomputes every bound from a
dated committed rate config, with a test-of-the-test that mutates the real module
attribute and asserts the suite fails.

The finding this issue's rule does not fix

A live series lost two repetitions to calls that ran 2,460.4 s and then failed,
while the client sat at 1,200 s. The arithmetic is 4 × 600 + 3 × 20 — four attempts
against a 600 s bound that lives in the lobes process, where no client value
can reach it.

Every hop between caller and model has its own bound, and the smallest binds. This
issue's rule is necessary and not sufficient. Filed as
lobes-cli#169.

And the thing that retires the class rather than raising it

Streaming, adopted as the default transport (deviation d3, classified risky
because it changed the transport under a series that had not yet run). Through a
43.45 s think the largest inter-chunk gap was 0.124 s — ~4,800× margin against
even a 600 s idle bound. Under streaming the gateway's read timeout stops being a
whole-request deadline and becomes an inter-chunk idle bound.

Two live findings a design could not have produced: the reasoning delta field on this
rig is delta.reasoning, not vLLM's documented reasoning_content; and the
terminal usage chunk requires stream_options: {include_usage: true} or token counts
vanish.

One lane is exempted and stays non-streaming: the scoped worker lane, whose request
clock is the width rung's outcome metric.

Reported to the sibling that would hit this identically

colleague#362. They dial the
same gateway and the same cortex, their _DEFAULT_TIMEOUT is 120 s (≈2,574 completion
tokens at the measured rate) — and _make_complete sends no max_tokens, so this
issue's rule cannot be applied there at all. A timeout that is underivable is a
different and more awkward problem than one that is merely too small.

• embodiment (Claude)