[jjxyxsjr]

Problem and behavior

The execution backstop (RATE_LIMIT_EXECUTION_MAX_WAIT_MS / per-connection overrides / the upstream fetch-start floor) is calibrated for interactive calls. A non-streaming request that legitimately asks for tens of thousands of output tokens can still be mid-decode when that deadline fires — the client gets a hard 504 while the upstream was healthy and working. Observed shape: max_tokens=32768 non-stream requests killed at exactly the base deadline, repeatedly, while the same calls streaming succeeded.

Changes

• New computeNonStreamExecutionExpirationMs(baseMs, maxTokens, env) in open-sse/services/rateLimitManager.ts, opt-in via OMNIROUTE_EXECUTION_EXPIRATION_MS: unset = base unchanged; auto = scale by max_tokens at a conservative ~20ms/token decode floor (50 tok/s), clamped to [baseMs, 15 min]; a number = fixed floor in ms.
• withRateLimit's deadline becomes a three-way max() — the caller floor can only raise the deadline; the per-connection and upstream-fetch-start floors (fix(resilience): requestQueue.maxWaitMs is applied as a post-dispatch execution expiration — legitimate long LLM calls die with false 504s #12025/fix(resilience): decouple rate-limit execution expiration from queue-wait budget; preserve errors in oversized call-log artifacts #12027 invariants) cannot be undercut.
• One wiring point: executeProviderRequest.ts passes the computed floor for non-streaming calls only; streaming is untouched (token output keeps the stream alive).
• Env documented in .env.example + docs/reference/ENVIRONMENT.md.

Validation

tests/unit/execution-expiration-scaling.test.ts — 7 assertions pin, per mode:

• default: base returned unchanged for any maxTokens (the compatibility property)
• auto: 32768 × 20ms scaled past the base; 1024 tokens keep the base (floor semantics); 1_000_000 tokens cap at 900s; 0/undefined keep the base
• numeric: 600000 fixed floor; 0 passes through as a no-op floor
• junk (junk, -5, 1e999→Infinity) ignored

tsc -p tsconfig.typecheck-core.json exit 0 · repo npm run lint exit 0 · check:env-doc-sync green for this variable (the gate does fail on the pristine release/v3.8.53 base for the unrelated pre-existing OMNIROUTE_MODEL_SYNC_TIMINGS gap — docs fix open as #18723).

Limitation: no end-to-end dispatch test — the helper is pure and the wiring is a 4-line floor passthrough into the existing max(); per-connection/upstream floors are covered by the existing suites.

Notes

• Unset = the only delta is a boolean env check — identical behavior, effectively zero cost.
• No new dependencies, no schema change, no UI. The 20ms/token floor and 15-min cap are deliberately conservative; happy to make them configurable.
• Base: release/v3.8.53 (active release branch); aware of the 3.8.55 v4-feature retarget policy.