Summary
The HTTP /chat (and /acp) turn deadline is a hardcoded 30s compile-time constant with no env override. A slow provider (cold local model, long generation, rate-limited hosted API) returns 504 to the client even though the turn succeeds and is billed server-side.
Evidence
crates/server/src/routes.rs:46 — const HTTP_TURN_TIMEOUT: Duration = Duration::from_secs(30); (applied at /chat and /acp).
Reproduction
First POST /chat after boot with a not-yet-resident Ollama model cold-loads (~56s observed) → client 504 {"error":"turn processing timed out"} at 30s; the turn completes server-side and mints a receipt + spends budget. The identical request once warm → 200 in ~2–4s.
Impact
Spurious 504s and duplicate work on client retry (the successful turn already minted a receipt and spent budget).
Suggested fix
Make the deadline configurable (e.g. ARDUR_HTTP_TURN_TIMEOUT_SECS) with a default above realistic provider cold-start/long-generation latency, or stream/keep-alive instead of a hard cut.
Summary
The HTTP
/chat(and/acp) turn deadline is a hardcoded 30s compile-time constant with no env override. A slow provider (cold local model, long generation, rate-limited hosted API) returns504to the client even though the turn succeeds and is billed server-side.Evidence
crates/server/src/routes.rs:46—const HTTP_TURN_TIMEOUT: Duration = Duration::from_secs(30);(applied at/chatand/acp).Reproduction
First
POST /chatafter boot with a not-yet-resident Ollama model cold-loads (~56s observed) → client504 {"error":"turn processing timed out"}at 30s; the turn completes server-side and mints a receipt + spends budget. The identical request once warm →200in ~2–4s.Impact
Spurious 504s and duplicate work on client retry (the successful turn already minted a receipt and spent budget).
Suggested fix
Make the deadline configurable (e.g.
ARDUR_HTTP_TURN_TIMEOUT_SECS) with a default above realistic provider cold-start/long-generation latency, or stream/keep-alive instead of a hard cut.