feat: add model request fitting and idle timeouts

Use advertised model output limits, fit requests to available context, preserve answer room for thinking budgets, and record effective request settings. Add configurable byte-idle transport timeouts without an absolute turn deadline.

Fixes #39

Signed-off-by: Hari Srinivasan <harisrini21@gmail.com>

Author: Haz3-jolt

README.md

@@ -129,6 +129,7 @@ A session belongs to the daemon, not to the terminal that created it. Closing a
 - `axl -r` opens the all-placement resume picker.
 - `axl <session-id>` resumes a known session directly.
 - `session.interrupt` is the explicit cancellation operation.
+- `/request` shows or changes the daemon-owned output ceiling and HTTP idle timeout. The default output ceiling is the model maximum, fitted to available context; the idle timeout is five minutes and refreshes on response bytes.
 - Daemon restart recovery reconciles accepted operations against canonical history before serving clients.
 
 Resume uses a frozen, paged snapshot followed by an acknowledged live event stream. The SDK rejects altered duplicates, detects gaps, and replaces a projection from an authoritative snapshot when a cursor cannot be resumed safely.

SETUP.md

@@ -66,7 +66,7 @@ axl daemon stop --interrupt --yes
 
 Use the same `--unsafe`, `--sandbox`, `--image`, or `--socket` selection as the running daemon. Status and stop do not require provider credentials. Restart refuses to switch data directories. Stop and restart refuse active work unless `--interrupt` explicitly authorizes cancellation. `--yes` confirms disconnecting clients. A changed confirmation snapshot requires a fresh command. Exit codes are 0 for success, 1 for errors, 2 for refused or stale confirmation, and 3 for a missing daemon on status or stop. Restart starts a missing daemon.
 
-An incompatible session wire fails loudly and points to these commands. No daemon is automatically replaced on a version mismatch. Host-control version 1 operates independently of session wire version 10 on a separate connection to the same owner-only Unix socket. It does not bypass the session handshake.
+An incompatible session wire fails loudly and points to these commands. No daemon is automatically replaced on a version mismatch. Host-control version 1 operates independently of session wire version 11 on a separate connection to the same owner-only Unix socket. It does not bypass the session handshake.
 
 If graceful cleanup fails or exceeds the host's ten-second wait, inspect `axl daemon status`. Shutdown can still be running. The TUI offers a separate force confirmation when available. From the CLI, explicitly request:
 
@@ -78,6 +78,24 @@ Force is accepted only after graceful shutdown has begun, and only by the same d
 
 Daemons from builds before host control cannot be recovered through these new commands. For that one-time transition, inspect the old process with operating-system tools, verify its command, owner, and socket, then send SIGTERM to that verified process. A PID in `.axl-data.lock` alone is not proof. Do not delete an active lock or kill every Node process. Once the old process has exited, normal startup reclaims its stale socket and lock. Preserved sessions remain resumable.
 
+### Model request limits
+
+Ordinary requests default to the selected model's advertised output maximum. Axl reduces that ceiling only when needed to fit the estimated input plus 4,096 reserved tokens inside the model context window. Every request sends an explicit provider output ceiling. The selected reasoning level remains unchanged. Providers that use explicit thinking-token budgets reserve at least 1,024 tokens for the answer.
+
+The HTTP idle timeout defaults to five minutes and applies separately while waiting for response headers and between response-body bytes. Streaming bytes, including SSE heartbeat comments, refresh it. It is not an absolute turn or request deadline. A timeout is reported separately from user cancellation and is never automatically retried after dispatch because provider acceptance may be uncertain.
+
+Show or change daemon-owned settings in the TUI:
+
+```text
+/request
+/request output 8192
+/request output model
+/request idle 300000
+/request idle disabled
+```
+
+For a new CLI session, use `--max-output-tokens <n|model>` and `--http-idle-timeout <milliseconds>`. Zero disables the idle timeout. Effective settings are recorded in canonical history and shown by `/status`. Axl has no numerical model-call ceiling or absolute turn-duration limit. Session budgets and proactive compaction remain separate features.
+
 
 Inspect local sandbox support without configuring provider credentials:
 

docs/architecture/web-protocol.md

@@ -13,13 +13,13 @@ This document specifies typed RPC, negotiation, errors, package ownership, and t
 
 ## Current baseline
 
-Wire version 10 uses newline-delimited JSON over a Unix socket. It includes typed request and result envelopes, initialization, capability negotiation, structured errors, idempotency keys, subscription identities, paged snapshots, acknowledged opaque cursors, presence, daemon security reporting, direct shell events, transient activity, session-bound blobs, workspace review, session profiles, web-tool selection, manual compaction, steering, follow-ups, and canonical model-retry attempts.
+Wire version 11 uses newline-delimited JSON over a Unix socket. It includes typed request and result envelopes, initialization, capability negotiation, structured errors, idempotency keys, subscription identities, paged snapshots, acknowledged opaque cursors, presence, daemon security reporting, direct shell events, transient activity, session-bound blobs, workspace review, session profiles, web-tool selection, manual compaction, steering, follow-ups, and canonical model-retry attempts.
 
-The TUI consumes these contracts through `packages/sdk`. Version 10 adds the terminal connection error `daemon_stopping`, so explicit shutdown does not trigger automatic replacement. Host-control version 1 is separate from session wire negotiation and is available only to trusted process hosts.
+The TUI consumes these contracts through `packages/sdk`. Version 11 adds daemon-owned model request settings plus canonical effective-request events. Host-control version 1 remains separate from session wire negotiation and is available only to trusted process hosts.
 
 ## Versioning
 
-The current wire version is 10. Version 8 introduced typed envelopes, initialization, errors, retry metadata, subscriptions, cursors, acknowledgements, and presence. Version 9 adds the canonical `model.retry_scheduled` event. Version 10 adds `daemon_stopping` as a pre-RPC and universal RPC error. Compatible capability additions that do not alter accepted wire data do not require a bump. Pre-1.0 clients require an exact wire-version match.
+The current wire version is 11. Version 8 introduced typed envelopes, initialization, errors, retry metadata, subscriptions, cursors, acknowledgements, and presence. Version 9 adds the canonical `model.retry_scheduled` event. Version 10 adds `daemon_stopping` as a pre-RPC and universal RPC error. Version 11 adds `config.request` and `model.request_configured` events and request settings in session create and configure RPCs. Compatible capability additions that do not alter accepted wire data do not require a bump. Pre-1.0 clients require an exact wire-version match.
 
 The daemon sends `hello` first:
 

packages/ai/README.md

@@ -4,3 +4,5 @@
 # `@axl/ai`
 
 This package keeps provider-specific behavior outside the kernel. It defines provider and model contracts, credential lookup, thinking levels, tool dialects, deterministic test models, and the Azure OpenAI Responses adapter with Axl's built-in model catalog.
+
+Ordinary requests use the selected model's advertised output maximum, reduced only to fit estimated input plus a 4,096-token context reserve. The adapter sends the effective ceiling explicitly. Model HTTP transport uses Undici with a configurable five-minute default inactivity timeout for headers and response-body bytes. Streaming bytes refresh the timeout, zero disables it, and no absolute request deadline is imposed.

packages/ai/package.json

@@ -24,6 +24,7 @@
     "typecheck": "tsc --noEmit"
   },
   "dependencies": {
-    "@axl/protocol": "workspace:*"
+    "@axl/protocol": "workspace:*",
+    "undici": "8.10.2"
   }
 }

packages/ai/src/index.ts

@@ -16,3 +16,4 @@ export * from "./sse.ts";
 export * from "./stream.ts";
 export * from "./thinking.ts";
 export * from "./usage.ts";
+export * from "./request-configuration.ts";

packages/ai/src/model.ts

@@ -67,6 +67,8 @@ export interface ModelRequest {
   readonly tools?: readonly ToolDeclaration[];
   readonly thinkingLevel?: ThinkingLevel;
   readonly maxOutputTokens?: number;
+  readonly httpIdleTimeoutMs?: number;
+  readonly estimatedInputTokens?: number;
   readonly toolChoice?: "auto" | "required" | "none";
   /** Resolves content-addressed media without placing bytes in canonical events. */
   readonly readBlob?: (reference: BlobReference) => Promise<Uint8Array>;

packages/ai/src/openai-responses.ts

@@ -1,5 +1,6 @@
 // SPDX-FileCopyrightText: 2026 Hari Srinivasan
 // SPDX-FileCopyrightText: 2026 Kaushik Kumar
+// SPDX-FileCopyrightText: 2026 Shaan Narendran
 // SPDX-License-Identifier: Apache-2.0
 
 // Axl-native OpenAI Responses codec and transport implementation.
@@ -12,6 +13,9 @@ import type {
   Usage,
 } from "@axl/protocol";
 
+import { EnvHttpProxyAgent, fetch as modelFetch } from "undici";
+import { fitModelRequest } from "./request-configuration.ts";
+
 import { AuthError, type ResolvedAuth } from "./auth.ts";
 import { assertModelSupports } from "./capabilities.ts";
 import type { AuthMethod, ModelInfo, ModelRequest, ModelStreamEvent } from "./model.ts";
@@ -20,6 +24,21 @@ import { decodeSseStream, type SseFrame } from "./sse.ts";
 
 /** OpenAI Responses rejects max_output_tokens below 16. */
 const MIN_OUTPUT_TOKENS = 16;
+// Independently implements Pi's byte-idle timeout semantics from http-dispatcher.ts at 6c87d9a02.
+// https://github.com/badlogic/pi-mono/blob/6c87d9a02/packages/coding-agent/src/core/http-dispatcher.ts
+// Model-only connection pooling. Per-dispatch overrides also override fetch's internal defaults.
+let modelDispatcher: EnvHttpProxyAgent | undefined;
+function dispatcherFor(timeoutMs: number) {
+  modelDispatcher ??= new EnvHttpProxyAgent({
+    allowH2: false,
+    connect: { autoSelectFamilyAttemptTimeout: 2_000 },
+  });
+  return modelDispatcher.compose(
+    (dispatch) => (options, handler) =>
+      dispatch({ ...options, headersTimeout: timeoutMs, bodyTimeout: timeoutMs }, handler),
+  );
+}
+const IDLE_TIMEOUT_CODES = new Set(["UND_ERR_HEADERS_TIMEOUT", "UND_ERR_BODY_TIMEOUT"]);
 const SAFE_CONNECT_FAILURES = new Set([
   "EAI_AGAIN",
   "ENOTFOUND",
@@ -133,9 +152,12 @@ export function encodeResponsesRequest(
     store: false,
   };
   if (request.system !== undefined) body.instructions = request.system;
-  if (request.maxOutputTokens !== undefined) {
-    body.max_output_tokens = Math.max(request.maxOutputTokens, MIN_OUTPUT_TOKENS);
-  }
+  const configuration = fitModelRequest(model, request);
+  if (configuration.maxOutputTokens < MIN_OUTPUT_TOKENS)
+    throw new ResponsesCodecError(
+      `OpenAI Responses needs at least ${MIN_OUTPUT_TOKENS} output tokens; the requested or available ceiling is ${configuration.maxOutputTokens}`,
+    );
+  body.max_output_tokens = configuration.maxOutputTokens;
   if (request.tools !== undefined && request.tools.length > 0) {
     body.tools = request.tools.map((tool) => ({
       type: "function",
@@ -354,7 +376,7 @@ export class OpenAiResponsesProvider implements ModelProvider {
   private readonly endpoint: ResponsesEndpoint;
   private readonly models: readonly ModelInfo[];
   private readonly resolveAuth: () => Promise<ResolvedAuth>;
-  private readonly fetchImpl: typeof fetch;
+  private readonly fetchImpl: typeof fetch | undefined;
 
   constructor(options: OpenAiResponsesProviderOptions) {
     this.id = options.id;
@@ -363,7 +385,7 @@ export class OpenAiResponsesProvider implements ModelProvider {
     this.endpoint = options.endpoint;
     this.models = options.models;
     this.resolveAuth = options.resolveAuth;
-    this.fetchImpl = options.fetch ?? fetch;
+    this.fetchImpl = options.fetch;
   }
 
   listModels(): Promise<readonly ModelInfo[]> {
@@ -384,7 +406,12 @@ export class OpenAiResponsesProvider implements ModelProvider {
     request: ModelRequest,
   ): AsyncGenerator<ModelStreamEvent, void, undefined> {
     let url: string;
-    let init: RequestInit;
+    let init: {
+      method: string;
+      headers: Record<string, string>;
+      body: string;
+      signal?: AbortSignal;
+    };
     try {
       const resolved = await this.resolveAuth();
       const body = encodeResponsesRequest(
@@ -420,9 +447,15 @@ export class OpenAiResponsesProvider implements ModelProvider {
       return;
     }
 
-    let response: Response;
+    let response: Pick<Response, "ok" | "status" | "headers" | "body">;
     try {
-      response = await this.fetchImpl(url, init);
+      response =
+        this.fetchImpl === undefined
+          ? await modelFetch(url, {
+              ...init,
+              dispatcher: dispatcherFor(fitModelRequest(model, request).httpIdleTimeoutMs),
+            })
+          : await this.fetchImpl(url, init);
     } catch (error) {
       const code = nestedErrorCode(error);
       const safeToRetry = code !== undefined && SAFE_CONNECT_FAILURES.has(code);
@@ -491,11 +524,23 @@ export class OpenAiResponsesProvider implements ModelProvider {
     request: ModelRequest,
     error: unknown,
     code: string,
-    requestPhase: "before_dispatch" | "streaming" | "unknown",
+    requestPhase: "before_dispatch" | "awaiting_response" | "streaming" | "unknown",
     retryable: boolean,
     category: ModelErrorCategory,
   ): ModelStreamEvent {
     if (request.signal?.aborted) return { type: "aborted" };
+    const transportCode = nestedErrorCode(error);
+    if (transportCode !== undefined && IDLE_TIMEOUT_CODES.has(transportCode)) {
+      return {
+        type: "error",
+        code: "model_request_idle_timeout",
+        message: `Provider transport was idle for ${request.httpIdleTimeoutMs ?? 300_000} ms while ${transportCode === "UND_ERR_HEADERS_TIMEOUT" ? "waiting for response headers" : "reading the response body"}`,
+        retryable: false,
+        category: "timeout",
+        requestPhase:
+          transportCode === "UND_ERR_HEADERS_TIMEOUT" ? "awaiting_response" : "streaming",
+      };
+    }
     return {
       type: "error",
       code,

packages/ai/src/provider-port.ts

@@ -5,18 +5,24 @@
 import type {
   BlobReference,
   ModelMessage,
+  ModelRequestConfiguration,
+  ModelRequestSettings,
   ModelStreamEvent,
   ThinkingLevel,
   ToolDeclaration,
 } from "@axl/protocol";
 
+import { DEFAULT_MODEL_REQUEST_SETTINGS } from "@axl/protocol";
+import { fitModelRequest } from "./request-configuration.ts";
+
 import type { ModelProvider } from "./provider.ts";
 import { normalizeModelStream } from "./stream.ts";
 
 export interface SessionPortOptions {
   readonly modelId: string;
   readonly thinkingLevel?: ThinkingLevel;
   readonly maxOutputTokens?: number;
+  readonly requestSettings?: ModelRequestSettings;
   readonly readBlob?: (reference: BlobReference) => Promise<Uint8Array>;
 }
 
@@ -27,6 +33,10 @@ interface PortTurnRequest {
   readonly maxOutputTokens?: number | undefined;
   readonly toolChoice?: "auto" | "required" | "none" | undefined;
   readonly signal?: AbortSignal | undefined;
+  readonly estimatedInputTokens?: number | undefined;
+  readonly onRequestConfigured?:
+    | ((configuration: ModelRequestConfiguration) => Promise<void>)
+    | undefined;
 }
 
 /**
@@ -41,19 +51,53 @@ export function modelPortForSession(
   return {
     stream: (request) =>
       normalizeModelStream(
-        provider.stream({
-          modelId: options.modelId,
-          ...(request.system === undefined ? {} : { system: request.system }),
-          messages: request.messages,
-          tools: request.tools,
-          ...(options.thinkingLevel === undefined ? {} : { thinkingLevel: options.thinkingLevel }),
-          ...(request.maxOutputTokens === undefined && options.maxOutputTokens === undefined
-            ? {}
-            : { maxOutputTokens: request.maxOutputTokens ?? options.maxOutputTokens }),
-          ...(request.toolChoice === undefined ? {} : { toolChoice: request.toolChoice }),
-          ...(options.readBlob === undefined ? {} : { readBlob: options.readBlob }),
-          ...(request.signal === undefined ? {} : { signal: request.signal }),
-        }),
+        (async function* () {
+          if (request.signal?.aborted) {
+            yield { type: "aborted" } as const;
+            return;
+          }
+          const model = (await provider.listModels()).find(
+            (candidate) => candidate.modelId === options.modelId,
+          );
+          if (model === undefined)
+            throw new Error(`Provider ${provider.id} has no model ${options.modelId}`);
+          const settings = options.requestSettings ?? DEFAULT_MODEL_REQUEST_SETTINGS;
+          const maxOutputTokens =
+            request.maxOutputTokens ??
+            options.maxOutputTokens ??
+            settings.maxOutputTokens ??
+            undefined;
+          const configuration = fitModelRequest(model, {
+            messages: request.messages,
+            tools: request.tools,
+            ...(request.system === undefined ? {} : { system: request.system }),
+            ...(request.estimatedInputTokens === undefined
+              ? {}
+              : { estimatedInputTokens: request.estimatedInputTokens }),
+            ...(maxOutputTokens === undefined ? {} : { maxOutputTokens }),
+            httpIdleTimeoutMs: settings.httpIdleTimeoutMs,
+          });
+          await request.onRequestConfigured?.(configuration);
+          if (request.signal?.aborted) {
+            yield { type: "aborted" } as const;
+            return;
+          }
+          yield* provider.stream({
+            modelId: options.modelId,
+            ...(request.system === undefined ? {} : { system: request.system }),
+            messages: request.messages,
+            tools: request.tools,
+            ...(options.thinkingLevel === undefined
+              ? {}
+              : { thinkingLevel: options.thinkingLevel }),
+            maxOutputTokens: configuration.maxOutputTokens,
+            httpIdleTimeoutMs: configuration.httpIdleTimeoutMs,
+            estimatedInputTokens: configuration.estimatedInputTokens,
+            ...(request.toolChoice === undefined ? {} : { toolChoice: request.toolChoice }),
+            ...(options.readBlob === undefined ? {} : { readBlob: options.readBlob }),
+            ...(request.signal === undefined ? {} : { signal: request.signal }),
+          });
+        })(),
         request.signal,
       ),
   };