timeout.chunkMs / stepMs count tool-execution time — unusable for tool-loop agents with long-running tools

[yonatan-shorani]

Description

timeout.chunkMs is documented as a streaming idle bound ("aborts if no new chunk is received within the specified duration"). In practice the chunk timer is reset in the transform over the streamWithToolResults pipeline and stays armed while tool executions run (it's only cleared after the step's tool outputs complete — same for the step timer). Tool execution produces no parts, so a tool that legitimately runs longer than chunkMs aborts a perfectly healthy request.

For tool-loop agents this makes both knobs unusable: our tools legitimately run 2-5 minutes (warehouse SQL, sandboxed code execution). Any chunkMs/stepMs tight enough to catch a hung provider (the thing we actually want bounded — a model that accepts the stream and never emits) also kills healthy requests mid-tool. Note timeout.toolMs already exists to bound tool execution separately, so the chunk/step timers covering tool time is redundant coverage of an already-bounded phase.

Repro (ai@7.0.28)

import { streamText, tool } from 'ai';
import { MockLanguageModelV4 } from 'ai/test';
import { z } from 'zod';
import type { LanguageModelV4StreamPart } from '@ai-sdk/provider';

const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));

function streamOf(parts: LanguageModelV4StreamPart[]) {
  return new ReadableStream<LanguageModelV4StreamPart>({
    start(c) {
      for (const p of parts) c.enqueue(p); // all chunks instantly — the model is fast
      c.close();
    },
  });
}

const model = new MockLanguageModelV4({
  doStream: async () => ({
    stream: streamOf([
      { type: 'tool-call', toolCallId: 'c1', toolName: 'slowTool', input: '{}' },
      { type: 'finish', finishReason: 'tool-calls', usage: { inputTokens: 1, outputTokens: 1, totalTokens: 2 } },
    ]),
  }),
});

async function run(label: string, timeout?: { chunkMs: number }) {
  const result = streamText({
    model,
    prompt: 'go',
    timeout,
    tools: {
      slowTool: tool({
        description: 'takes 300ms — stand-in for a database query',
        inputSchema: z.object({}),
        execute: async () => (await sleep(300), { ok: true }),
      }),
    },
    onError: ({ error }) => console.log(`[${label}] onError:`, (error as Error).message ?? error),
  });
  const parts: string[] = [];
  for await (const p of result.fullStream) parts.push(p.type);
  console.log(`[${label}] parts:`, parts.join(', '));
}

await run('no timeout');
await run('chunkMs: 150', { chunkMs: 150 });

Output:

[no timeout]   parts: start, start-step, tool-call, tool-result, finish-step, finish
[chunkMs: 150] parts: start, start-step, tool-call, abort

Every model chunk arrived instantly; the abort fires during the 300ms tool execution.

Suggested fix (either or both)

1. Pause/clear the chunk and step timers while tool executions are pending — so chunkMs measures model silence only, matching the docs' "streaming only" framing, and stepMs bounds model generation rather than generation + tools (toolMs already owns the tool phase).
2. Add timeout.firstChunkMs — a per-model-call time-to-first-chunk bound. This is the highest-value timeout for production failover: provider errors (429/529) can be failed over to another model, but a provider that accepts the stream and never emits currently can't be distinguished from a slow prefill except by first-chunk deadline. Today implementing that requires wrapping/peeking the model stream in middleware.

Happy to contribute a PR for either direction if maintainers agree on the shape.

AI SDK Version

• ai: 7.0.28 (also verified on 7.0.22)
• @ai-sdk/provider: 4.x
• node: 26

[lgrammel]

Identify issue type

Identified as Bug
Confidence: High

Rationale

This is a concrete mismatch between documented timeout behavior and current execution: streaming timers abort requests during legitimate tool work. Although the issue also proposes firstChunkMs, the demonstrated existing behavior failure makes this a bug.

Evidence

• timeout.chunkMs is documented as aborting when no new streaming chunk arrives, but it remains active during tool execution.
• The reproduction shows an immediate model response followed by a healthy 300 ms tool execution being aborted with chunkMs: 150.
• The issue reports the same timer behavior for stepMs, despite toolMs separately controlling tool execution time.

[Divyansh151005]

I'll take this one.

Understanding: streamText keeps stepMs / chunkMs armed across client-side tool execution, so a healthy long-running tool can abort the stream even though toolMs already bounds that phase. firstChunkMs landed separately; the remaining gap is clearing the per-step model timers when the model call ends.

Approach:

• Clear step/chunk (and first-chunk) timers on model-call-end / model-stream error, before pending client tool work finishes
• Keep totalMs and toolMs active during tools
• Add regression coverage for stepMs and chunkMs not aborting during tool execution

I'll open a PR shortly.