Tool call events are buffered until stream ends (flush) causing perceived streaming delay

[metaslim]

Description

When using @openrouter/ai-sdk-provider with multi-step tool calling (e.g., via Vercel AI SDK's streamText + stepCountIs), the streaming response appears buffered to the end user. Text deltas stream fine, but something in the tool-call streaming path causes a perceived delay before the tool executes and the next LLM step begins.

Switching to @ai-sdk/openai with baseURL: 'https://openrouter.ai/api/v1' (same API, different SDK client) resolves the issue — streaming becomes real-time token-by-token.

Bug Found: Missing tool-input-end in multi-chunk tool call path

In src/chat/index.ts, the single-chunk tool call path (lines ~905-949) correctly emits the full sequence:

• tool-input-start → tool-input-delta → tool-input-end → tool-call

But the multi-chunk merge path (lines ~990-1013) skips tool-input-end:

• tool-input-delta → tool-call (missing tool-input-end before tool-call)

// Multi-chunk path — emits tool-call but NO tool-input-end before it
if (isParsableJson(toolCall.function.arguments)) {
  // ❌ Missing: controller.enqueue({ type: 'tool-input-end', id: toolCall.id });
  controller.enqueue({
    type: 'tool-call',
    toolCallId: toolCall.id ?? generateId(),
    toolName: toolCall.function.name,
    input: toolCall.function.arguments,
    ...
  });
  toolCall.sent = true;
}

This may cause downstream consumers (like the AI SDK's stream processor) to wait for the missing tool-input-end event, contributing to the perceived buffering.

Reproduction

1. Use @openrouter/ai-sdk-provider v2.x with ai SDK v6.x
2. Configure streamText() with tools and stepCountIs(n) for multi-step
3. Send a message that triggers a tool call where arguments arrive across multiple chunks
4. Observe that streaming feels delayed/buffered compared to @ai-sdk/openai

Workaround

Use @ai-sdk/openai with OpenRouter's base URL instead:

import { createOpenAI } from '@ai-sdk/openai';

const openrouterProvider = createOpenAI({
  apiKey: process.env.OPENROUTER_API_KEY,
  baseURL: 'https://openrouter.ai/api/v1',
});

// Force chat completions API (not Responses API)
export const openrouter = (modelId: string) => openrouterProvider.chat(modelId);

Environment

• `@openrouter/ai-sdk-provider`: 2.2.3
• `ai`: 6.0.81
• Node.js: 24
• Model tested: `openai/gpt-5.2`

[metaslim]

Additional testing: We disabled Braintrust's wrapAISDK (which wraps the AI SDK with a TracingTransformStream + tee()) and tested with the OpenRouter SDK alone — the buffering still occurs intermittently.

This confirms the issue is within @openrouter/ai-sdk-provider's stream transform itself, not caused by downstream wrapping or middleware.

Switching to @ai-sdk/openai with baseURL: 'https://openrouter.ai/api/v1' (same API, simpler transform) consistently streams token-by-token without buffering.

[metaslim]

Payload size comparison: We compared the same chat request using both @openrouter/ai-sdk-provider and @ai-sdk/openai (with baseURL: 'https://openrouter.ai/api/v1'), same model (openai/gpt-5.2), same parameters.

Raw HTTP to/from OpenRouter API (server-side)

Call	OpenRouter SDK Request	OpenAI SDK Request
1st LLM call	44.1KB	44.1KB
Title generation	0.4KB	0.4KB
Query rephrase	3.3KB	3.3KB
2nd LLM call (with tool results)	79.1KB	72.3KB

Raw HTTP responses from OpenRouter's API are similar (~158KB vs ~170KB). The requests are nearly identical — same model, same params.

SSE payload to browser (what the user sees)

SDK	Final SSE payload size
@openrouter/ai-sdk-provider	145KB
@ai-sdk/openai	59KB

The OpenRouter SDK produces a 2.5x larger SSE payload to the browser despite receiving similar raw data from the API. The SDK's transform emits additional stream events (response-metadata, reasoning-start/end markers, extra tool-input events, source events, provider metadata) that inflate the toUIMessageStreamResponse() output. More bytes = more for the browser to parse = perceived buffering/delay in streaming.

This combined with the heavier per-chunk processing in the transform (400+ lines vs ~50 lines in @ai-sdk/openai) explains the streaming latency.

[metaslim]

Root cause of SSE payload bloat: reasoning metadata on every event

Digging deeper into why the SSE payload is 145KB vs 59KB, the primary culprit is reasoning metadata that the OpenRouter SDK injects into the stream.

Per-event comparison

@openrouter/ai-sdk-provider text-delta:

{"type":"text-delta","id":"gen-1770863686-MDnBuLhPEGx96cDQcgK4","delta":"Open"}

@ai-sdk/openai text-delta:

{"type":"text-delta","id":"0","delta":"Open"}

Reasoning events (OpenRouter SDK only)

The SDK emits reasoning-start, reasoning-delta, and reasoning-end events (~90+ events per response). Each reasoning-delta carries a providerMetadata object:

{
  "type": "reasoning-delta",
  "id": "gen-1770863677-xSrTbsu1iIQ24tecRRU5",
  "delta": "word",
  "providerMetadata": {
    "openrouter": {
      "reasoning_details": [
        {"type": "reasoning.summary", "summary": "word", "format": "openai-responses-v1", "index": 0}
      ]
    }
  }
}

The real cost: repeated arrays + encrypted blocks

The reasoning-end and tool-input-available events repeat the entire reasoning summary array (all ~90 entries) plus an encrypted reasoning block (~3KB+):

{
  "type": "reasoning.encrypted",
  "data": "gAAAAABpjTxC-MLpmwnDJ7N6JLm01GKbjmN...  (3KB+)",
  "id": "rs_0de7edf5ea7387b001698d3c...",
  "format": "openai-responses-v1",
  "index": 0
}

This pattern appears on both the reasoning-end event AND the tool-input-available event, doubling the cost.

Event count comparison (same query, same model)

Event type	OpenRouter SDK	OpenAI SDK
reasoning-start/delta/end	~90+	0
tool-input-* events	~22 + providerMetadata	~22 clean
text-delta events	~180	~180
Total SSE lines	~300+	~210

No documented way to disable

We attempted providerOptions: { openrouter: { reasoning: { exclude: true } } } but there doesn't appear to be a documented way to prevent the SDK from attaching providerMetadata to stream events. The bloat is in the SDK's transform layer, not in the raw API response from OpenRouter.

Conclusion

The @ai-sdk/openai + baseURL workaround avoids this entirely by using the chat completions API format, which doesn't surface reasoning events. The functional response content is identical — same model, same tools, same quality — just 57KB instead of 140KB over the wire.

[softmarshmallow]

Same issue, even setting https://github.com/OpenRouterTeam/ai-sdk-provider?tab=readme-ov-file#fine-grained-tool-streaming

Tried with suggested solution (openai + openrouter base url), seems not to work

[robert-j-y]

Thank you for opening this issue, I will look at this today.

[shijinkrishna]

Same issue, even setting https://github.com/OpenRouterTeam/ai-sdk-provider?tab=readme-ov-file#fine-grained-tool-streaming

Tried with suggested solution (openai + openrouter base url), seems not to work

I am also facing the same issue. The issue is only with Anthropic models