Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/fix-tool-input-end-streaming.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
"@openrouter/ai-sdk-provider": patch
---

Fix missing `tool-input-end` event in multi-chunk and flush tool call streaming paths

The multi-chunk tool call merge path and the flush path for unsent tool calls were missing the `tool-input-end` event before emitting `tool-call`. This diverged from the stream event protocol used by `@ai-sdk/openai`, which consistently emits `tool-input-start → tool-input-delta → tool-input-end → tool-call`.

The flush path for unsent tool calls also now emits the full `tool-input-start → tool-input-delta → tool-input-end` sequence before `tool-call`, matching the reference implementation.
95 changes: 95 additions & 0 deletions e2e/issues/issue-413-tool-input-end-streaming.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
/**
* Regression test for GitHub issue #413
* https://github.com/OpenRouterTeam/ai-sdk-provider/issues/413
*
* Issue: "Tool call events are buffered until stream ends (flush) causing
* perceived streaming delay"
*
* The reporter observed that streaming tool calls with @openrouter/ai-sdk-provider
* were missing `tool-input-end` events in the multi-chunk tool call path, diverging
* from the protocol used by @ai-sdk/openai. The single-chunk path correctly emitted
* tool-input-start -> tool-input-delta -> tool-input-end -> tool-call, but the
* multi-chunk merge path skipped tool-input-end before tool-call.
*
* This test verifies that streamText with tool calls emits the complete event
* sequence including tool-input-end before tool-call.
*/
import { streamText, tool } from 'ai';
import { describe, expect, it, vi } from 'vitest';
import { z } from 'zod/v4';
import { createOpenRouter } from '@/src';

vi.setConfig({
testTimeout: 60_000,
});

describe('Issue #413: Tool call streaming should emit tool-input-end before tool-call', () => {
const openrouter = createOpenRouter({
apiKey: process.env.OPENROUTER_API_KEY,
baseUrl: `${process.env.OPENROUTER_API_BASE}/api/v1`,
});

it('should emit tool-input-end before tool-call with openai/gpt-4.1-nano', async () => {
const getWeather = tool({
description: 'Gets the current weather for a given city',
inputSchema: z.object({
city: z.string().describe('The city to get weather for'),
}),
execute: async ({ city }) => {
return { city, temperature: 72, condition: 'sunny' };
},
});

const response = streamText({
model: openrouter('openai/gpt-4.1-nano'),
prompt: 'What is the weather in San Francisco? Use the getWeather tool.',
tools: { getWeather },
toolChoice: 'required',
});

const events: string[] = [];
for await (const event of response.fullStream) {
events.push(event.type);
}

expect(events).toContain('tool-input-start');
expect(events).toContain('tool-input-end');
expect(events).toContain('tool-call');

const toolInputEndIndex = events.indexOf('tool-input-end');
const toolCallIndex = events.indexOf('tool-call');
expect(toolInputEndIndex).toBeLessThan(toolCallIndex);
});

it('should emit tool-input-end before tool-call with openai/gpt-4.1-mini', async () => {
const getWeather = tool({
description: 'Gets the current weather for a given city',
inputSchema: z.object({
city: z.string().describe('The city to get weather for'),
}),
execute: async ({ city }) => {
return { city, temperature: 72, condition: 'sunny' };
},
});

const response = streamText({
model: openrouter('openai/gpt-4.1-mini'),
prompt: 'What is the weather in San Francisco? Use the getWeather tool.',
tools: { getWeather },
toolChoice: 'required',
});

const events: string[] = [];
for await (const event of response.fullStream) {
events.push(event.type);
}

expect(events).toContain('tool-input-start');
expect(events).toContain('tool-input-end');
expect(events).toContain('tool-call');

const toolInputEndIndex = events.indexOf('tool-input-end');
const toolCallIndex = events.indexOf('tool-call');
expect(toolInputEndIndex).toBeLessThan(toolCallIndex);
});
});
4 changes: 4 additions & 0 deletions src/chat/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1626,6 +1626,10 @@ describe('doStream', () => {
id: 'call_O17Uplv4lJvD6DVdIvFFeRMw',
delta: '"}',
},
{
type: 'tool-input-end',
id: 'call_O17Uplv4lJvD6DVdIvFFeRMw',
},
{
type: 'tool-call',
toolCallId: 'call_O17Uplv4lJvD6DVdIvFFeRMw',
Expand Down
33 changes: 29 additions & 4 deletions src/chat/index.ts
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚩 Pre-existing: initial chunk arguments are not emitted as a delta in the multi-chunk path

When the first chunk of a tool call includes non-empty but non-parsable-JSON arguments (e.g., {"val), those arguments are stored in toolCall.function.arguments at line 889 but never emitted as a tool-input-delta. The tool-input-start and first tool-input-delta are only emitted on the second chunk (merge path, lines 968-987), which only includes the merge chunk's delta, not the initial arguments. In the existing test (src/chat/index.test.ts:1476), the initial chunk has arguments: "" (empty string), so this isn't visible. This is a pre-existing issue not introduced by this PR, but worth noting for correctness if providers ever send non-empty partial arguments in the first tool call chunk.

(Refers to lines 884-893)

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Original file line number Diff line number Diff line change
Expand Up @@ -992,6 +992,11 @@ export class OpenRouterChatLanguageModel implements LanguageModelV3 {
toolCall.function?.arguments != null &&
isParsableJson(toolCall.function.arguments)
) {
controller.enqueue({
type: 'tool-input-end',
id: toolCall.id,
});

// Only attach reasoning_details to the first tool call to avoid
// duplicating thinking blocks for parallel tool calls (Claude)
controller.enqueue({
Expand Down Expand Up @@ -1045,16 +1050,36 @@ export class OpenRouterChatLanguageModel implements LanguageModelV3 {
if (finishReason.unified === 'tool-calls') {
for (const toolCall of toolCalls) {
if (toolCall && !toolCall.sent) {
const toolInput = isParsableJson(toolCall.function.arguments)
? toolCall.function.arguments
: '{}';

if (!toolCall.inputStarted) {
controller.enqueue({
type: 'tool-input-start',
id: toolCall.id,
toolName: toolCall.function.name,
});
}

controller.enqueue({
type: 'tool-input-delta',
id: toolCall.id,
delta: toolInput,
});

controller.enqueue({
type: 'tool-input-end',
id: toolCall.id,
});

// Only attach reasoning_details to the first tool call to avoid
// duplicating thinking blocks for parallel tool calls (Claude)
controller.enqueue({
type: 'tool-call',
toolCallId: toolCall.id ?? generateId(),
toolName: toolCall.function.name,
// Coerce invalid arguments to an empty JSON object
input: isParsableJson(toolCall.function.arguments)
? toolCall.function.arguments
: '{}',
input: toolInput,
providerMetadata: !reasoningDetailsAttachedToToolCall
? {
openrouter: {
Expand Down