Skip to content

fix(core,openai): fix v1 streaming tool calls dropped for chat completions#35983

Open
Mason Daugherty (mdrxy) wants to merge 3 commits intomasterfrom
mdrxy/fix-completions
Open

fix(core,openai): fix v1 streaming tool calls dropped for chat completions#35983
Mason Daugherty (mdrxy) wants to merge 3 commits intomasterfrom
mdrxy/fix-completions

Conversation

@mdrxy
Copy link
Member

@mdrxy Mason Daugherty (mdrxy) commented Mar 17, 2026

Streaming with output_version="v1" and bound tools silently dropped tool calls from content_blocks on the merged AIMessageChunk.

Two root causes:

  1. BaseChatOpenAI._convert_chunk_to_generation_chunk set content=[] on the usage-only chunk while content chunks used content="", corrupting the merged content via string/list type mismatch
  2. content_blocks short-circuited for output_version="v1" even when content was a raw string rather than a list of ContentBlock dicts, returning the string directly instead of routing through the model_provider translator.

Changes

  • Remove the content = [] override on usage-only (empty-choices) chunks in BaseChatOpenAI._convert_chunk_to_generation_chunk, keeping content as "" so all Chat Completions streaming chunks have consistent string content through merge
  • Propagate output_version="v1" to content chunks (non-empty choices) in _convert_chunk_to_generation_chunk — previously only the usage-only chunk carried it
  • Guard the v1 short-circuit in both AIMessage.content_blocks and AIMessageChunk.content_blocks with isinstance(self.content, list) so string content falls through to the model_provider translator, which correctly builds ContentBlock dicts from tool_calls/tool_call_chunks

Revert basetenlabs/langchain-baseten#6 when released

@github-actions github-actions bot added size: M 200-499 LOC internal fix For PRs that implement a fix integration PR made that is related to a provider partner package integration core `langchain-core` package issues & PRs openai `langchain-openai` package issues & PRs and removed fix For PRs that implement a fix labels Mar 17, 2026
@codspeed-hq
Copy link

codspeed-hq bot commented Mar 17, 2026

Merging this PR will improve performance by 36.7%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 13 improved benchmarks
✅ 6 untouched benchmarks
⏩ 17 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
WallTime test_import_time[PydanticOutputParser] 571.8 ms 485.7 ms +17.73%
WallTime test_import_time[InMemoryRateLimiter] 182.8 ms 157.5 ms +16.03%
WallTime test_import_time[LangChainTracer] 475.3 ms 416.1 ms +14.23%
WallTime test_import_time[tool] 569.9 ms 480.9 ms +18.51%
WallTime test_import_time[BaseChatModel] 556 ms 478.6 ms +16.18%
WallTime test_import_time[Document] 196.2 ms 166.1 ms +18.11%
WallTime test_async_callbacks_in_sync 25.8 ms 18.9 ms +36.7%
WallTime test_import_time[HumanMessage] 274.5 ms 237.6 ms +15.56%
WallTime test_import_time[RunnableLambda] 526.3 ms 439.8 ms +19.67%
WallTime test_import_time[ChatPromptTemplate] 668.1 ms 562.1 ms +18.86%
WallTime test_import_time[Runnable] 521.5 ms 444.6 ms +17.29%
WallTime test_import_time[CallbackManager] 329.8 ms 286.3 ms +15.19%
WallTime test_import_time[InMemoryVectorStore] 639.9 ms 535.6 ms +19.47%

Comparing mdrxy/fix-completions (5ce9944) with master (69a7b9c)2

Open in CodSpeed

Footnotes

  1. 17 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on master (2bad58a) during the generation of this report, so 69a7b9c was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core `langchain-core` package issues & PRs fix For PRs that implement a fix integration PR made that is related to a provider partner package integration internal openai `langchain-openai` package issues & PRs size: M 200-499 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant