fix: preserve Gemini thought_signature in LiteLLM multi-turn tool calls#1982
fix: preserve Gemini thought_signature in LiteLLM multi-turn tool calls#1982giulio-leone wants to merge 2 commits intostrands-agents:mainfrom
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
/strands review |
|
Assessment: Comment This is a well-understood bugfix that correctly addresses the missing Review Details
The core fix logic is sound and the test suite covers the key scenarios well. |
|
@giulio-leone thanks for the PR! The review agent left some comments which would be good to address. Can you please check them and lint and rebase to pass the failing CI? |
When using Gemini thinking models (e.g., gemini-2.5-flash) through the LiteLLM model provider, multi-turn conversations with tool calls fail because thought_signature is lost during the response-to-request round trip. LiteLLM encodes Gemini's thought_signature into the tool call ID using a __thought__ separator. The OpenAI parent format_chunk passes this through as-is, but the signature is never extracted into Strands' reasoningSignature field, which the streaming layer already supports. Changes: - Override format_chunk in LiteLLMModel to detect __thought__ in tool call IDs and provider_specific_fields, extracting the signature into reasoningSignature for proper streaming layer storage - Override format_request_message_tool_call to re-encode reasoningSignature back into the tool call ID when it is not already present, ensuring LiteLLM can reconstruct the Gemini-native format - Add 7 unit tests covering extraction from ID, extraction from provider_specific_fields, no-signature passthrough, encoding, double-encode prevention, and full round-trip preservation Closes strands-agents#1764
004f0ba to
1d8b4dd
Compare
|
Assessment: Approve All three issues from the previous review have been addressed cleanly. The Review Details
Clean fix with solid test coverage. Nice work addressing the review feedback. |
1d8b4dd to
27e005f
Compare
|
Assessment: Approve All feedback from the previous two review rounds has been addressed. No new issues found. Review Details
|
Issue
Closes #1764
Description
When using Gemini thinking models (e.g.,
gemini-2.5-flash,gemini-3-flash) through the LiteLLM model provider, multi-turn conversations with tool calls fail with:Root Cause
Gemini thinking models require a
thought_signatureon each function call in multi-turn conversations. LiteLLM encodes this signature into the tool call ID using a__thought__separator (e.g.,call_abc__thought__base64sig). However, Strands'OpenAIModel.format_chunk()passes the encoded ID through without extracting the signature into thereasoningSignaturefield that the streaming layer already supports.While the encoded ID survives the round-trip in simple cases, the signature must also be explicitly stored in
reasoningSignatureso that:provider_specific_fields.thought_signature(an alternative LiteLLM transport) is also capturedChanges
src/strands/models/litellm.py:format_chunkto detect__thought__in tool call IDs and/orprovider_specific_fields.thought_signature, extracting the signature intoreasoningSignatureon thecontentBlockStarteventformat_request_message_tool_callto re-encodereasoningSignatureback into the tool call ID when not already present, ensuring LiteLLM can reconstruct the Gemini-native formattests/strands/models/test_litellm.py:provider_specific_fields, no-signature passthrough, signature encoding, double-encode prevention, and full round-trip preservationChecklist