Skip to content

fix: preserve Gemini thought_signature in LiteLLM multi-turn tool calls#1982

Open
giulio-leone wants to merge 2 commits intostrands-agents:mainfrom
giulio-leone:fix/litellm-thought-signature-preservation
Open

fix: preserve Gemini thought_signature in LiteLLM multi-turn tool calls#1982
giulio-leone wants to merge 2 commits intostrands-agents:mainfrom
giulio-leone:fix/litellm-thought-signature-preservation

Conversation

@giulio-leone
Copy link
Copy Markdown
Contributor

Issue

Closes #1764

Description

When using Gemini thinking models (e.g., gemini-2.5-flash, gemini-3-flash) through the LiteLLM model provider, multi-turn conversations with tool calls fail with:

function call 'current_time' in the 2. content block is missing a thought_signature

Root Cause

Gemini thinking models require a thought_signature on each function call in multi-turn conversations. LiteLLM encodes this signature into the tool call ID using a __thought__ separator (e.g., call_abc__thought__base64sig). However, Strands' OpenAIModel.format_chunk() passes the encoded ID through without extracting the signature into the reasoningSignature field that the streaming layer already supports.

While the encoded ID survives the round-trip in simple cases, the signature must also be explicitly stored in reasoningSignature so that:

  1. The streaming infrastructure can properly track it through content block lifecycle
  2. The signature can be re-encoded into the tool call ID even when the ID is cleaned or regenerated
  3. provider_specific_fields.thought_signature (an alternative LiteLLM transport) is also captured

Changes

src/strands/models/litellm.py:

  • Override format_chunk to detect __thought__ in tool call IDs and/or provider_specific_fields.thought_signature, extracting the signature into reasoningSignature on the contentBlockStart event
  • Override format_request_message_tool_call to re-encode reasoningSignature back into the tool call ID when not already present, ensuring LiteLLM can reconstruct the Gemini-native format

tests/strands/models/test_litellm.py:

  • 7 new unit tests covering: extraction from encoded ID, extraction from provider_specific_fields, no-signature passthrough, signature encoding, double-encode prevention, and full round-trip preservation

Checklist

  • I've read CONTRIBUTING.md
  • I've updated or added tests to cover changes
  • I've verified all existing tests pass (48/48)
  • This change is backward compatible (no breaking changes)

⚠️ This reopens #1888 which was accidentally closed due to fork deletion.

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 7, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@opieter-aws
Copy link
Copy Markdown
Contributor

/strands review

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 7, 2026

Assessment: Comment

This is a well-understood bugfix that correctly addresses the missing thought_signature preservation for Gemini thinking models via LiteLLM. The approach aligns well with the existing pattern used by the native Gemini provider (gemini.py), and the round-trip test demonstrates the fix works end-to-end.

Review Details
  • Unrelated changes: The PR includes a pyproject.toml dependency bound change (litellm <=1.82.6<2.0.0) and a tests_integ/conftest.py change (adding COHERE_API_KEY) that are unrelated to the thought_signature fix and should be split into separate PRs.
  • Code organization: The thought signature extraction block in format_chunk is ~30 lines and could benefit from extraction into a private helper method for readability and testability.
  • Test coverage gap: The function-level provider_specific_fields extraction path (line 295) lacks a dedicated test — all three signature sources should have coverage.
  • Documentation: No public API changes — no docs PR needed.

The core fix logic is sound and the test suite covers the key scenarios well.

@opieter-aws
Copy link
Copy Markdown
Contributor

@giulio-leone thanks for the PR! The review agent left some comments which would be good to address. Can you please check them and lint and rebase to pass the failing CI?

When using Gemini thinking models (e.g., gemini-2.5-flash) through the
LiteLLM model provider, multi-turn conversations with tool calls fail
because thought_signature is lost during the response-to-request
round trip.

LiteLLM encodes Gemini's thought_signature into the tool call ID using
a __thought__ separator. The OpenAI parent format_chunk passes this
through as-is, but the signature is never extracted into Strands'
reasoningSignature field, which the streaming layer already supports.

Changes:
- Override format_chunk in LiteLLMModel to detect __thought__ in tool
  call IDs and provider_specific_fields, extracting the signature into
  reasoningSignature for proper streaming layer storage
- Override format_request_message_tool_call to re-encode
  reasoningSignature back into the tool call ID when it is not already
  present, ensuring LiteLLM can reconstruct the Gemini-native format
- Add 7 unit tests covering extraction from ID, extraction from
  provider_specific_fields, no-signature passthrough, encoding,
  double-encode prevention, and full round-trip preservation

Closes strands-agents#1764
@opieter-aws opieter-aws force-pushed the fix/litellm-thought-signature-preservation branch from 004f0ba to 1d8b4dd Compare April 10, 2026 09:42
@github-actions
Copy link
Copy Markdown

Assessment: Approve

All three issues from the previous review have been addressed cleanly. The _extract_thought_signature helper is well-structured with clear priority ordering and early returns, and the function-level PSF test fills the coverage gap. Only one minor inconsistency remains (decorator ordering).

Review Details
  • Previous feedback: All 3 prior comments addressed — extraction refactored to @staticmethod helper, function-level PSF test added, fallback comment reworded.
  • New finding: Minor decorator order inconsistency (@classmethod @override vs file convention @override @classmethod) — non-blocking.
  • Rebase note: Branch is behind main. When rebasing, ensure the litellm<=1.82.6 security pin from fix: CRITICAL: Hard pin litellm<=1.82.6 to mitigate supply chain attack #1961 is preserved (do not revert to <2.0.0).

Clean fix with solid test coverage. Nice work addressing the review feedback.

@opieter-aws opieter-aws force-pushed the fix/litellm-thought-signature-preservation branch from 1d8b4dd to 27e005f Compare April 10, 2026 10:02
@github-actions github-actions bot added size/m and removed size/m labels Apr 10, 2026
@opieter-aws opieter-aws deployed to manual-approval April 10, 2026 10:02 — with GitHub Actions Active
@github-actions
Copy link
Copy Markdown

Assessment: Approve

All feedback from the previous two review rounds has been addressed. No new issues found.

Review Details
  • Prior feedback (4 items): All resolved — extraction helper refactored to @staticmethod, function-level PSF test added, fallback comment reworded, decorator order fixed.
  • Code quality: Clean implementation with defensive typing, clear priority ordering in _extract_thought_signature, and proper delegation to the parent OpenAIModel.
  • Data flow: Verified the full round-trip: format_chunk extracts signature → streaming layer preserves reasoningSignatureformat_request_message_tool_call re-encodes it → tool result IDs match via preserved toolUseId.
  • Testing: 8 tests covering all extraction paths, encoding/decoding, double-encode prevention, and end-to-end round-trip. Codecov confirms full coverage.
  • Documentation: Bugfix with no public API changes — no docs PR needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] missing thought signature preservation for thinking models with LiteLLM Model Provider

2 participants