Skip to content

fix: noisy Gemini 3 parallel tool-call replay warnings when only the first call in a batch has a thought signature#16986

Open
lgrammel wants to merge 4 commits into
mainfrom
bug-16298-2
Open

fix: noisy Gemini 3 parallel tool-call replay warnings when only the first call in a batch has a thought signature#16986
lgrammel wants to merge 4 commits into
mainfrom
bug-16298-2

Conversation

@lgrammel

@lgrammel lgrammel commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Background

Gemini 3 parallel tool-call replays can legitimately include one signed function call followed by unsigned calls, but the converter treated each unsigned call as dropped metadata and emitted noisy skip-signature warnings.

Summary

The Google message converter now tracks contiguous assistant tool-call batches and skips the Gemini 3 missing-signature sentinel for unsigned calls that follow a real signed call in the same batch.

Testing

Updated the focused converter regression to cover a signed-first/unsigned-second Gemini 3 parallel tool-call batch without relying on the reproduction fixture, and updated the reproduction example to assert the fixed no-warning outcome.

End-to-end Validation

Example examples/ai-functions/src/reproduction/issue-16298-gemini3-parallel-tool-calls.ts; live API command pnpm -C examples/ai-functions exec tsx src/reproduction/issue-16298-gemini3-parallel-tool-calls.ts; observed two parallel get_weather calls with one present thought signature, one absent thought signature, WARNINGS: [], and fixed: true.

Related Issues

Fixes #16298

Co-authored-by: mttzzz 16290052+mttzzz@users.noreply.github.com
Co-authored-by: lgrammel 205036+lgrammel@users.noreply.github.com

@lgrammel lgrammel changed the title reproduction: Gemini 3 parallel function calls: converter injects skipthoughtsignaturevalidator fix: Gemini 3 parallel tool-call batches warn and inject skip_thought_signature_validator for valid unsigned sibling calls Jul 9, 2026
@lgrammel lgrammel marked this pull request as ready for review July 9, 2026 11:59
@lgrammel

lgrammel commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator Author

Test and verify with 3 parallel tool calls.

@lgrammel lgrammel changed the title fix: Gemini 3 parallel tool-call batches warn and inject skip_thought_signature_validator for valid unsigned sibling calls fix: Gemini 3 parallel function-call replay warns and injects skip_thought_signature_validator for valid unsigned calls after a signed batch call Jul 9, 2026
@lgrammel

lgrammel commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed PR comment 4924897879.

Test and verify with 3 parallel tool calls.

Expanded the converter regression and reproduction script from two to three parallel get_weather calls. The live reproduction now asserts at least three calls with one signed and two unsigned parts, and it passed with no skip_thought_signature_validator warning.

@lgrammel

lgrammel commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator Author

In the end-to-end-verification, test that things work in multi-step roundtrips, i.e. when there are tool results for 3 parallel calls and the another llm request (e.g. via setting stopWhen: to like 5 steps)

@lgrammel lgrammel changed the title fix: Gemini 3 parallel function-call replay warns and injects skip_thought_signature_validator for valid unsigned calls after a signed batch call fix: noisy Gemini 3 parallel tool-call replay warnings when only the first call in a batch has a thought signature Jul 9, 2026
@lgrammel

lgrammel commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed PR comment 4924897879.

Test and verify with 3 parallel tool calls.

The converter regression and reproduction remain expanded to three parallel get_weather calls. I also strengthened the live reproduction so the three-call case is validated through tool-result replay and a follow-up LLM step.

@lgrammel

lgrammel commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed PR comment 4925008508.

Addressed PR comment 4924897879.

Test and verify with 3 parallel tool calls.

Expanded the converter regression and reproduction script from two to three parallel get_weather calls. The live reproduction now asserts at least three calls with one signed and two unsigned parts, and it passed with no skip_thought_signature_validator warning.

Confirmed. I kept the three parallel-call coverage in place and built on it by asserting the live run includes matching tool results and final text from the subsequent model request.

@lgrammel

lgrammel commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed PR comment 4925319502.

In the end-to-end-verification, test that things work in multi-step roundtrips, i.e. when there are tool results for 3 parallel calls and the another llm request (e.g. via setting stopWhen: to like 5 steps)

Updated the reproduction to use stopWhen: stepCountIs(5), require at least three parallel tool calls with matching tool results, and assert that a later LLM step produces final text. The live run completed with 2 steps, 3 tool calls, 3 tool results, one signed call, two unsigned calls, no skip_thought_signature_validator warning, and fixed: true.

@lgrammel

lgrammel commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator Author

Bugfix review

Outcome: approved

Fixes issue

Status: fully-addresses

The PR directly fixes the reported noisy Gemini 3 parallel tool-call replay path: convertToGoogleMessages now tracks contiguous assistant tool-call batches and skips sentinel injection/warnings for unsigned later calls after a real signed call in the same batch.

Concerns: none.

Side effects

Risk: low

The behavior change is narrowly scoped to Gemini 3 missing-thoughtSignature mitigation for later tool calls in the same contiguous assistant tool-call batch; standalone or fully unsigned tool calls still inject the sentinel and warn as before.

Concerns: none.

Performance

Risk: low

The implementation adds only a boolean flag while mapping assistant content parts, with no meaningful memory or runtime overhead.

Concerns: none.

Backwards compatibility

Risk: low

No stored data is mutated. Replayed stored transcripts with one signed parallel call followed by unsigned calls now serialize without the sentinel for the unsigned calls, which is the intended fix; fully missing-signature transcripts retain the prior mitigation.

Concerns: none.

Security

Risk: none

The change does not add new parsing, network access, persistence, or trust boundaries; it only changes whether a documented sentinel string is added to selected Gemini 3 tool-call parts.

Concerns: none.

Testing

Status: appropriate

The PR adds a focused regression test for a Gemini 3 batch with three parallel tool calls where only the first has a thoughtSignature, and existing tests still cover missing-signature sentinel injection and warning behavior.

Concerns: none.

Verification

Inspected git diff origin/main...HEAD; reviewed the converter, regression test, reproduction example, and changeset. Ran pnpm -C packages/google test:node src/convert-to-google-messages.test.ts, pnpm -C packages/google test:edge src/convert-to-google-messages.test.ts, pnpm -C packages/google type-check, and pnpm -C examples/ai-functions type-check; all passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gemini 3 parallel function calls: converter injects skip_thought_signature_validator + warns every turn (only the first parallel call is signed)

1 participant