thought_signature

[mohamedDev]

Hello,

With ai-proxy, since the OpenAI-compatible API does not expose an equivalent field, the gateway cannot currently generate or reconstruct this value automatically during request translation.

Supporting this would require provider-specific state management within the gateway to capture the thought_signature from Gemini responses and re-inject it into subsequent tool calls, which is not supported today.

@dosu, how can we move forward?

[dosubot[bot]]

Hey @mohamedDev,

You've correctly identified the core challenge — the ai-proxy plugin is stateless by design, processing each request independently with no session-level context preserved between calls [1]. There's no mechanism today to capture a field like thought_signature from a Gemini response and automatically re-inject it into a subsequent request.

Here are the potential paths forward:

1. Client-side management (workaround, works today)
The simplest approach: have your client capture thought_signature from Gemini responses and include it in the next request body. Since model_options allows arbitrary key-value pairs via additionalProperties = true [2], and the plugin merges these into the upstream request [3], the field can pass through to Gemini if the client includes it explicitly.

2. Provider-specific hook in the newer architecture
The plugin's recent three-layer architecture [4] introduced provider-aware rewrite_request_body hooks [5] and explicit field whitelisting [6]. A Gemini-specific hook could potentially be written to handle thought_signature mapping — but this still wouldn't solve the statefulness problem since hooks operate per-request.

3. Feature request for stateful session support
Supporting this properly would require a new capability: request-scoped or session-scoped state storage within the gateway (e.g., using shared memory or an external store like Redis) to persist provider-specific response fields and inject them into subsequent requests. This would be a significant architectural addition.

If you'd like to move this forward as a feature, I'd suggest opening a GitHub issue describing the use case (Gemini tool calls requiring thought_signature continuity) and proposing either option 2 or 3. There are no existing issues or PRs tracking this specific need today.

In the meantime, option 1 (client-side management) is the most practical path.

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.

[janiussyafiq]

Hi @mohamedDev, no gateway-side state is needed here, and I could not reproduce a gap in APISIX. Two clarifications first:

1. ai-proxy does not translate between the OpenAI format and Gemini's native generateContent format. The gemini provider proxies to Google's own OpenAI-compatible endpoint (/v1beta/openai/chat/completions).
2. That compatibility layer already round-trips thought signatures statelessly: responses carry them at tool_calls[].extra_content.google.thought_signature, and the client sends them back unchanged in the next request's history. See https://ai.google.dev/gemini-api/docs/openai

Environment

• APISIX 3.17.0 (apache/apisix:3.17.0-debian, standalone mode), gateway on :9080
• Model: gemini-3-flash-preview

Route

routes:
  - uri: /chat
    plugins:
      ai-proxy:
        provider: gemini
        auth:
          header:
            Authorization: "Bearer $GEMINI_API_KEY"
#END

Step 1: tool call through APISIX, signature comes back

curl -s http://127.0.0.1:9080/chat -H 'Content-Type: application/json' -d '{
  "model": "gemini-3-flash-preview",
  "messages": [{"role":"user","content":"What is the weather in Kuala Lumpur right now? You must use the get_weather tool."}],
  "tools": [{"type":"function","function":{"name":"get_weather","description":"Get current weather for a city",
    "parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}]
}'

Response (HTTP 200):

{
  "id": "GQ96apaKJJXIjuMP-N2s2Q4",
  "model": "gemini-3-flash-preview",
  "choices": [{
    "finish_reason": "tool_calls",
    "index": 0,
    "message": {
      "role": "assistant",
      "tool_calls": [{
        "extra_content": {
          "google": { "thought_signature": "EuYCCuMCARFNMg9ubbt9iH0M...<truncated>" }
        },
        "function": { "arguments": "{\"city\":\"Kuala Lumpur\"}", "name": "get_weather" },
        "id": "3qslm5ut",
        "type": "function"
      }]
    }
  }]
}

Identical shape to calling Google directly (verified as a control). With "stream": true the signature also arrives through APISIX in the SSE delta:

data: {"choices":[{"delta":{"role":"assistant","tool_calls":[{"extra_content":{"google":{"thought_signature":"EtkCCtYC...<truncated>"}},...

Step 2: turn 2 through APISIX with the signature echoed, works

curl -s http://127.0.0.1:19080/chat -H 'Content-Type: application/json' -d '{
  "model": "gemini-3-flash-preview",
  "messages": [
    {"role":"user","content":"What is the weather in Kuala Lumpur right now? You must use the get_weather tool."},
    {"role":"assistant","tool_calls":[{"id":"3qslm5ut","type":"function",
      "function":{"name":"get_weather","arguments":"{\"city\":\"Kuala Lumpur\"}"},
      "extra_content":{"google":{"thought_signature":"EuYCCuMCARFNMg9ubbt9iH0M...<full value from step 1>"}}}]},
    {"role":"tool","tool_call_id":"3qslm5ut","content":"{\"temp_c\": 31, \"condition\": \"thunderstorm\"}"}
  ],
  "tools": [{"type":"function","function":{"name":"get_weather","description":"Get current weather for a city",
    "parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}]
}'

Response (HTTP 200): "content": "The weather in Kuala Lumpur is currently 31°C with thunderstorms."

Step 3: same request with extra_content removed, fails

Same command as step 2, with the extra_content object deleted from the assistant's tool call. Response (HTTP 400, from Gemini):

[{"error":{"code":400,"message":"Function call is missing a thought_signature in functionCall parts. This is required for tools to work correctly, and missing thought_signature may lead to degraded model performance. Additional data, function call `default_api:get_weather` , position 2. Please refer to https://ai.google.dev/gemini-api/docs/thought-signatures for more details.","status":"INVALID_ARGUMENT"}}]

Conclusion

The signature flows through APISIX untouched in both directions (non-streaming and streaming). Step 3 is the only failure I could produce, and that request left the gateway exactly as the client sent it, so the 400 means the field was missing from the client's request. Several OpenAI SDK wrappers rebuild message history from typed objects and drop unknown fields like extra_content (for example openai/openai-agents-python#2137); that is worth checking first.

If you can capture a request where the signature enters APISIX but does not reach Gemini, please share it along with your APISIX version and we will treat it as a gateway bug.

[mohamedDev]

Thanks for the clarification. That was very helpful.