Skip to content

fix(stream): preserve non-negative tool call indexes#247

Open
bokmann wants to merge 1 commit into
aws-samples:mainfrom
bokmann:fix/stream-tool-call-index
Open

fix(stream): preserve non-negative tool call indexes#247
bokmann wants to merge 1 commit into
aws-samples:mainfrom
bokmann:fix/stream-tool-call-index

Conversation

@bokmann
Copy link
Copy Markdown

@bokmann bokmann commented May 15, 2026

Summary

  • fix OpenAI-compatible streaming tool-call indexes so they stay non-negative
  • map Bedrock contentBlockIndex values to zero-based tool-call ordinals instead of subtracting 1
  • add a regression test covering the first streamed tool call and multiple tool calls in one stream

Problem

Streaming tool-call chunks were being emitted with tool_calls[].index = -1 for the first tool call. Strict OpenAI-compatible clients reject that shape and fail during streamed tool use.

I reproduced this against a live deployment with a request like:

{
  "model": "us.anthropic.claude-sonnet-4-6",
  "stream": true,
  "stream_options": {"include_usage": true},
  "messages": [{"role": "user", "content": "Use a tool call. Return weather for Boston."}],
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get weather",
      "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"]
      }
    }
  }]
}

Before this patch, the stream contained chunks like:

{"delta":{"tool_calls":[{"index":-1,"id":"tooluse_...","type":"function","function":{"name":"get_weather","arguments":""}}]}}

That matches the existing negative-index bug report in #203.

Fix

The converter was using contentBlockIndex - 1 as the OpenAI tool_calls[].index. That is not safe, because contentBlockIndex is a content-block position, not a tool-call ordinal.

This patch keeps a per-stream mapping from Bedrock contentBlockIndex to the next zero-based tool-call ordinal and reuses that mapping for later deltas from the same tool call.

Validation

Ran the focused regression test inside the project runtime image:

docker run --rm \
  -e PYTHONPATH=/work/src \
  -v /tmp/bedrock-access-gateway-bokmann:/work \
  -w /work \
  --entrypoint python \
  919493069736.dkr.ecr.us-east-1.amazonaws.com/bedrock-proxy-api-fabro@sha256:2ef96c1248daee22db1eee404f3fe15e55d728dc937251730e4b64a1e41f5f4e \
  -m unittest tests.test_bedrock_stream_tool_call_index -v

Fixes #203.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tthe index value is negative, violating the ensureNotNegative validation rule and causing the program to crash.

1 participant