[BUG]: `invoke_model_with_response_stream` token metrics are all 0 in LLMObs traces

[jpr175]

Tracer Version(s)

4.10.4

Python Version(s)

3.12.13

Pip Version(s)

uv 0.11.24

Bug Report

When calling invoke_model_with_response_stream with the Bedrock Runtime client, the LLMObs span records all token usage metrics as 0. The trace tag cost_estimate_status:skipped_tokens_missing confirms that the integration detects this gap but does not recover from it.

The token data is present in the stream — it arrives in two events that ddtrace's patch appears to miss:

• message_start → message.usage — contains input token count
• message_delta → usage — contains output token count

Reproduction Code

import json
import boto3
from ddtrace import patch
from ddtrace.llmobs import LLMObs

# 1. Patch botocore to intercept Bedrock requests
patch(botocore=True)

def main():
    # Configure the Bedrock Runtime client
    LLMObs.enable()
    client = boto3.client("bedrock-runtime", region_name="eu-central-1")
    model_id = "eu.anthropic.claude-haiku-4-5-20251001-v1:0"

    # Standard payload for the Claude 3 family
    body = {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 150,
        "messages": [
            {
                "role": "user",
                "content": "Explain what observability is in one sentence."
            }
        ]
    }

    print(f"Calling invoke_model_with_response_stream API ({model_id})...\n")

    # Call the API in stream mode
    response = client.invoke_model_with_response_stream(
        modelId=model_id,
        contentType="application/json",
        accept="application/json",
        body=json.dumps(body)
    )

    # Process the stream
    stream = response.get("body")
    if stream:
        for event in stream:
            chunk = event.get("chunk")
            if chunk:
                # Decode the binary chunk
                chunk_data = json.loads(chunk.get("bytes").decode("utf-8"))

                # Print the text as it streams
                if chunk_data.get("type") == "content_block_delta":
                    print(chunk_data["delta"]["text"], end="", flush=True)

                # Highlight the arrival of usage metrics in the stream.
                # This is the part that ddtrace generally misses when patching the stream.
                elif chunk_data.get("type") == "message_start":
                    print(f"\n\n[AWS INFO] Input tokens: {chunk_data['message']['usage']}")
                elif chunk_data.get("type") == "message_delta":
                    print(f"\n[AWS INFO] Output tokens: {chunk_data['usage']}")

                # AWS also adds its own metrics at the end (amazon-bedrock-invocationMetrics)
                if "amazon-bedrock-invocationMetrics" in chunk_data:
                    print(f"\n[AWS INFO] Bedrock Metrics: {chunk_data['amazon-bedrock-invocationMetrics']}")

    # Force sending traces to Datadog before exiting
    LLMObs.flush()
    print("\n\nExecution completed. Check the Datadog LLMObs dashboard.")

if __name__ == "__main__":
    main()

trace_6a2bd0a4.json

Error Logs

No response

Libraries in Use

Using CPython 3.12.13
Creating virtual environment at: .venv
Resolved 14 packages in 3ms
Installed 13 packages in 93ms

• boto3==1.43.36
• botocore==1.43.36
• bytecode==0.18.1
• ddtrace==4.10.5
• envier==0.6.1
• jmespath==1.1.0
• opentelemetry-api==1.43.0
• python-dateutil==2.9.0.post0
• s3transfer==0.19.0
• six==1.17.0
• typing-extensions==4.15.0
• urllib3==2.7.0
• wrapt==2.2.2

Operating System

Darwin Kernel Version 25.5.0