Skip to content

[bot] Anthropic streaming accumulator mislabels server_tool_use/mcp_tool_use content blocks as generic tool_use #210

Description

@braintrust-bot

<!-- provider-gap-audit: anthropic-claudestream-server-tool-use-mislabeled -->

What instrumentation is missing (or, in this case, incorrect)

The Anthropic streaming accumulator overwrites the traced type of every content block that receives an input_json_delta event, always setting it to the literal string "tool_use" — even when the block was originally started as server_tool_use (server-side tools, e.g. web search, code execution) or mcp_tool_use (MCP connector tool calls). This causes streamed traces in Braintrust to misreport which kind of tool call actually happened, collapsing three semantically distinct block types into one.

Where it breaks

trace/internal/claudestream.go, addDelta (lines 90-97):

case "input_json_delta":
	partial, ok := delta["partial_json"].(string)
	if !ok {
		return false
	}
	block["type"] = "tool_use"   // always overwrites, regardless of original block type
	builder.WriteString(partial)
	return partial != ""

content_block_start (lines 41-49) correctly seeds block with the provider's original type (tool_use, server_tool_use, or mcp_tool_use) via cloneClaudeMap. But every subsequent input_json_delta for that block index unconditionally clobbers it back to "tool_use", discarding the original classification before Output() (lines 122-152) renders the final assistant message that gets attached to the span.

By contrast, the non-streaming path (trace/contrib/anthropic/messages.go) passes the API's content array through untouched, so a non-streaming response correctly preserves server_tool_use/mcp_tool_use types. Only the streaming path introduces this data-correctness bug.

Braintrust docs status: unclear

https://www.braintrust.dev/docs/integrations/ai-providers/anthropic confirms server_tool_use is tracked as a metric ("Server-side tool usage counters (for example, server_tool_use_web_search_requests)") and states "every messages.create call (including streaming) emits a span," but does not document how tool-use content blocks are labeled during streaming specifically. The docs don't explicitly promise the block type is preserved verbatim, but the existence of dedicated server_tool_use/mcp_tool_use metrics implies the type should stay consistent between streaming and non-streaming traces — which it currently does not.

Upstream sources

Local repo files inspected

  • trace/internal/claudestream.goAdd, addDelta (lines 31-118), Output (lines 122-152)
  • trace/contrib/anthropic/messages.go — non-streaming content handling (no equivalent rewrite; used as the correct-behavior reference)
  • trace/contrib/anthropic/traceanthropic.goparseUsageTokens (confirms server_tool_use usage metrics are otherwise tracked elsewhere in the integration)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions