fix(otel): label retrieval and agent metrics correctly and emit gen_ai.provider.name

[yassin-berriai]

TLDR

Problem this solves:

• Every vector-store and agent metric was labelled operation.name=chat
• Provider rode the deprecated gen_ai.system key, invisible to GenAI dashboards
• A providerless request minted a permanent Unknown provider series

How it solves it:

• Map search and RAG query to retrieval, agent sends to invoke_agent
• Give vector-store management a vendor name, since semconv has none
• Emit gen_ai.provider.name via the existing resolve_provider helper
• Omit the provider label instead of inventing a value

Relevant issues

Linear ticket

Resolves LIT-4954

Resolves LIT-4959

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

The branch was rebased onto current staging after these captures, so the SHAs quoted below are the pre-rebase ones the measurements were actually taken at. They map to the current commits in order: 8818ac2056 -> b8266b2810, bb885fa0f7 -> 0b26c2c595, 61c6742e3c -> 72a176debe, on base 551e5d097c -> 4eecf7a050. Nothing was re-measured, so the old SHAs are quoted as-is rather than relabelled with commits the numbers did not come from

One live proxy, OTel v2 with the console metric exporter, a real OpenAI chat completion and a real OpenAI vector-store search (a vector store built from an uploaded file; both calls hit api.openai.com and cost real money). No database, no mocks

# scratch config (port derived from the ticket number)
cat otel_4954_config.yaml
model_list:
  - model_name: gpt-5.6
    litellm_params:
      model: openai/gpt-5.6
      api_key: os.environ/OPENAI_API_KEY

litellm_settings:
  callbacks: ["otel"]

general_settings:
  master_key: sk-1234

# the exporter must not inherit a real collector endpoint from .env, hence the empty export
lsof -iTCP:4954 -sTCP:LISTEN || echo "4954 free"
set -a; source .env; set +a
export OTEL_ENDPOINT="" OTEL_EXPORTER=console LITELLM_OTEL_V2=1 \
       LITELLM_OTEL_INTEGRATION_ENABLE_METRICS=1 PYTHONUNBUFFERED=1
python litellm/proxy/proxy_cli.py --config otel_4954_config.yaml --port 4954 > run.log 2>&1 &

# 1. real chat completion
curl -sS -w "\nHTTP %{http_code}\n" http://127.0.0.1:4954/v1/chat/completions \
  -H 'Content-Type: application/json' -H 'Authorization: Bearer sk-1234' \
  -d '{"model":"gpt-5.6","messages":[{"role":"user","content":"Reply with exactly: otel attribute proof"}]}'

# 2. real vector-store search
curl -sS -w "\nHTTP %{http_code}\n" http://127.0.0.1:4954/v1/vector_stores/$VS/search \
  -H 'Content-Type: application/json' -H 'Authorization: Bearer sk-1234' \
  -d '{"query":"which attribute carries the provider value?"}'

# 3. read the exported metric datapoints. gen_ai.framework is the last attribute of a
#    metric datapoint's attribute set, so -B4 of it is the semconv block
grep -B4 '"gen_ai.framework": "litellm"' run.log | grep -v '^--$' | sed 's/^ *//' | sort | uniq -c

Both calls returned 200 at both SHAs; the chat completion answered otel attribute proof for 14 prompt / 6 completion tokens and the search returned its indexed chunk at score 0.93

Before, at 551e5d097c:

 103 "attributes": {
 103 "gen_ai.framework": "litellm",
 103 "gen_ai.operation.name": "chat",
  75 "gen_ai.request.model": "gpt-5.6",
  28 "gen_ai.request.model": null,
  75 "gen_ai.system": "openai",
  28 "gen_ai.system": null,

Every datapoint says chat, the vector-store ones included, and gen_ai.provider.name appears nowhere. The vector-store datapoint in full:

"gen_ai.operation.name": "chat",
"gen_ai.system": null,
"gen_ai.request.model": null,
"gen_ai.framework": "litellm",

After, at 8818ac2056:

  21 "attributes": {
  21 "gen_ai.framework": "litellm",
  15 "gen_ai.operation.name": "chat",
   6 "gen_ai.operation.name": "retrieval",
  15 "gen_ai.provider.name": "openai",
  15 "gen_ai.request.model": "gpt-5.6",
   6 "gen_ai.request.model": null,
  15 "gen_ai.system": "openai",

The chat datapoint carries both spellings and the search is now its own operation:

"gen_ai.operation.name": "chat",
"gen_ai.provider.name": "openai",
"gen_ai.system": "openai",
"gen_ai.request.model": "gpt-5.6",
"gen_ai.framework": "litellm",

"gen_ai.operation.name": "retrieval",
"gen_ai.request.model": null,
"gen_ai.framework": "litellm",

The retrieval datapoint carries no provider label at all, where before it carried gen_ai.system: null. The counts differ between runs only because the console exporter re-exports the cumulative datapoints every 5 seconds and the before run stayed up longer; the distinct attribute sets are what matters

Vector-store management, from the Greptile follow-up

Same rig, driving the management surface instead: create, retrieve, list, file-list and delete, all real OpenAI calls through the proxy, all HTTP 200

VS=$(curl -sS http://127.0.0.1:4954/v1/vector_stores "${H[@]}" -d '{"name":"litswarm-4954-mgmt"}' | jq -r .id)
curl -sS -o /dev/null -w "HTTP %{http_code}\n" "http://127.0.0.1:4954/v1/vector_stores/$VS"        "${H[@]}"
curl -sS -o /dev/null -w "HTTP %{http_code}\n" "http://127.0.0.1:4954/v1/vector_stores?limit=1"    "${H[@]}"
curl -sS -o /dev/null -w "HTTP %{http_code}\n" "http://127.0.0.1:4954/v1/vector_stores/$VS/files"  "${H[@]}"
curl -sS -o /dev/null -w "HTTP %{http_code}\n" -X DELETE "http://127.0.0.1:4954/v1/vector_stores/$VS" "${H[@]}"

grep -B4 '"gen_ai.framework": "litellm"' run.log \
  | grep -E 'operation.name|provider.name|gen_ai.system' | sed 's/^ *//' | sort | uniq -c

Before, at 8818ac2056 (the first commit on this branch, which mapped only the search):

  20 "gen_ai.operation.name": "chat",
   6 "gen_ai.provider.name": "openai",
   6 "gen_ai.system": "openai",

After, at bb885fa0f7:

   6 "gen_ai.operation.name": "litellm.vector_store_file_management",
  12 "gen_ai.operation.name": "litellm.vector_store_management",
   6 "gen_ai.provider.name": "openai",
   6 "gen_ai.system": "openai",

Not one datapoint reads chat any more. This also settles that the concern was live rather than theoretical: these management calls are @client-decorated and routed, so they reach the same success hook a completion does

Streaming A2A, from the Bugbot follow-up

Cursor Bugbot flagged that a streamed A2A turn still records as chat. It does, but not for the reason given, and the difference matters: _build_streaming_logging_obj in litellm/a2a_protocol/main.py builds its Logging by hand and never calls update_environment_variables, which is the only place call_type is written into model_call_details (litellm/litellm_core_utils/litellm_logging.py:553; Logging.__init__ seeds that dict at line 398 without it). The recorder was therefore seeing call_type as None and taking the not call_type branch. Adding the map entry alone changes nothing, which the middle capture below shows

A different rig from the vector-store one above: a real A2A agent, built on the a2a-sdk's own server routes, streaming a task plus three status updates plus a completed status, registered in agent_list and driven over HTTP through the proxy. Three streamed turns and one non-streamed turn per run, the non-streamed one being the control that was already labelled correctly. No database, no mocks

python agent_server.py 8959                       # real A2A agent, /.well-known/agent-card.json + JSON-RPC
export OTEL_ENDPOINT="" OTEL_EXPORTER=console LITELLM_OTEL_V2=1 \
       LITELLM_OTEL_INTEGRATION_ENABLE_METRICS=1
python litellm/proxy/proxy_cli.py --config config.yaml --port 4959 > run.log 2>&1 &

for i in 1 2 3; do
  curl -sS -N -X POST http://127.0.0.1:4959/a2a/streaming-proof-agent \
    -H 'Content-Type: application/json' -H 'Authorization: Bearer sk-1234' \
    -d "{\"jsonrpc\":\"2.0\",\"id\":\"$i\",\"method\":\"message/stream\",\"params\":{\"message\":{\"role\":\"user\",\"parts\":[{\"kind\":\"text\",\"text\":\"streamed turn $i\"}],\"messageId\":\"m$i\"}}}"
done
curl -sS -X POST http://127.0.0.1:4959/a2a/streaming-proof-agent \
  -H 'Content-Type: application/json' -H 'Authorization: Bearer sk-1234' \
  -d '{"jsonrpc":"2.0","id":"4","method":"message/send","params":{"message":{"role":"user","parts":[{"kind":"text","text":"non-streamed turn"}],"messageId":"m4"}}}'

All four calls HTTP 200 at every SHA, and the agent streamed its chunks back through the proxy each time. Attribute sets counted by brace-matching each "attributes": {...} object out of the exporter output and json.loads-ing it, since a regex stops at the first } inside a JSON-string label value and silently merges distinct series

Before, at bb885fa0f7:

x18  operation='chat'          provider=None          model='a2a_agent/otel-streaming-proof-agent'
x6   operation='invoke_agent'  provider='a2a_agent'   model='a2a_agent/otel-streaming-proof-agent'

With only the map entry added, which is the fix the finding implies:

x12  operation='chat'          provider=None          model='a2a_agent/otel-streaming-proof-agent'
x4   operation='invoke_agent'  provider='a2a_agent'   model='a2a_agent/otel-streaming-proof-agent'

After, at 61c6742e3c:

x12  operation='invoke_agent'  provider=None          model='a2a_agent/otel-streaming-proof-agent'
x4   operation='invoke_agent'  provider='a2a_agent'   model='a2a_agent/otel-streaming-proof-agent'

Not one datapoint reads chat. The 3:1 split is the three streamed turns against the one non-streamed control

The streamed datapoints carry no gen_ai.provider.name while the non-streamed ones carry a2a_agent, which is this PR's own omit-rather-than-invent rule working as intended: the recorder reads the provider from litellm_params["custom_llm_provider"], and on the non-bridge streaming path those params legitimately carry none. Making the two paths agree means changing what litellm_params holds for A2A streaming, which the cost calculator also reads, so it belongs in its own change

Type

🐛 Bug Fix

Changes

resolve_operation in litellm/integrations/otel/model/semconv.py mapped completion, text completion, embedding, responses and call_mcp_tool, and defaulted everything else to chat. Vector-store searches (vector_store_search / avector_store_search) and A2A agent sends (send_message / asend_message) therefore recorded their duration and cost into the chat series, so chat latency read off a GenAI dashboard silently included retrieval and agent time. Both now map to the values the GenAI convention defines for them: retrieval, whose registry entry names the OpenAI vector-store search API specifically, and invoke_agent. An unmapped call type still falls back to chat so a series is never unlabelled, but it now logs the call type at debug, because a silent default is how this landed in the first place

The rest of the vector-store family goes with it, which is the Greptile follow-up. The store lifecycle (vector_store_create / _retrieve / _list / _update / _delete) and the six file operations (_file_create / _file_list / _file_retrieve / _file_content / _file_update / _file_delete), each in both the sync and a-prefixed spelling, were all still landing on chat. Note these live as router factory_function call types rather than CallTypes members, so the enum understates the family; the router's call_type= strings are the authoritative list. /rag/query (query / aquery) goes to retrieval too, since it reaches the same recorder and is the same operation as a vector-store search, and mapping one retrieval surface but not the other would leave the defect half-fixed. /rag/ingest is a write with no semconv equivalent and no retrieval or agent confusion, so naming it belongs with the RAG owners rather than here

The convention names no operation for vector-store management, so those take vendor values, litellm.vector_store_management and litellm.vector_store_file_management, one per REST resource, under the litellm. prefix so a value the convention adds later can never collide. The convention's own note on gen_ai.operation.name directs instrumentation to use a system-specific name when no predefined value applies, which is the same allowance resolve_provider already relies on for unmapped providers, so this is the sanctioned move rather than a workaround. Two alternatives were considered and rejected. Excluding management calls from the GenAI metrics altogether is arguably cleaner semantically, since creating a vector store is not a GenAI client operation, but it deletes series an operator may be watching today and re-adding deleted telemetry is much harder than renaming a label, so it stays available as a follow-up instead of being decided here. Mapping them onto the semconv memory-store family (create_memory_store, delete_memory_store, and friends) was rejected outright: litellm vector stores hold documents, not an agent's memory records, and borrowing those names would push document-admin calls into whatever charts agent-memory operations, which is the same defect this PR exists to fix

For scope, the wider unmapped set (video, containers, files, skills, interactions, image edit, moderation, OCR) is out of this PR. Those are not retrieval or agent operations, so they are not what the ticket describes, and several need a product decision about whether they belong in gen_ai.client.* at all. The debug log added here is the mechanism that surfaces them rather than leaving the next person to discover it from a chart

The provider label was gen_ai.system, which the convention has moved to its deprecated registry in favor of gen_ai.provider.name. The v2 span path already resolves the semconv value through resolve_provider (bedrock becomes aws.bedrock); metrics never called it. They do now, and gen_ai.provider.name joins the metric-attribute allowlist so an operator can include or exclude it like any other label

The sharpest call here: gen_ai.system is dual-emitted with the raw litellm provider string it has always carried, not the mapped value. A query for gen_ai.system="bedrock" is exactly what the dual emission exists to keep working, so writing aws.bedrock into the deprecated key would break those queries while looking like compatibility. The new key gets the resolved semconv value; the old key's series stays byte-identical to what it emits today

Keeping the old key at all follows the precedent litellm/integrations/prometheus.py sets for renamed metrics. The label has shipped long enough that someone's dashboard is filtering on it, and a rename with no overlap breaks that dashboard silently at upgrade time rather than loudly. It should come out a release from now

The "Unknown" default is gone. A request litellm cannot attribute to a provider now carries no provider label, rather than a placeholder that mints a real, permanently-cardinal series aggregating every unattributable request into one bucket no operator can act on. The live capture shows this was not hypothetical: the vector-store search reached the recorder with custom_llm_provider set to None, so it was already emitting gen_ai.system: null

I left the v1 litellm/integrations/opentelemetry.py metric path on gen_ai.system (the only edit there is the one-line allowlist addition). Its gen_ai.operation.name is itself gated behind the gen_ai_latest_experimental opt-in and hardcodes chat otherwise, so renaming just its provider key would produce a half-migrated attribute set on an integration slated for deletion, while leaving it untouched keeps v1's output byte-identical and compatible with the v2 dual emission

The streamed A2A path needed a second change, not just a map entry. Its logging object is built by hand in _build_streaming_logging_obj rather than through update_environment_variables, so call_type never reached model_call_details and every callback, not only the metric recorder, saw a streamed agent turn with no call type at all. It is stamped there now alongside the model and custom_llm_provider the same function already sets, and asend_message_streaming joins the map. There is no sync spelling to add: A2A streaming is async-only, and this is the only call_type literal anywhere under litellm/a2a_protocol

The rebase brought in the metric-attribute ceiling that landed on staging as #34828, and gen_ai.provider.name had to join METRIC_ATTRIBUTE_CEILING for the new label to survive it. That is the correct side of that cap rather than a widening: the value is a fixed enum from resolve_provider, bounded exactly like the gen_ai.system entry it sits next to, and it is caller-independent. Without the entry the ceiling silently stripped the attribute this PR exists to add, which is what the three provider tests caught during the rebase

The sibling failure-path PR (LIT-4955) has since merged as #35152, and its plumbing/metrics.py changes are folded in by the rebase

Tests extend three mapped files: tests/test_litellm/integrations/otel/test_otel_v2_sources_of_truth.py for the call-type mapping, test_otel_v2_metrics.py for the recorded attributes, and tests/test_litellm/a2a_protocol/test_main.py for the streaming logging object's call type, which drive the real success hook into an in-memory metric reader. There is a test per call-type group pinning its operation name, one asserting every vendor value stays under the litellm. prefix, and the unmapped-fallback test uses a synthetic call type that can never accidentally become mapped

The streaming commit adds a test that pins the root cause rather than the map: reverting the stamping alone fails test_streaming_logging_obj_carries_call_type_into_model_call_details with a KeyError, and reverting the map entry alone fails the three mapping and recorder cases. A new test_every_call_type_the_a2a_package_stamps_is_an_agent_operation scans litellm/a2a_protocol for call_type="..." literals and asserts each resolves to invoke_agent, so a future spelling added there fails there instead of quietly landing in the chat series. tests/test_litellm/a2a_protocol plus tests/test_litellm/integrations/otel are green at 394 passed

All three commits were mutation-checked by reverting the production change and keeping the tests. For the first: 11 failed, 48 passed, then 59 passed restored. For the follow-up: 29 failed, 59 passed, then 88 passed restored, so every one of the 29 new cases fails without the mapping. The two full mapped files plus the v1 test_opentelemetry.py are green at 577 passed

QA runbook

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Changes observability labels and time-series cardinality for OTel v2 metrics; behavior is well-tested but operators’ dashboards and alerts keyed on old operation or provider attributes may need review.

Overview
OTel v2 GenAI metrics now use semconv-aligned labels so dashboards and cost/latency series are not skewed by mis-tagged operations.

Operation names: Vector-store search and RAG query map to gen_ai.operation.name=retrieval; A2A sends map to invoke_agent. Vector-store and file admin call types get vendor values litellm.vector_store_management and litellm.vector_store_file_management instead of defaulting to chat. Unmapped call_type values still fall back to chat but emit a debug log so new surfaces are visible.

Provider labels: Metrics emit gen_ai.provider.name via resolve_provider (e.g. bedrock → aws.bedrock), with gen_ai.system dual-emitted using the raw litellm provider string for backward compatibility. Calls with no provider omit both labels (no more permanent "Unknown" series). gen_ai.provider.name is on the metric-attribute allowlist.

Scope: v1 opentelemetry.py only gains the allowlist entry; v2 GenAIMetricRecorder._common_attributes implements the behavior. Tests cover mappings, dual emission, absent provider, and filterability.

^{Reviewed by Cursor Bugbot for commit bb885fa. Bugbot is set up for automated code reviews on this repo. Configure here.}

[yassin-berriai]

@greptileai

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR corrects OpenTelemetry GenAI metric attribution.

• Maps vector-store search and RAG queries to retrieval operations.
• Maps all current vector-store lifecycle and file-management call types to namespaced management operations.
• Maps A2A message sends, including streaming sends, to agent invocation and propagates the streaming call type into callback metadata.
• Emits the semconv provider-name attribute while retaining the deprecated system attribute for compatibility and omitting provider labels when unavailable.
• Extends tests for operation resolution, provider attributes, filtering, and A2A streaming metadata.

Confidence Score: 5/5

The PR appears safe to merge.

The previously reported vector-store CRUD mislabeling is resolved because the current production lifecycle and file-management call-type spellings are covered by the new operation map, and no blocking failure remains.

Important Files Changed

Filename	Overview
litellm/integrations/otel/model/semconv.py	Adds complete operation mappings for the currently routed vector-store lifecycle and file-management call types, resolving the previously reported chat-series contamination.
litellm/integrations/otel/plumbing/metrics.py	Records resolved operation and provider attributes while preserving the deprecated raw provider label for compatibility.
litellm/a2a_protocol/main.py	Propagates the streaming A2A call type into callback metadata so agent metrics resolve correctly.
litellm/integrations/opentelemetry.py	Adds the new provider-name attribute to the supported metric attribute allowlist.
tests/test_litellm/integrations/otel/test_otel_v2_sources_of_truth.py	Covers vector-store, retrieval, agent, vendor-namespace, and fallback operation resolution.
tests/test_litellm/integrations/otel/test_otel_v2_metrics.py	Verifies emitted provider and operation attributes through the metric recorder.
tests/test_litellm/a2a_protocol/test_main.py	Verifies that streaming A2A logging metadata carries the required call type.

_{Reviews (5): Last reviewed commit: "fix(otel): give the streaming A2A path a..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[yassin-berriai]

Fixed in bb885fa0f7; the P1 was right, the map was incomplete.

Mapping only the search left the store lifecycle (vector_store_create / _retrieve / _list / _update / _delete) and the six file operations (_file_create / _file_list / _file_retrieve / _file_content / _file_update / _file_delete), both spellings each, still falling through to chat. A live run at the previous commit confirmed it was not theoretical: all 20 metric datapoints from a create, retrieve, list, file-list and delete came out labelled chat. At bb885fa0f7 the same calls produce 12 litellm.vector_store_management and 6 litellm.vector_store_file_management datapoints and zero chat. Full before/after output is in the PR body.

Worth flagging for anyone auditing the family: these live as router factory_function call types, not CallTypes members, so the enum understates the set and the router's call_type= strings are the authoritative list.

The convention names no operation for vector-store management, so those two are vendor values under the litellm. prefix, one per REST resource. Its note on gen_ai.operation.name directs instrumentation to use a system-specific name when no predefined value applies, the same allowance resolve_provider relies on for unmapped providers. Excluding management calls from the GenAI metrics entirely was the alternative and is arguably cleaner semantically, but it deletes series an operator may be watching and is much harder to reverse than a rename, so it stays a follow-up rather than being decided here. Mapping onto the semconv memory-store family was rejected: litellm vector stores hold documents, not agent memory records, and those names would push document admin into whatever charts agent memory.

/rag/query (query / aquery) also maps to retrieval, since it reaches the same recorder and is the same operation as a vector-store search. /rag/ingest is a write with no semconv equivalent and no retrieval or agent confusion, so naming it belongs with the RAG owners.

Tests: one per call-type group pinning its operation name, one asserting every vendor value stays litellm.-prefixed, and the unmapped-fallback test still uses a synthetic call type that can never accidentally become mapped. Mutation-checked at 29 failed / 59 passed with the mapping reverted, 88 passed restored; mapped suites green at 577 passed.

@greptileai

[greptile-apps]

The fix is complete and correct. The mapping is exhaustive — all 10 lifecycle verbs × 2 prefixes for management, 6 file verbs × 2 prefixes for file management, plus search/RAG/agent — and the live capture confirms zero chat leakage from any of them.

A few observations on the follow-up commit:

1. verbose_logger.debug in resolve_operation — good addition. The silent fallback was exactly why retrieval calls spent time in the chat series unnoticed. The message includes the call type string, so it's actionable.

2. "vector_store_file_content" — worth a cross-check: is this the actual call_type string the router sets, or is it something like "vector_store_file_retrieve_content"? The router's factory_function path is the authoritative list per your own note, and the file-content endpoint is the one easiest to misname. If the router spells it differently this entry would silently fall through to chat with the new debug log as the only signal.

3. query / aquery collision risk — these are short, generic strings. If any other litellm call type is ever named query (e.g. a future SQL/graph query surface), it will silently map to retrieval. A comment on those two map entries noting "RAG /rag/query endpoint" would make the intent auditable without a code change.

4. Test coverage is solid — mutation-check numbers (29 failed / 88 passed) confirm 1:1 test-to-mapping coverage, and the synthetic-call-type test for the fallback path is the right approach to prevent that test from accidentally becoming vacuous.

5. gen_ai.system moved into GenAI constants — previously it was only in opentelemetry.py as a raw string. Good to have it centralized even as a deprecated constant; the docstring deprecation notice is clear.

The P1 concern is resolved. The mapping is complete, tested, and live-verified.

[yassin-berriai]

The misc / Run tests red on this PR is base breakage, not this diff.

The sole failure is tests/test_litellm/interactions/test_openapi_compliance.py::TestRequestCompliance::test_content_schema_uses_discriminator, which asserts a discriminator on the interactions Content union. Nothing in this PR touches interactions.

Evidence it is base:

• it reproduces locally on a clean tree branched from 551e5d097c with no interactions changes (1 failed, 12 passed in that file, no network or credentials involved)
• the identical single failure is on four unrelated PRs of mine touching different subsystems, and on an open PR from a different author

Tracked as LIT-4975. Leaving it red rather than pushing a no-op to re-roll, since a re-run will fail the same way until the base is fixed.

[yassin-berriai]

@greptileai

[yassin-berriai]

Root cause found for the misc / Run tests red on this PR, and it is not this diff. Fix is up as #35161.

tests/test_litellm/interactions/test_openapi_compliance.py::TestRequestCompliance::test_content_schema_uses_discriminator fetches Google's Interactions OpenAPI document over the network at run time and required an OpenAPI discriminator on the Content union. Google removed that keyword and now pins type with a const on each variant, so the assertion fails against the live spec on every open PR and will keep failing until the test is changed. Tracked as LIT-4975.

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_otel_genai_metric_attributes (72a176d) with litellm_internal_staging (4eecf7a)}

[yucheng-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit bb885fa. Configure here.}

[cursor]

Streaming agent still labeled chat

Medium Severity

send_message / asend_message map to invoke_agent, but the streaming A2A path records call_type as asend_message_streaming and still falls through to chat. That path dispatches the same success handlers, so streaming agent latency and cost keep polluting chat series.

 

^{Reviewed by Cursor Bugbot for commit bb885fa. Configure here.}

[yucheng-berri]

is it legit?

[yassin-berriai]

Legit, and worse than it reads. Fixed in 61c6742e3c.

The symptom is exactly right: a streamed A2A turn records as chat. The mechanism is only half right, and the half that is missing is why the one-line map entry Bugbot implies is not enough. I added "asend_message_streaming" to the map first, reran the live rig, and every streaming datapoint still came out chat.

_build_streaming_logging_obj (litellm/a2a_protocol/main.py) builds its Logging by hand and never calls update_environment_variables, which is the only place call_type is written into model_call_details (litellm/litellm_core_utils/litellm_logging.py:553); Logging.__init__ seeds that dict at line 398 without it. So the recorder was reading kwargs.get("call_type") is None and falling back to chat on the not call_type branch. The call type never reached the map at all, which also means it never reached any other callback either. The fix stamps it, and maps the streaming spelling.

Live proof

One proxy on the real streaming path, no mocks: a real A2A agent (built on the a2a-sdk's own server routes, streaming a task plus three status updates plus a completed status), registered in agent_list, driven over HTTP through POST /a2a/{agent} with method: message/stream. OTel v2 with the console metric exporter, no database. Each run sends 3 streamed turns and 1 non-streamed turn, so the non-streamed one is the control that was already correct.

python agent_server.py 8959                       # real A2A agent, /.well-known/agent-card.json + JSON-RPC
export OTEL_ENDPOINT="" OTEL_EXPORTER=console LITELLM_OTEL_V2=1 \
       LITELLM_OTEL_INTEGRATION_ENABLE_METRICS=1
python litellm/proxy/proxy_cli.py --config config.yaml --port 4959 > run.log 2>&1 &

for i in 1 2 3; do
  curl -sS -N -X POST http://127.0.0.1:4959/a2a/streaming-proof-agent \
    -H 'Content-Type: application/json' -H 'Authorization: Bearer sk-1234' \
    -d "{\"jsonrpc\":\"2.0\",\"id\":\"$i\",\"method\":\"message/stream\",\"params\":{\"message\":{\"role\":\"user\",\"parts\":[{\"kind\":\"text\",\"text\":\"streamed turn $i\"}],\"messageId\":\"m$i\"}}}"
done
curl -sS -X POST http://127.0.0.1:4959/a2a/streaming-proof-agent \
  -H 'Content-Type: application/json' -H 'Authorization: Bearer sk-1234' \
  -d '{"jsonrpc":"2.0","id":"4","method":"message/send","params":{"message":{"role":"user","parts":[{"kind":"text","text":"non-streamed turn"}],"messageId":"m4"}}}'

All four calls HTTP 200 at every SHA, and the agent streamed its chunks back through the proxy each time. Datapoint attribute sets are counted by brace-matching each "attributes": {...} object out of the exporter output and json.loads-ing it, rather than by regex, since a regex stops at the first } inside a JSON-string label value and silently merges distinct series.

Before, at bb885fa0f7:

x18  operation='chat'          provider=None          model='a2a_agent/otel-streaming-proof-agent'
x6   operation='invoke_agent'  provider='a2a_agent'   model='a2a_agent/otel-streaming-proof-agent'

With only the map entry added, which is the fix the finding implies:

x12  operation='chat'          provider=None          model='a2a_agent/otel-streaming-proof-agent'
x4   operation='invoke_agent'  provider='a2a_agent'   model='a2a_agent/otel-streaming-proof-agent'

Unchanged, because the call type was never arriving.

After, at 61c6742e3c:

x12  operation='invoke_agent'  provider=None          model='a2a_agent/otel-streaming-proof-agent'
x4   operation='invoke_agent'  provider='a2a_agent'   model='a2a_agent/otel-streaming-proof-agent'

Not one datapoint reads chat. The 3:1 split is the 3 streamed turns against the 1 non-streamed control.

Tests

Three, each mutation-checked by reverting the production line and rerunning:

• test_main.py::test_streaming_logging_obj_carries_call_type_into_model_call_details pins the root cause, that the hand-built logging object carries its call type into model_call_details. This is the one that fails on the pre-fix tree with a KeyError, i.e. it catches the real defect rather than the map.
• test_otel_v2_sources_of_truth.py::test_agent_message_is_an_invoke_agent_operation gains the streaming spelling, and a new test_every_call_type_the_a2a_package_stamps_is_an_agent_operation scans litellm/a2a_protocol for call_type="..." literals and asserts each one resolves to invoke_agent, so a future spelling added there fails here instead of quietly landing in the chat series.
• test_otel_v2_metrics.py::test_agent_message_is_not_labelled_as_chat is parametrized over both spellings, driving the real success hook into an in-memory reader.

Reverting the map entry alone fails 3, reverting the stamping alone fails 1; both restored, tests/test_litellm/a2a_protocol plus tests/test_litellm/integrations/otel are green at 394 passed.

One thing deliberately left alone

The streamed datapoints carry no gen_ai.provider.name, while the non-streamed ones carry a2a_agent. That is this PR's own rule working as intended rather than a second bug in this fix: the recorder reads the provider from litellm_params["custom_llm_provider"], and on the non-bridge streaming path those params legitimately carry no provider, so the label is omitted instead of invented. Making the two paths agree means changing what litellm_params holds for A2A streaming, which the cost calculator also reads, so it belongs in its own change rather than riding along here.

[yassin-berriai]

@greptileai please review the current head 61c6742e3c

New commit addresses the Cursor Bugbot finding on the A2A streaming path. The finding was real but its stated mechanism was incomplete: adding "asend_message_streaming" to _OPERATION_BY_CALL_TYPE on its own changes nothing, because _build_streaming_logging_obj builds its Logging by hand and never calls update_environment_variables, the only place call_type is written into model_call_details. The recorder was seeing call_type=None. The commit stamps the call type there and maps the streaming spelling; a live A2A streaming rig confirms all 12 streamed datapoints move from chat to invoke_agent, with the middle capture showing the map-only fix was inert. Full before/mid/after in the PR body and on the review thread.

[yassin-berriai]

@greptileai please review the current head b262547459

Rebased onto current staging, which had moved 48 commits and turned the PR CONFLICTING; the merge ref could not be built, which is why only 3 checks had run. Three conflicts, all in files the newly-merged #35152 (failure-path metrics) and #34828 (metric-attribute ceiling) also touch.

One of them is a real integration change worth a look rather than a mechanical resolution: #34828 added METRIC_ATTRIBUTE_CEILING, a cap applied before the operator filter, and gen_ai.provider.name was not in it, so the ceiling silently stripped the label this PR exists to add. Three provider tests caught it during the rebase. The attribute is now in the ceiling, which is the correct side of that cap: its value comes from resolve_provider, a fixed enum bounded exactly like the gen_ai.system entry it sits beside, and it is caller-independent, so it cannot grow cardinality with traffic.

Commit SHAs all changed. The proof captures in the body were taken pre-rebase and are quoted at their original SHAs with a mapping note, rather than relabelled with commits the numbers did not come from. tests/test_litellm/integrations/otel, tests/test_litellm/a2a_protocol and tests/test_litellm/integrations/test_opentelemetry.py are green at 675 passed on the rebased tree; ruff format --check and ruff check clean on all four changed production files.

[yassin-berriai]

@greptileai please review the current head 72a176debe

Correcting my previous comment: it named b262547459, which no longer exists. That commit was re-stamped to 72a176debe with an identical tree (verified git rev-parse HEAD^{tree} unchanged) purely to force a fresh ref-update event; GitHub had accepted the force-push at the git layer but never associated it with the PR, so head.sha stayed pinned to the pre-rebase commit, no workflows triggered, and only 3 checks showed. The re-stamp cleared it and the full 38-job matrix is now running.

No content changed between those two SHAs. The review targets are still the two commits described above: the A2A streaming call-type fix, and the gen_ai.provider.name addition to METRIC_ATTRIBUTE_CEILING that the rebase made necessary.