Skip to content

Commit 3a07cb4

Browse files
alwayys-afkalwayys-afk
authored
fix(telemetry): emit OTel-standard gen_ai.usage.cache_read.input_tokens across providers (0xPlaygrounds#1666)
* fix: emit OTel-standard cache_read.input_tokens across providers Provider spans declared `gen_ai.usage.cached_tokens` (non-OTel) but the shared telemetry helpers (`SpanCombinator::record_token_usage`, `openai_chat_completions_compatible::record_usage`) write the OTel- standard `gen_ai.usage.cache_read.input_tokens`. Because `tracing` silently drops `.record()` for fields not declared on the span, cached- token values were being computed and thrown away on every provider whose span did not declare the OTel name. Declare `gen_ai.usage.cache_read.input_tokens` on every affected span across OpenAI, Azure, Cohere, Gemini (incl. interactions_api), Groq, DeepSeek, HuggingFace, Mistral, OpenRouter, Together, xAI, Copilot, and llamafile. Emit `cache_read.input_tokens` from the OpenAI chat- completions compatible helper and the OpenAI Responses API record sites. Do not emit `cache_creation.input_tokens` on OpenAI-family spans — those APIs have no cache-creation concept and a hardcoded 0 would be misleading rather than informative. Anthropic, which does report cache creation, is unchanged. Remove the non-OTel `gen_ai.usage.cached_tokens` attribute from every path this change touches: drop the span declaration and, on paths whose only recording sites are modified here, drop the record calls too. Only the OTel-standard attribute is emitted from these paths. Spans that record cache tokens entirely through their own inline `span.record("gen_ai.usage.cached_tokens", ...)` calls (e.g. non- streaming paths of Groq, DeepSeek, Copilot; standalone files like Galadriel, Hyperbolic, Mira, Moonshot, Ollama, Perplexity, ChatGPT; non-streaming Together and xAI) are out of scope for this change and continue to emit only `cached_tokens`. * fix: emit cache_read.input_tokens on 5 providers missed by prior commit Galadriel, Hyperbolic, Mira, Moonshot, and Perplexity all route their streaming paths through `send_compatible_streaming_request`, whose shared `record_usage` helper was updated in the prior commit to write the OTel-standard `gen_ai.usage.cache_read.input_tokens`. These five provider spans still declared `gen_ai.usage.cached_tokens`, so the newly-recorded value was silently dropped by `tracing` — and since none of these files inline-records `cached_tokens` either, their streaming paths were emitting no cache-read metric at all after the prior commit landed. Rename the declaration to `gen_ai.usage.cache_read.input_tokens` on both the non-streaming and streaming span in each file, matching the pattern the prior commit already applied across the other providers. The non-streaming rename is a no-op (no recorder targets the field on that path) but keeps both spans in each file consistent. * fix: complete cache_read.input_tokens rename across remaining providers Prior two commits partially renamed gen_ai.usage.cached_tokens to the canonical OTel GenAI attribute gen_ai.usage.cache_read.input_tokens but left several providers (and some streaming-vs-non-streaming paths within a provider) emitting the old name. Finish the rename in chatgpt, copilot, deepseek (non-streaming), groq (non-streaming), ollama, together/completion, and xai/completion so every span consistently emits cache_read.input_tokens.
1 parent de43a16 commit 3a07cb4

32 files changed

Lines changed: 66 additions & 65 deletions

‎rig/rig-core/src/providers/azure.rs‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -709,7 +709,7 @@ where
709709
gen_ai.response.model = tracing::field::Empty,
710710
gen_ai.usage.output_tokens = tracing::field::Empty,
711711
gen_ai.usage.input_tokens = tracing::field::Empty,
712-
gen_ai.usage.cached_tokens = tracing::field::Empty,
712+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
713713
)
714714
} else {
715715
tracing::Span::current()
@@ -809,7 +809,7 @@ where
809809
gen_ai.response.model = tracing::field::Empty,
810810
gen_ai.usage.output_tokens = tracing::field::Empty,
811811
gen_ai.usage.input_tokens = tracing::field::Empty,
812-
gen_ai.usage.cached_tokens = tracing::field::Empty,
812+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
813813
)
814814
} else {
815815
tracing::Span::current()

‎rig/rig-core/src/providers/chatgpt/mod.rs‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -493,7 +493,7 @@ where
493493
gen_ai.response.model = tracing::field::Empty,
494494
gen_ai.usage.output_tokens = tracing::field::Empty,
495495
gen_ai.usage.input_tokens = tracing::field::Empty,
496-
gen_ai.usage.cached_tokens = tracing::field::Empty,
496+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
497497
gen_ai.input.messages = tracing::field::Empty,
498498
gen_ai.output.messages = tracing::field::Empty,
499499
)
@@ -510,7 +510,7 @@ where
510510
span.record("gen_ai.usage.output_tokens", response.usage.output_tokens);
511511
span.record("gen_ai.usage.input_tokens", response.usage.input_tokens);
512512
span.record(
513-
"gen_ai.usage.cached_tokens",
513+
"gen_ai.usage.cache_read.input_tokens",
514514
response.usage.cached_input_tokens,
515515
);
516516
Ok(response)
@@ -575,7 +575,7 @@ where
575575
gen_ai.response.model = tracing::field::Empty,
576576
gen_ai.usage.output_tokens = tracing::field::Empty,
577577
gen_ai.usage.input_tokens = tracing::field::Empty,
578-
gen_ai.usage.cached_tokens = tracing::field::Empty,
578+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
579579
)
580580
} else {
581581
tracing::Span::current()

‎rig/rig-core/src/providers/cohere/completion.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -643,7 +643,7 @@ where
643643
gen_ai.response.model = self.model,
644644
gen_ai.usage.output_tokens = tracing::field::Empty,
645645
gen_ai.usage.input_tokens = tracing::field::Empty,
646-
gen_ai.usage.cached_tokens = tracing::field::Empty,
646+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
647647
)
648648
} else {
649649
tracing::Span::current()

‎rig/rig-core/src/providers/cohere/streaming.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,7 @@ where
111111
gen_ai.response.model = self.model,
112112
gen_ai.usage.output_tokens = tracing::field::Empty,
113113
gen_ai.usage.input_tokens = tracing::field::Empty,
114-
gen_ai.usage.cached_tokens = tracing::field::Empty,
114+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
115115
)
116116
} else {
117117
tracing::Span::current()

‎rig/rig-core/src/providers/copilot/mod.rs‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -681,7 +681,7 @@ where
681681
gen_ai.response.model = tracing::field::Empty,
682682
gen_ai.usage.output_tokens = tracing::field::Empty,
683683
gen_ai.usage.input_tokens = tracing::field::Empty,
684-
gen_ai.usage.cached_tokens = tracing::field::Empty,
684+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
685685
)
686686
} else {
687687
tracing::Span::current()
@@ -705,7 +705,7 @@ where
705705
usage.total_tokens - usage.prompt_tokens,
706706
);
707707
span.record(
708-
"gen_ai.usage.cached_tokens",
708+
"gen_ai.usage.cache_read.input_tokens",
709709
usage
710710
.prompt_tokens_details
711711
.as_ref()
@@ -762,7 +762,7 @@ where
762762
gen_ai.response.model = tracing::field::Empty,
763763
gen_ai.usage.output_tokens = tracing::field::Empty,
764764
gen_ai.usage.input_tokens = tracing::field::Empty,
765-
gen_ai.usage.cached_tokens = tracing::field::Empty,
765+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
766766
)
767767
} else {
768768
tracing::Span::current()
@@ -782,7 +782,7 @@ where
782782
span.record("gen_ai.usage.input_tokens", usage.input_tokens);
783783
span.record("gen_ai.usage.output_tokens", usage.output_tokens);
784784
span.record(
785-
"gen_ai.usage.cached_tokens",
785+
"gen_ai.usage.cache_read.input_tokens",
786786
usage
787787
.input_tokens_details
788788
.as_ref()
@@ -843,7 +843,7 @@ where
843843
gen_ai.response.model = tracing::field::Empty,
844844
gen_ai.usage.output_tokens = tracing::field::Empty,
845845
gen_ai.usage.input_tokens = tracing::field::Empty,
846-
gen_ai.usage.cached_tokens = tracing::field::Empty,
846+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
847847
)
848848
} else {
849849
tracing::Span::current()
@@ -885,7 +885,7 @@ where
885885
gen_ai.response.model = tracing::field::Empty,
886886
gen_ai.usage.output_tokens = tracing::field::Empty,
887887
gen_ai.usage.input_tokens = tracing::field::Empty,
888-
gen_ai.usage.cached_tokens = tracing::field::Empty,
888+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
889889
)
890890
} else {
891891
tracing::Span::current()
@@ -1042,7 +1042,7 @@ where
10421042
span.record("gen_ai.usage.input_tokens", final_usage.input_tokens);
10431043
span.record("gen_ai.usage.output_tokens", final_usage.output_tokens);
10441044
span.record(
1045-
"gen_ai.usage.cached_tokens",
1045+
"gen_ai.usage.cache_read.input_tokens",
10461046
final_usage
10471047
.input_tokens_details
10481048
.as_ref()

‎rig/rig-core/src/providers/deepseek.rs‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -560,7 +560,7 @@ where
560560
gen_ai.response.model = tracing::field::Empty,
561561
gen_ai.usage.output_tokens = tracing::field::Empty,
562562
gen_ai.usage.input_tokens = tracing::field::Empty,
563-
gen_ai.usage.cached_tokens = tracing::field::Empty,
563+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
564564
)
565565
} else {
566566
tracing::Span::current()
@@ -600,7 +600,7 @@ where
600600
response.usage.completion_tokens,
601601
);
602602
span.record(
603-
"gen_ai.usage.cached_tokens",
603+
"gen_ai.usage.cache_read.input_tokens",
604604
response
605605
.usage
606606
.prompt_tokens_details
@@ -673,7 +673,7 @@ where
673673
gen_ai.response.model = tracing::field::Empty,
674674
gen_ai.usage.output_tokens = tracing::field::Empty,
675675
gen_ai.usage.input_tokens = tracing::field::Empty,
676-
gen_ai.usage.cached_tokens = tracing::field::Empty,
676+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
677677
)
678678
} else {
679679
tracing::Span::current()

‎rig/rig-core/src/providers/galadriel.rs‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -553,7 +553,7 @@ where
553553
gen_ai.response.model = tracing::field::Empty,
554554
gen_ai.usage.output_tokens = tracing::field::Empty,
555555
gen_ai.usage.input_tokens = tracing::field::Empty,
556-
gen_ai.usage.cached_tokens = tracing::field::Empty,
556+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
557557
)
558558
} else {
559559
tracing::Span::current()
@@ -653,7 +653,7 @@ where
653653
gen_ai.response.model = tracing::field::Empty,
654654
gen_ai.usage.output_tokens = tracing::field::Empty,
655655
gen_ai.usage.input_tokens = tracing::field::Empty,
656-
gen_ai.usage.cached_tokens = tracing::field::Empty,
656+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
657657
gen_ai.input.messages = serde_json::to_string(&request.messages)?,
658658
gen_ai.output.messages = tracing::field::Empty,
659659
)

‎rig/rig-core/src/providers/gemini/completion.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,7 @@ where
102102
gen_ai.response.model = tracing::field::Empty,
103103
gen_ai.usage.output_tokens = tracing::field::Empty,
104104
gen_ai.usage.input_tokens = tracing::field::Empty,
105-
gen_ai.usage.cached_tokens = tracing::field::Empty,
105+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
106106
)
107107
} else {
108108
tracing::Span::current()

‎rig/rig-core/src/providers/gemini/interactions_api/mod.rs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,7 @@ where
132132
gen_ai.response.model = tracing::field::Empty,
133133
gen_ai.usage.output_tokens = tracing::field::Empty,
134134
gen_ai.usage.input_tokens = tracing::field::Empty,
135+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
135136
)
136137
} else {
137138
tracing::Span::current()

‎rig/rig-core/src/providers/gemini/interactions_api/streaming.rs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@ where
6262
gen_ai.response.model = tracing::field::Empty,
6363
gen_ai.usage.output_tokens = tracing::field::Empty,
6464
gen_ai.usage.input_tokens = tracing::field::Empty,
65+
gen_ai.usage.cache_read.input_tokens = tracing::field::Empty,
6566
)
6667
} else {
6768
tracing::Span::current()

0 commit comments

Comments
 (0)