fix(llmobs/ai): surface prompt cache tokens for Vercel AI SDK integration across all supported providers (#8530)

* fix(llmobs/ai): surface prompt cache tokens for AI SDK Bedrock integration

Reads `ai.usage.cachedInputTokens` (AI SDK v6+ standardized attribute) and
`ai.response.providerMetadata.bedrock.usage.cache{Read,Write}InputTokens` and
tags them as `cache_read_input_tokens` / `cache_write_input_tokens` on the
LLMObs span. Without this, prompt-cached Bedrock requests via the Vercel AI
SDK appeared as 100% non-cached input in LLMObs, breaking cost and
cache-hit-rate dashboards.

Tests use the Vertex AI fetch-stub pattern (passing a mock fetch directly
to createAmazonBedrock) because @ai-sdk/amazon-bedrock uses globalThis.fetch
which nock cannot intercept. Two real-shape Bedrock Converse API fixtures
(cache read + cache write) exercise the extraction across ai 5.x / 6.x and
the compatible @ai-sdk/amazon-bedrock 3.x / 4.x versions.

Bumps the `ai` cap in test versions/package.json from 6.0.39 to 6.0.185 so
tests run against the SDK version that emits cache attributes on the
doGenerate span (older v6 versions only emit them on the parent generateText
span). Supported versions table regenerated.

Closes MLOB-7519
Refs MLOS-633

* fix(llmobs/ai): skip zero cache values and polyfill crypto for Node 18

The AI SDK sets `ai.usage.cachedInputTokens=0` on every language model
span regardless of provider, which caused our plugin to tag
`cache_read_input_tokens: 0` on spans that don't actually use prompt
caching (e.g., OpenAI calls). This broke existing OpenAI tests that
assert exact metric key counts. Skip emitting zero values so non-caching
spans stay clean — the cost estimator math is unchanged because
subtracting 0 is a no-op.

Additionally, `@ai-sdk/amazon-bedrock` signs requests with `aws4fetch`
which uses `globalThis.crypto` for SHA256/HMAC. Node 19+ exposes that as
a global, but Node 18 (still supported and used in CI) does not. Polyfill
from `node:crypto.webcrypto` in the Bedrock test setup.

Bedrock cache test assertions updated to expect `undefined` (not `0`)
for the absent cache field in each scenario.

* test(llmobs/ai): add Anthropic prompt cache tests and parameterize for additional providers

Extracts the Bedrock prompt-cache integration tests into a generic
`describeProviderCacheTests()` helper that takes a provider config
object (display name, package mapper, model builder, optional env
vars). Adds Anthropic as the second consumer, exercising the
`providerMetadata.anthropic.cache{Read,Creation}InputTokens` branch
of `getProviderCacheTokens()` that was previously uncovered by tests
(Codecov diff coverage gap).

Anthropic SDK has the same multi-version normalization quirk as
Bedrock (verified in source): `@ai-sdk/anthropic@3.x` normalizes
`inputTokens.total = inputTokens + cacheCreation + cacheRead` via
`convertToLanguageModelV3Usage`, while `@2.x` and `@1.x` pass the
raw fresh count through. The generic test helper exercises all
three majors via `withVersions`.

Additional providers (OpenAI, Google, etc.) can be added by passing
one more config object to `describeProviderCacheTests()` — no
duplicated test orchestration code.

* test(llmobs/ai): add OpenAI prompt cache tests across Chat Completions and Responses APIs

Proves the cache_read capture in this plugin is provider-agnostic and
not specific to Bedrock or Anthropic. OpenAI exercises the
`ai.usage.cachedInputTokens` standardized-attribute path — by virtue of
covering OpenAI, this also transitively validates Google Gemini, xAI,
and any other provider whose `@ai-sdk/<name>` package populates the
standardized attribute.

Two consumers added because `@ai-sdk/openai` v1/v2/v3 route the default
`openai(modelId)` factory to the Responses API, while `.chat(modelId)`
routes to Chat Completions. The two endpoints have different response
shapes and the SDK has separate parsing for each:

- Chat Completions:  usage.prompt_tokens_details.cached_tokens
- Responses API:     usage.input_tokens_details.cached_tokens

Both endpoints surface cache_read via `ai.usage.cachedInputTokens` on
the doGenerate span, but covering both paths future-proofs the plugin
against any SDK regression that might affect only one endpoint —
relevant because OpenAI is actively migrating customers from Chat
Completions to Responses.

Extends `describeProviderCacheTests()` helper with two new config
options to support providers whose semantics differ from
Bedrock/Anthropic:
- `scenarios`: which scenarios to test (OpenAI lacks per-request
  cache_write, so it passes ['cache-read'] only)
- `getExpectedMetrics`: version-aware expectations (OpenAI's
  `prompt_tokens` / `input_tokens` is always the sum at the API
  level, so `ai.usage.inputTokens` is 4448 across all ai versions —
  unlike Bedrock/Anthropic where the v5-paired SDK passes raw
  fresh through and the v6-paired SDK normalizes)

18 tests now exercise the full provider matrix:
- Bedrock (cache_read + cache_write) x 3 ai versions = 6
- Anthropic (cache_read + cache_write) x 3 ai versions = 6
- OpenAI Chat Completions (cache_read) x 3 ai versions = 3
- OpenAI Responses API (cache_read) x 3 ai versions = 3

* test(llmobs/ai): add Google Gemini prompt cache tests

Google Gemini exercises a third upstream API shape distinct from both
the Bedrock/Anthropic family (provider-specific cache fields in
providerMetadata) and the OpenAI-compatible family (Chat Completions /
Responses with shared field names).

Google's response uses `usageMetadata.cachedContentTokenCount` at the
top level of the response (not nested under a provider namespace), and
the `@ai-sdk/google` provider package maps it to
`ai.usage.cachedInputTokens` via the standardized attribute path.
Like OpenAI, Google's `promptTokenCount` already includes cached
tokens at the API level, so `ai.usage.inputTokens` is the sum across
all ai versions.

Per-request cache write metric is not exposed by Google (context
caching requires a separate API call to create the cache), so only
cache-read is tested.

By covering three distinct API shapes (Bedrock/Anthropic native, OpenAI
shared, Google native), we establish that the plugin's cache token
capture is genuinely provider-agnostic via the AI SDK v5+ standardized
attribute, not just inferentially supported. Transitively this also
covers xAI (which uses @ai-sdk/openai-compatible under the hood — same
code paths as OpenAI).

21 tests now cover the full provider matrix.

* fix(llmobs/ai): normalize inputTokens to the sum convention used by bedrockruntime

Addresses Codex review feedback: some provider/SDK combinations leave
`ai.usage.inputTokens` as the raw fresh count while still surfacing
cache_write tokens in `providerMetadata`. A downstream consumer doing
`nonCached = input - cacheRead - cacheWrite` would compute a negative
number, and the upcoming backend partial-tokens UI warning would fire
even though the cost math is otherwise correct via the token_normalizer
clamp.

Specifically affected stacks:
- ai@5 + @ai-sdk/amazon-bedrock@3.x (bedrock@3 doesn't normalize)
- ai@4-5 + @ai-sdk/anthropic@1-2.x (older anthropic SDK doesn't normalize)

Matches the convention in bedrockruntime.js:157 — input_tokens is always
the sum of fresh + cache_read + cache_write.

Detection uses a math check: if `inputTokens < cacheRead + cacheWrite`,
the value cannot already be a sum (would be impossible for fresh to be
less than cached). This is the only signal that reliably differentiates
"raw fresh" from "already normalized" without false positives on stacks
where the upstream SDK has already done the work (ai@6 + bedrock@4 /
anthropic@3, OpenAI, Google).

Tests updated for ai@5 cache-write scenarios where `input_tokens` is now
the normalized sum (4448) instead of raw fresh (23).

* chore: regenerate supported versions after rebase

* chore: sync versions/package.json with master and add provider sdk caps

Author: jessicagamio

packages/datadog-plugin-ai/test/resources/anthropic-cache-read.json

@@ -0,0 +1,15 @@
+{
+  "id": "msg_01abc123def456",
+  "type": "message",
+  "role": "assistant",
+  "model": "claude-3-5-haiku-20241022",
+  "content": [{ "type": "text", "text": "Datadog LLM Observability provides end-to-end visibility." }],
+  "stop_reason": "end_turn",
+  "stop_sequence": null,
+  "usage": {
+    "input_tokens": 23,
+    "output_tokens": 44,
+    "cache_creation_input_tokens": 0,
+    "cache_read_input_tokens": 4425
+  }
+}

packages/datadog-plugin-ai/test/resources/anthropic-cache-write.json

@@ -0,0 +1,15 @@
+{
+  "id": "msg_01xyz789ghi012",
+  "type": "message",
+  "role": "assistant",
+  "model": "claude-3-5-haiku-20241022",
+  "content": [{ "type": "text", "text": "Datadog LLM Observability provides end-to-end visibility." }],
+  "stop_reason": "end_turn",
+  "stop_sequence": null,
+  "usage": {
+    "input_tokens": 23,
+    "output_tokens": 53,
+    "cache_creation_input_tokens": 4425,
+    "cache_read_input_tokens": 0
+  }
+}

packages/datadog-plugin-ai/test/resources/bedrock-cache-read.json

@@ -0,0 +1,17 @@
+{
+  "stopReason": "end_turn",
+  "output": {
+    "message": {
+      "role": "assistant",
+      "content": [{ "type": "text", "text": "Datadog LLM Observability provides end-to-end visibility." }]
+    }
+  },
+  "usage": {
+    "inputTokens": 23,
+    "outputTokens": 44,
+    "totalTokens": 4492,
+    "cacheReadInputTokens": 4425,
+    "cacheWriteInputTokens": 0
+  },
+  "metrics": { "latencyMs": 1000 }
+}

packages/datadog-plugin-ai/test/resources/bedrock-cache-write.json

@@ -0,0 +1,17 @@
+{
+  "stopReason": "end_turn",
+  "output": {
+    "message": {
+      "role": "assistant",
+      "content": [{ "type": "text", "text": "Datadog LLM Observability provides end-to-end visibility." }]
+    }
+  },
+  "usage": {
+    "inputTokens": 23,
+    "outputTokens": 53,
+    "totalTokens": 76,
+    "cacheReadInputTokens": 0,
+    "cacheWriteInputTokens": 4425
+  },
+  "metrics": { "latencyMs": 1000 }
+}

packages/datadog-plugin-ai/test/resources/google-cache-read.json

@@ -0,0 +1,19 @@
+{
+  "candidates": [
+    {
+      "content": {
+        "role": "model",
+        "parts": [
+          { "text": "Datadog LLM Observability provides end-to-end visibility." }
+        ]
+      },
+      "finishReason": "STOP"
+    }
+  ],
+  "usageMetadata": {
+    "promptTokenCount": 4448,
+    "candidatesTokenCount": 44,
+    "totalTokenCount": 4492,
+    "cachedContentTokenCount": 4425
+  }
+}

packages/datadog-plugin-ai/test/resources/openai-cache-read.json

@@ -0,0 +1,33 @@
+{
+  "id": "chatcmpl-abc123def456",
+  "object": "chat.completion",
+  "created": 1779284000,
+  "model": "gpt-4o-mini-2024-07-18",
+  "choices": [
+    {
+      "index": 0,
+      "message": {
+        "role": "assistant",
+        "content": "Datadog LLM Observability provides end-to-end visibility."
+      },
+      "logprobs": null,
+      "finish_reason": "stop"
+    }
+  ],
+  "usage": {
+    "prompt_tokens": 4448,
+    "completion_tokens": 44,
+    "total_tokens": 4492,
+    "prompt_tokens_details": {
+      "cached_tokens": 4425,
+      "audio_tokens": 0
+    },
+    "completion_tokens_details": {
+      "reasoning_tokens": 0,
+      "audio_tokens": 0,
+      "accepted_prediction_tokens": 0,
+      "rejected_prediction_tokens": 0
+    }
+  },
+  "system_fingerprint": "fp_test123"
+}

packages/datadog-plugin-ai/test/resources/openai-responses-cache-read.json

@@ -0,0 +1,33 @@
+{
+  "id": "resp_abc123def456",
+  "object": "response",
+  "created_at": 1779284000,
+  "status": "completed",
+  "model": "gpt-4o-mini-2024-07-18",
+  "incomplete_details": null,
+  "output": [
+    {
+      "type": "message",
+      "id": "msg_abc123",
+      "status": "completed",
+      "role": "assistant",
+      "content": [
+        {
+          "type": "output_text",
+          "text": "Datadog LLM Observability provides end-to-end visibility.",
+          "annotations": []
+        }
+      ]
+    }
+  ],
+  "usage": {
+    "input_tokens": 4448,
+    "input_tokens_details": {
+      "cached_tokens": 4425
+    },
+    "output_tokens": 44,
+    "output_tokens_details": {
+      "reasoning_tokens": 0
+    }
+  }
+}

packages/dd-trace/src/llmobs/plugins/ai/util.js

@@ -66,11 +66,22 @@ function getOperation (span) {
 }
 
 /**
- * Get the LLM token usage from the span tags
- * Supports both AI SDK v4 (promptTokens/completionTokens) and v5 (inputTokens/outputTokens)
- * @template T extends {inputTokens: number, outputTokens: number, totalTokens: number}
- * @param {T} tags
- * @returns {Pick<T, 'inputTokens' | 'outputTokens' | 'totalTokens'>}
+ * Get the LLM token usage from the span tags.
+ *
+ * Supports both AI SDK v4 (promptTokens/completionTokens) and v5+
+ * (inputTokens/outputTokens), and surfaces prompt-cache metrics for providers
+ * that report them. The AI SDK convention is that `inputTokens` already
+ * includes cached tokens, so cache reads are reported as a subset of input
+ * tokens rather than added on top.
+ *
+ * @param {SpanTags} tags
+ * @returns {{
+ *   inputTokens?: number,
+ *   outputTokens?: number,
+ *   totalTokens?: number,
+ *   cacheReadTokens?: number,
+ *   cacheWriteTokens?: number
+ * }}
  */
 function getUsage (tags) {
   const usage = {}
@@ -87,9 +98,84 @@ function getUsage (tags) {
   const totalTokens = tags['ai.usage.totalTokens'] ?? (inputTokens + outputTokens)
   if (!Number.isNaN(totalTokens)) usage.totalTokens = totalTokens
 
+  // Prompt-cache metrics. AI SDK v6 standardizes cache READ tokens via
+  // `ai.usage.cachedInputTokens`; cache WRITE tokens (and earlier AI SDK
+  // versions / providers that don't fill `cachedInputTokens`) are only
+  // available through provider-specific `ai.response.providerMetadata`.
+  // Skip zero values: the AI SDK sets `cachedInputTokens=0` on every span
+  // regardless of provider, so emitting it would add noise to spans that
+  // don't actually use prompt caching (e.g. OpenAI).
+  const providerCache = getProviderCacheTokens(tags['ai.response.providerMetadata'])
+
+  const cacheReadTokens = tags['ai.usage.cachedInputTokens'] ?? providerCache.cacheReadTokens
+  if (cacheReadTokens) usage.cacheReadTokens = cacheReadTokens
+
+  if (providerCache.cacheWriteTokens) usage.cacheWriteTokens = providerCache.cacheWriteTokens
+
+  // Normalize `inputTokens` to the sum convention used by `bedrockruntime.js`.
+  // Some SDK combinations (e.g. `ai@5` + `@ai-sdk/amazon-bedrock@3`) pass the
+  // raw fresh count through, which makes `nonCached = input - cacheRead -
+  // cacheWrite` go negative downstream.
+  //
+  // Detection: if `inputTokens < cacheSum`, the value cannot already be a sum
+  // that includes them (non-negative arithmetic). This is provider/version
+  // agnostic and won't double-count on stacks where the SDK already
+  // normalized (`ai@6` + `bedrock@4` / `anthropic@3`, OpenAI, Google).
+  if (usage.inputTokens != null) {
+    const cacheSum = (usage.cacheReadTokens || 0) + (usage.cacheWriteTokens || 0)
+    if (usage.inputTokens < cacheSum) {
+      usage.inputTokens += cacheSum
+      if (usage.totalTokens != null) {
+        usage.totalTokens = usage.inputTokens + (usage.outputTokens || 0)
+      }
+    }
+  }
+
   return usage
 }
 
+/**
+ * Extract prompt-cache token counts from the stringified
+ * `ai.response.providerMetadata` attribute.
+ *
+ * The AI SDK does not standardize cache WRITE tokens on the usage object, and
+ * earlier versions / providers may also omit `ai.usage.cachedInputTokens`, so
+ * we read the provider-specific shape directly. Only Bedrock and Anthropic
+ * are handled here as they are the providers that report cache writes today.
+ *
+ * @see https://ai-sdk.dev/providers/ai-sdk-providers/amazon-bedrock#cache-points
+ * @see https://ai-sdk.dev/providers/ai-sdk-providers/anthropic#cache-control
+ *
+ * @param {string | undefined} providerMetadataJson
+ * @returns {{ cacheReadTokens?: number, cacheWriteTokens?: number }}
+ */
+function getProviderCacheTokens (providerMetadataJson) {
+  if (!providerMetadataJson) return {}
+
+  const metadata = getJsonStringValue(providerMetadataJson, null)
+  if (!metadata || typeof metadata !== 'object') return {}
+
+  const result = {}
+
+  const bedrockUsage = metadata.bedrock?.usage
+  if (bedrockUsage) {
+    if (bedrockUsage.cacheReadInputTokens != null) result.cacheReadTokens = bedrockUsage.cacheReadInputTokens
+    if (bedrockUsage.cacheWriteInputTokens != null) result.cacheWriteTokens = bedrockUsage.cacheWriteInputTokens
+  }
+
+  const anthropic = metadata.anthropic
+  if (anthropic) {
+    if (result.cacheReadTokens == null && anthropic.cacheReadInputTokens != null) {
+      result.cacheReadTokens = anthropic.cacheReadInputTokens
+    }
+    if (result.cacheWriteTokens == null && anthropic.cacheCreationInputTokens != null) {
+      result.cacheWriteTokens = anthropic.cacheCreationInputTokens
+    }
+  }
+
+  return result
+}
+
 /**
  * Safely JSON parses a string value with a default fallback
  * @template T typeof defaultValue