Skip to content

feat(agenticopenai): expose prompt cache write usage - #983

Open
sin-bufan wants to merge 2 commits into
cloudwego:mainfrom
sin-bufan:codex/agenticopenai-cache-write-tokens
Open

feat(agenticopenai): expose prompt cache write usage#983
sin-bufan wants to merge 2 commits into
cloudwego:mainfrom
sin-bufan:codex/agenticopenai-cache-write-tokens

Conversation

@sin-bufan

Copy link
Copy Markdown
Contributor

What type of PR is this?

feat

Check the PR title.

  • This PR title matches the required format.
  • The description is user-oriented and clear.
  • No separate user documentation update is required; this exposes response billing metadata to downstream integrations.

(Optional) Translate the PR title into Chinese.

feat(agenticopenai): 暴露提示词缓存写入用量

(Optional) More detailed description for this PR(en: English/zh: Chinese).

en:

OpenAI Responses for GPT-5.6 and later reports usage.input_tokens_details.cache_write_tokens separately from cached_tokens because cache writes and reads have different billing rates. agenticopenai currently preserves only cache reads, so downstream observability integrations cannot calculate prompt-cache cost accurately.

This change follows the message.Extra pattern established by #978:

  • Exposes positive cache_write_tokens as _eino_openai_cache_write_tokens.
  • Adds GetCacheWriteTokens for downstream consumers.
  • Covers both Generate and response lifecycle streaming events.
  • Keeps cache writes separate from PromptTokenDetails.CachedTokens.
  • Reads the field from the OpenAI SDK preserved JSON ExtraFields until the SDK adds a typed property.
  • Omits absent, zero, negative, malformed, fractional, and overflowing values.
  • Verifies the value survives schema.ConcatAgenticMessages.

Tests:

  • go test -gcflags=all=-N\ -l ./...

zh(optional):

OpenAI Responses 的 cache_write_tokens 与 cached_tokens 使用不同计费价格。本改动参考 #978,通过 AgenticMessage.Extra 同时在 Generate 和流式响应中透传缓存写入 token,供 Langfuse 等下游准确计算成本。

(Optional) Which issue(s) this PR fixes:

Related to #978.

@shentongmartin shentongmartin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the implementation and tests. The provider-specific metadata design is consistent with the existing pattern, Generate and Stream paths are covered, and no merge blockers were found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants