Opt-in genai-prices pricing for non-Anthropic models#9
Open
dsfaccini wants to merge 1 commit into
Open
Conversation
The built-in cost table only covers Anthropic models (opus/sonnet/haiku), so sessions run against an Anthropic-compatible endpoint via ANTHROPIC_BASE_URL (z.ai/GLM, Moonshot/Kimi, MiniMax, ...) get no operation.cost. Setting LOGFIRE_GENAI_PRICES=true prices any model genai-prices knows, matched by model name. It is opt-in and lazily imported: users install genai-prices themselves into the hooks' Python, so the default install stays stdlib-only and offline. If the flag is set but the package is missing, cost is simply omitted (with a diagnostic) rather than erroring. genai-prices' Usage.input_tokens is the inclusive total (cache tokens are subsets billed at their own rates), the inverse of Anthropic's convention, so the combined input is passed and genai-prices splits it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
David's AICA here:
Problem
The cost table (
MODEL_PRICING) only covers Anthropic models (opus/sonnet/haiku, matched by substring). When Claude Code is pointed at an Anthropic-compatible endpoint viaANTHROPIC_BASE_URL— z.ai/GLM, Moonshot/Kimi, MiniMax, etc. — the model name matches nothing,get_model_pricesreturnsNone, andoperation.costis omitted. Token usage is recorded, but there's no cost.Change
Adds an opt-in fallback to
genai-pricesfor any non-Anthropic model:LOGFIRE_GENAI_PRICES=true.genai-pricesthemselves into the Python that runs the hooks. The default install stays stdlib-only and offline;hooks.json,pyproject.toml, and the invocation are unchanged.gen_ai.provider.name/gen_ai.system) reflects the resolved provider (e.g.moonshotai,minimax) instead of hardcodedanthropic.Token mapping note
genai-prices treats
Usage.input_tokensas the inclusive total, withcache_write_tokens/cache_read_tokensas subsets billed at their own rates — the inverse of Anthropic's convention (whereinput_tokensis uncached-only and cache tokens are additional). The patch passes the combined input (raw_input + cache_creation + cache_read) and lets genai-prices split it, so cached tokens are priced correctly.Testing
Verified the full matrix via the actual span-builder (
build_child_spans_from_calls):Anthropic pricing path is byte-for-byte unchanged.
ruff checkclean (no new findings).