fix(messages): fall back to stripped model name for cost calculation#4877
fix(messages): fall back to stripped model name for cost calculation#4877JasonCZMeng wants to merge 1 commit intopydantic:mainfrom
Conversation
Some providers like OpenRouter return model names with a date suffix (e.g. `openai/gpt-5.2-20251211`) that `genai-prices` does not recognize. When the initial cost lookup fails, strip the trailing `-YYYYMMDD` suffix and retry. Fixes pydantic#4839
| try: | ||
| return calc_price( | ||
| self.usage, | ||
| self.model_name, | ||
| provider_id=self.provider_name, | ||
| genai_request_timestamp=self.timestamp, | ||
| ) | ||
| except LookupError: | ||
| pass | ||
| # Some providers (e.g. OpenRouter) return model names with a date suffix | ||
| # like "openai/gpt-5.2-20251211". Strip the suffix and retry. | ||
| normalized = _strip_model_date_suffix(self.model_name) | ||
| if normalized != self.model_name: | ||
| if self.provider_url: | ||
| try: | ||
| return calc_price( | ||
| self.usage, | ||
| normalized, | ||
| provider_api_url=self.provider_url, | ||
| genai_request_timestamp=self.timestamp, | ||
| ) | ||
| except LookupError: | ||
| pass | ||
| return calc_price( | ||
| self.usage, | ||
| normalized, | ||
| provider_id=self.provider_name, | ||
| genai_request_timestamp=self.timestamp, | ||
| ) | ||
| raise LookupError(f'Could not find pricing for model {self.model_name!r}') |
There was a problem hiding this comment.
🟡 Duplicate try/except LookupError blocks violate rule:894 and cause inconsistent error messages
The cost() method repeats the same two-step fallback pattern (try provider_api_url, then try provider_id) twice — once for the original model name and once for the normalized (date-stripped) name. This violates rule:894 from pydantic_ai_slim/pydantic_ai/AGENTS.md ("Consolidate try/except blocks catching the same exception into one block") and rule:559 from agent_docs/index.md ("Consolidate duplicate logic across conditional branches").
Beyond the style violation, there's also an inconsistency: when the model name has a date suffix and all lookups fail, the final unwrapped calc_price on line 1808 raises a LookupError with calc_price's own message (referencing the normalized name). But when there's no date suffix, the explicit raise LookupError(...) on line 1814 references the original name. A loop-based consolidation would fix both issues.
| try: | |
| return calc_price( | |
| self.usage, | |
| self.model_name, | |
| provider_id=self.provider_name, | |
| genai_request_timestamp=self.timestamp, | |
| ) | |
| except LookupError: | |
| pass | |
| # Some providers (e.g. OpenRouter) return model names with a date suffix | |
| # like "openai/gpt-5.2-20251211". Strip the suffix and retry. | |
| normalized = _strip_model_date_suffix(self.model_name) | |
| if normalized != self.model_name: | |
| if self.provider_url: | |
| try: | |
| return calc_price( | |
| self.usage, | |
| normalized, | |
| provider_api_url=self.provider_url, | |
| genai_request_timestamp=self.timestamp, | |
| ) | |
| except LookupError: | |
| pass | |
| return calc_price( | |
| self.usage, | |
| normalized, | |
| provider_id=self.provider_name, | |
| genai_request_timestamp=self.timestamp, | |
| ) | |
| raise LookupError(f'Could not find pricing for model {self.model_name!r}') | |
| # Build candidate model names: original first, then date-stripped variant. | |
| names = [self.model_name] | |
| normalized = _strip_model_date_suffix(self.model_name) | |
| if normalized != self.model_name: | |
| names.append(normalized) | |
| for name in names: | |
| if self.provider_url: | |
| try: | |
| return calc_price( | |
| self.usage, | |
| name, | |
| provider_api_url=self.provider_url, | |
| genai_request_timestamp=self.timestamp, | |
| ) | |
| except LookupError: | |
| pass | |
| try: | |
| return calc_price( | |
| self.usage, | |
| name, | |
| provider_id=self.provider_name, | |
| genai_request_timestamp=self.timestamp, | |
| ) | |
| except LookupError: | |
| pass | |
| raise LookupError(f'Could not find pricing for model {self.model_name!r}') |
Was this helpful? React with 👍 or 👎 to provide feedback.
| def test_strip_model_date_suffix(): | ||
| """_strip_model_date_suffix removes YYYYMMDD date suffixes from model names.""" | ||
| from pydantic_ai.messages import _strip_model_date_suffix | ||
|
|
||
| # Date suffix should be stripped | ||
| assert _strip_model_date_suffix('openai/gpt-5.2-20251211') == 'openai/gpt-5.2' | ||
| assert _strip_model_date_suffix('gpt-4o-2024-08-06-20240806') == 'gpt-4o-2024-08-06' | ||
| assert _strip_model_date_suffix('claude-sonnet-4-5-20250514') == 'claude-sonnet-4-5' | ||
|
|
||
| # No date suffix — unchanged | ||
| assert _strip_model_date_suffix('gpt-5.2') == 'gpt-5.2' | ||
| assert _strip_model_date_suffix('openai/gpt-5.2') == 'openai/gpt-5.2' | ||
| assert _strip_model_date_suffix('claude-sonnet-4-5') == 'claude-sonnet-4-5' | ||
|
|
||
| # Short numbers should NOT be stripped (not 8-digit date) | ||
| assert _strip_model_date_suffix('gpt-4o-mini-123') == 'gpt-4o-mini-123' |
There was a problem hiding this comment.
🟡 Test directly tests private _strip_model_date_suffix function instead of public API
test_strip_model_date_suffix imports and tests _strip_model_date_suffix (a private function prefixed with _) directly. This violates rule:177 from tests/AGENTS.md: "Test through public APIs, not private methods (prefixed with _) or helpers." The date-stripping behavior should be tested through the public ModelResponse.cost() method (which is what test_model_response_cost_with_date_suffix already does), making this test redundant.
Prompt for agents
Remove the test_strip_model_date_suffix function (lines 1392-1407 in tests/test_messages.py) entirely. The date-stripping logic it tests is an internal implementation detail of ModelResponse.cost(), and is already exercised through the public API by test_model_response_cost_with_date_suffix. If additional edge-case coverage is desired (e.g. short numeric suffixes not being stripped), add those as additional test cases within test_model_response_cost_with_date_suffix by constructing ModelResponse objects with those model names.
Was this helpful? React with 👍 or 👎 to provide feedback.
Summary
When providers like OpenRouter return model names with a date suffix (e.g.
openai/gpt-5.2-20251211),genai-pricesdoesn't recognize them andModelResponse.cost()raisesLookupError. This causesoperation.costto be missing from traces.Fix
Added a fallback in
ModelResponse.cost(): when the initialcalc_pricelookup fails, strip the trailing-YYYYMMDDdate suffix from the model name and retry. The regex-\d{8}$matches exactly 8-digit date suffixes at the end of the string.The fallback chain is now:
provider_api_url(most specific)provider_idprovider_api_urlprovider_idLookupErrorTests
test_strip_model_date_suffix: Unit test for the suffix stripping helpertest_model_response_cost_with_date_suffix: Integration test verifyingModelResponse.cost()works with date-suffixed model namesFixes #4839