-
Notifications
You must be signed in to change notification settings - Fork 1.9k
fix(messages): fall back to stripped model name for cost calculation #4877
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1387,3 +1387,53 @@ def test_args_as_dict_raise_if_invalid_non_dict_json(): | |
| part = ToolCallPart(tool_name='test_tool', args='[1, 2, 3]') | ||
| with pytest.raises(AssertionError): | ||
| part.args_as_dict(raise_if_invalid=True) | ||
|
|
||
|
|
||
| def test_strip_model_date_suffix(): | ||
| """_strip_model_date_suffix removes YYYYMMDD date suffixes from model names.""" | ||
| from pydantic_ai.messages import _strip_model_date_suffix | ||
|
|
||
| # Date suffix should be stripped | ||
| assert _strip_model_date_suffix('openai/gpt-5.2-20251211') == 'openai/gpt-5.2' | ||
| assert _strip_model_date_suffix('gpt-4o-2024-08-06-20240806') == 'gpt-4o-2024-08-06' | ||
| assert _strip_model_date_suffix('claude-sonnet-4-5-20250514') == 'claude-sonnet-4-5' | ||
|
|
||
| # No date suffix — unchanged | ||
| assert _strip_model_date_suffix('gpt-5.2') == 'gpt-5.2' | ||
| assert _strip_model_date_suffix('openai/gpt-5.2') == 'openai/gpt-5.2' | ||
| assert _strip_model_date_suffix('claude-sonnet-4-5') == 'claude-sonnet-4-5' | ||
|
|
||
| # Short numbers should NOT be stripped (not 8-digit date) | ||
| assert _strip_model_date_suffix('gpt-4o-mini-123') == 'gpt-4o-mini-123' | ||
devin-ai-integration[bot] marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
|
|
||
| def test_model_response_cost_with_date_suffix(): | ||
| """ModelResponse.cost() should fall back to stripped model name when date suffix prevents lookup.""" | ||
| from decimal import Decimal | ||
|
|
||
| from pydantic_ai.messages import ModelResponse | ||
| from pydantic_ai.usage import RequestUsage | ||
|
Comment on lines
+1394
to
+1397
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 Inline imports in Rule:464 from Prompt for agentsWas this helpful? React with 👍 or 👎 to provide feedback. |
||
|
|
||
| usage = RequestUsage(input_tokens=100, output_tokens=50) | ||
|
|
||
| # A model name that genai-prices recognizes | ||
| response_clean = ModelResponse( | ||
| parts=[], | ||
| model_name='gpt-5.2', | ||
| provider_name='openai', | ||
| usage=usage, | ||
| ) | ||
| cost_clean = response_clean.cost() | ||
| assert cost_clean.total_price > Decimal('0') | ||
|
|
||
| # Same model but with date suffix (as returned by OpenRouter) | ||
| response_dated = ModelResponse( | ||
| parts=[], | ||
| model_name='openai/gpt-5.2-20251211', | ||
| provider_name='litellm', | ||
| usage=usage, | ||
| ) | ||
| cost_dated = response_dated.cost() | ||
| # Should succeed via fallback and return same pricing | ||
| assert cost_dated.total_price > Decimal('0') | ||
| assert cost_dated.total_price == cost_clean.total_price | ||
|
Comment on lines
+1392
to
+1421
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🚩 Test relies on specific genai-prices model data being present The test at Was this helpful? React with 👍 or 👎 to provide feedback. |
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔴 Module-level
_strip_model_date_suffixis a single-use delegation wrapper that should be inlined (rule:14, rule:176)_strip_model_date_suffixis a module-level function used exactly once insideModelResponse.cost()(pydantic_ai_slim/pydantic_ai/messages.py:1801). The function body is a single delegation call:return _DATE_SUFFIX_RE.sub('', model_name). Rule:14 says to inline single-use helpers that only wrap delegation, and rule:176 says to scope helpers to their single usage site rather than defining them at module level. The regex constant_DATE_SUFFIX_REat module level is fine per rule:499, but the wrapper function should be inlined at the call site.Prompt for agents
Was this helpful? React with 👍 or 👎 to provide feedback.