Skip to content

Update the models metadata#74

Merged
ArtyomVancyan merged 2 commits intomainfrom
update-models-metadata-923e193
Apr 28, 2026
Merged

Update the models metadata#74
ArtyomVancyan merged 2 commits intomainfrom
update-models-metadata-923e193

Conversation

@ArtyomVancyan
Copy link
Copy Markdown
Member

This PR updates the models metadata to keep the repository up-to-date with the upstream model costs.

@ArtyomVancyan
Copy link
Copy Markdown
Member Author

@copilot, please fix the following issue and update this PR:

tests/lib/test_formatter.py ................................F..F.        [ 78%]
tests/lib/test_topology.py .....                                         [ 89%]
tests/lib/test_watching.py .....                                         [100%]

=================================== FAILURES ===================================
____________________ TestCosts.test_known_model_with_tokens ____________________

self = <lib.test_formatter.TestCosts object at 0x7f30b9925b10>

    def test_known_model_with_tokens(self):
        result = Formatter.costs("openai/gpt-4o-mini", 1000, 2000)
>       assert result == {"cached": "0.0", "total": "0.0"}
E       AssertionError: assert {'cached': '0...al': '0.0012'} == {'cached': '0...total': '0.0'}
E         
E         Differing items:
E         {'total': '0.0012'} != {'total': '0.0'}
E         {'cached': '0.00008'} != {'cached': '0.0'}
E         Use -v to get more diff

tests/lib/test_formatter.py:307: AssertionError
____________________ TestMetrics.test_cost_for_known_model _____________________

self = <lib.test_formatter.TestMetrics object at 0x7f30b9924040>

    def test_cost_for_known_model(self):
        run = make_metrics_run(
            extra={"ls_model_name": "openai/gpt-4o-mini"},
            outputs={"total_tokens": 2000, "cached_tokens": 1000},
        )
        result = Formatter.metrics(run)
>       assert result["costs"] == {"cached": "0.0", "total": "0.0"}
E       AssertionError: assert {'cached': '0...al': '0.0012'} == {'cached': '0...total': '0.0'}
E         
E         Differing items:
E         {'total': '0.0012'} != {'total': '0.0'}
E         {'cached': '0.00008'} != {'cached': '0.0'}
E         Use -v to get more diff

tests/lib/test_formatter.py:327: AssertionError
=========================== short test summary info ============================
FAILED tests/lib/test_formatter.py::TestCosts::test_known_model_with_tokens
FAILED tests/lib/test_formatter.py::TestMetrics::test_cost_for_known_model - ...
========================= 2 failed, 45 passed in 2.83s =========================

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 28, 2026

@copilot, please fix the following issue and update this PR:

...

Fixed in 59bb0d7. The PR updated openai/gpt-4o-mini costs from zero to cache_read: 0.08, output: 0.6, so I updated both test assertions to reflect the actual computed costs: {"cached": "0.00008", "total": "0.0012"}.

@ArtyomVancyan ArtyomVancyan merged commit a2b9757 into main Apr 28, 2026
16 checks passed
@ArtyomVancyan ArtyomVancyan deleted the update-models-metadata-923e193 branch April 28, 2026 06:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants