Skip to content

Commit a852ead

Browse files
vsraccubitsclaude
andcommitted
feat: initial implementation of bud-model-catalog SDK
Multi-source LLM model catalog that fetches metadata from LiteLLM and truefoundry/models, merges with cost-accurate pricing, filters deprecated models, and returns a unified catalog keyed by TensorZero provider/model. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
0 parents  commit a852ead

24 files changed

Lines changed: 2127 additions & 0 deletions

.github/workflows/ci.yml

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
9+
jobs:
10+
test:
11+
runs-on: ubuntu-latest
12+
strategy:
13+
matrix:
14+
python-version: ["3.10", "3.11", "3.12", "3.13"]
15+
16+
steps:
17+
- uses: actions/checkout@v4
18+
19+
- name: Install uv
20+
uses: astral-sh/setup-uv@v4
21+
with:
22+
version: "latest"
23+
24+
- name: Set up Python ${{ matrix.python-version }}
25+
run: uv python install ${{ matrix.python-version }}
26+
27+
- name: Install dependencies
28+
run: uv sync --all-extras
29+
30+
- name: Run linting
31+
run: uv run ruff check src/ tests/
32+
33+
- name: Run tests
34+
run: uv run pytest tests/ -v --cov=src/bud_model_catalog --cov-report=xml
35+
36+
- name: Upload coverage
37+
uses: codecov/codecov-action@v4
38+
if: matrix.python-version == '3.12'
39+
with:
40+
files: ./coverage.xml
41+
fail_ci_if_error: false
42+
43+
build:
44+
runs-on: ubuntu-latest
45+
steps:
46+
- uses: actions/checkout@v4
47+
48+
- name: Install uv
49+
uses: astral-sh/setup-uv@v4
50+
with:
51+
version: "latest"
52+
53+
- name: Build package
54+
run: uv build
55+
56+
- name: Check package
57+
run: uvx twine check dist/*

.github/workflows/publish.yml

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
name: Publish to PyPI
2+
3+
on:
4+
release:
5+
types: [published]
6+
7+
jobs:
8+
build:
9+
name: Build distribution
10+
runs-on: ubuntu-latest
11+
steps:
12+
- uses: actions/checkout@v4
13+
14+
- name: Install uv
15+
uses: astral-sh/setup-uv@v4
16+
with:
17+
version: "latest"
18+
19+
- name: Build package
20+
run: uv build
21+
22+
- name: Store distribution packages
23+
uses: actions/upload-artifact@v4
24+
with:
25+
name: python-package-distributions
26+
path: dist/
27+
28+
publish-pypi:
29+
name: Publish to PyPI
30+
needs: build
31+
runs-on: ubuntu-latest
32+
environment:
33+
name: pypi
34+
url: https://pypi.org/p/bud-model-catalog
35+
permissions:
36+
id-token: write
37+
38+
steps:
39+
- name: Download distributions
40+
uses: actions/download-artifact@v4
41+
with:
42+
name: python-package-distributions
43+
path: dist/
44+
45+
- name: Publish to PyPI
46+
uses: pypa/gh-action-pypi-publish@release/v1
47+
48+
publish-testpypi:
49+
name: Publish to TestPyPI
50+
needs: build
51+
runs-on: ubuntu-latest
52+
if: github.event.release.prerelease
53+
environment:
54+
name: testpypi
55+
url: https://test.pypi.org/p/bud-model-catalog
56+
permissions:
57+
id-token: write
58+
59+
steps:
60+
- name: Download distributions
61+
uses: actions/download-artifact@v4
62+
with:
63+
name: python-package-distributions
64+
path: dist/
65+
66+
- name: Publish to TestPyPI
67+
uses: pypa/gh-action-pypi-publish@release/v1
68+
with:
69+
repository-url: https://test.pypi.org/legacy/

.gitignore

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Byte-compiled / optimized / DLL files
2+
__pycache__/
3+
*.pyc
4+
5+
# Virtual environments
6+
.venv/
7+
venv/
8+
9+
# Package metadata
10+
*.egg-info/
11+
12+
# Build artifacts
13+
dist/
14+
build/
15+
16+
# Generated output
17+
catalog.json
18+
19+
# Environment variables
20+
.env
21+
22+
# Tool caches
23+
.pytest_cache/
24+
.mypy_cache/
25+
.ruff_cache/

README.md

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
# bud-model-catalog
2+
3+
Multi-source LLM model catalog with cost-accurate pricing. Fetches model metadata from [LiteLLM](https://github.com/BerriAI/litellm) and [truefoundry/models](https://github.com/truefoundry/models), merges them with cost-accurate pricing, filters deprecated models, and returns a unified catalog keyed by TensorZero provider/model.
4+
5+
## Install
6+
7+
```bash
8+
pip install bud-model-catalog
9+
```
10+
11+
## Quick Start
12+
13+
```python
14+
from bud_model_catalog import CatalogClient
15+
16+
# Synchronous usage
17+
result = CatalogClient().fetch_catalog_sync()
18+
print(f"Fetched {len(result.models)} models")
19+
print(f"Stats: {result.stats}")
20+
```
21+
22+
## Async Usage
23+
24+
```python
25+
import asyncio
26+
from bud_model_catalog import CatalogClient, CatalogConfig
27+
28+
async def main():
29+
config = CatalogConfig(include_deprecated=True, timeout=60)
30+
client = CatalogClient(config)
31+
result = await client.fetch_catalog()
32+
33+
for key, model in list(result.models.items())[:5]:
34+
print(f"{key}: input={model.get('input_cost_per_token')}")
35+
36+
asyncio.run(main())
37+
```
38+
39+
Or use the module-level convenience function:
40+
41+
```python
42+
from bud_model_catalog import fetch_catalog
43+
44+
result = await fetch_catalog()
45+
```
46+
47+
## Configuration
48+
49+
All options are passed via `CatalogConfig`:
50+
51+
| Field | Type | Default | Description |
52+
|-------|------|---------|-------------|
53+
| `litellm_url` | `str` | GitHub raw URL | URL to the LiteLLM model prices JSON |
54+
| `ai_models_url` | `str` | GitHub archive URL | URL to the truefoundry/models ZIP archive |
55+
| `timeout` | `int` | `30` | HTTP request timeout in seconds (must be > 0) |
56+
| `include_deprecated` | `bool` | `False` | Whether to include deprecated models in output |
57+
| `max_retries` | `int` | `2` | Maximum retry attempts per HTTP request (with exponential backoff) |
58+
| `cache` | `bool` | `True` | Enable ETag-based conditional GET caching across calls |
59+
60+
```python
61+
from bud_model_catalog import CatalogConfig
62+
63+
config = CatalogConfig(
64+
timeout=60,
65+
include_deprecated=True,
66+
max_retries=3,
67+
cache=True,
68+
)
69+
```
70+
71+
Validation is enforced at construction time:
72+
73+
```python
74+
CatalogConfig(timeout=-1) # ValueError: timeout must be positive
75+
CatalogConfig(litellm_url="not-a-url") # ValueError: must be an HTTP(S) URL
76+
```
77+
78+
## Error Handling
79+
80+
```python
81+
from bud_model_catalog import CatalogClient, CatalogConfig, SourceFetchError
82+
83+
try:
84+
result = CatalogClient().fetch_catalog_sync()
85+
except SourceFetchError as e:
86+
print(f"Failed to fetch data: {e}")
87+
```
88+
89+
- `SourceFetchError` — raised when LiteLLM fetch fails (HTTP error, invalid JSON, timeout)
90+
- ai-models failures are handled gracefully — the SDK falls back to LiteLLM-only costs
91+
92+
## API Reference
93+
94+
### `CatalogClient`
95+
96+
Main entry point for fetching the catalog.
97+
98+
- `CatalogClient(config=None)` — create a client with optional `CatalogConfig`
99+
- `await client.fetch_catalog()` — async fetch, returns `CatalogResult`
100+
- `client.fetch_catalog_sync()` — sync wrapper, safe in both sync and async contexts
101+
102+
### `CatalogResult`
103+
104+
Pydantic model returned from fetch operations.
105+
106+
- `models: dict[str, dict]` — merged model catalog keyed by `{provider}/{model}`
107+
- `stats: MergeStats` — merge statistics
108+
- `litellm_fetched_at: datetime` — timestamp of LiteLLM fetch
109+
- `ai_models_fetched_at: datetime | None` — timestamp of ai-models fetch (None if failed/skipped)
110+
111+
### `MergeStats`
112+
113+
- `total_litellm` — total models from LiteLLM source
114+
- `total_output` — models in final output
115+
- `matched` — models matched with ai-models data
116+
- `unmatched` — models without ai-models match
117+
- `deprecated_removed` — models filtered as deprecated
118+
- `cost_fields_updated` — individual cost field values updated from ai-models
119+
120+
## Logging
121+
122+
The SDK uses Python's `logging` module. Enable output to see fetch/merge details:
123+
124+
```python
125+
import logging
126+
logging.basicConfig(level=logging.INFO)
127+
```
128+
129+
Key log messages:
130+
- `INFO` — fetch counts, merge statistics, cache hits
131+
- `WARNING` — ai-models fallback, malformed YAML files skipped, retry attempts
132+
133+
## Development
134+
135+
```bash
136+
# Install dev dependencies
137+
pip install -e ".[dev]"
138+
139+
# Run tests
140+
pytest -v
141+
142+
# Lint
143+
ruff check src/ tests/
144+
```

example/fetch_catalog.py

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
"""Example: fetch and inspect the Bud model catalog."""
2+
3+
from __future__ import annotations
4+
5+
import argparse
6+
import json
7+
import sys
8+
from collections import Counter
9+
10+
from bud_model_catalog import CatalogClient, CatalogConfig, SourceFetchError
11+
12+
13+
def main() -> None:
14+
parser = argparse.ArgumentParser(description="Fetch the Bud model catalog and print a summary.")
15+
parser.add_argument("--output", "-o", help="Write the full model catalog to a JSON file.")
16+
parser.add_argument(
17+
"--include-deprecated",
18+
action="store_true",
19+
default=False,
20+
help="Include deprecated models in the catalog (default: exclude them).",
21+
)
22+
args = parser.parse_args()
23+
24+
# ── 1. Fetch catalog with CLI-driven config ──────────────────────────
25+
config = CatalogConfig(include_deprecated=args.include_deprecated)
26+
print(f"Fetching catalog (include_deprecated={args.include_deprecated}) ...")
27+
client = CatalogClient(config)
28+
29+
try:
30+
result = client.fetch_catalog_sync()
31+
except SourceFetchError as exc:
32+
print(f"Failed to fetch catalog: {exc}", file=sys.stderr)
33+
sys.exit(1)
34+
35+
# ── 2. Inspect results ──────────────────────────────────────────────
36+
stats = result.stats
37+
print(f"\nTotal models in catalog : {stats.total_output}")
38+
print(f"LiteLLM source models : {stats.total_litellm}")
39+
print(f"Matched with ai-models : {stats.matched}")
40+
print(f"Unmatched : {stats.unmatched}")
41+
print(f"Deprecated removed : {stats.deprecated_removed}")
42+
print(f"Cost fields updated : {stats.cost_fields_updated}")
43+
print(f"\nLiteLLM fetched at : {result.litellm_fetched_at}")
44+
print(f"AI-models fetched at : {result.ai_models_fetched_at}")
45+
46+
# ── 3. Browse by provider ───────────────────────────────────────────
47+
provider_counts: Counter[str] = Counter()
48+
for info in result.models.values():
49+
provider_counts[info.get("litellm_provider", "unknown")] += 1
50+
51+
print(f"\n{'Provider':<30} {'Models':>6}")
52+
print("-" * 38)
53+
for provider, count in provider_counts.most_common(10):
54+
print(f"{provider:<30} {count:>6}")
55+
if len(provider_counts) > 10:
56+
print(f"... and {len(provider_counts) - 10} more providers")
57+
58+
# ── 4. Inspect a single model ───────────────────────────────────────
59+
first_key = next(iter(result.models))
60+
model = result.models[first_key]
61+
print(f"\nSample model: {first_key}")
62+
for field in ("litellm_provider", "max_tokens", "max_input_tokens", "max_output_tokens",
63+
"input_cost_per_token", "output_cost_per_token"):
64+
if field in model:
65+
print(f" {field}: {model[field]}")
66+
67+
# ── 5. Optional JSON dump ────────────────────────────────────────────
68+
if args.output:
69+
with open(args.output, "w") as fh:
70+
json.dump(result.models, fh, indent=2, default=str)
71+
print(f"\nCatalog written to {args.output}")
72+
73+
74+
if __name__ == "__main__":
75+
main()

pyproject.toml

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
[build-system]
2+
requires = ["hatchling"]
3+
build-backend = "hatchling.build"
4+
5+
[project]
6+
name = "bud-model-catalog"
7+
version = "0.1.0"
8+
description = "Multi-source LLM model catalog with cost-accurate pricing"
9+
requires-python = ">=3.10"
10+
dependencies = [
11+
"httpx>=0.27",
12+
"pydantic>=2.0",
13+
"pyyaml>=6.0",
14+
]
15+
16+
[project.optional-dependencies]
17+
dev = ["pytest>=8.0", "pytest-asyncio>=0.23", "respx>=0.21", "ruff>=0.4"]
18+
19+
[tool.hatch.build.targets.wheel]
20+
packages = ["src/bud_model_catalog"]
21+
22+
[tool.pytest.ini_options]
23+
asyncio_mode = "auto"

0 commit comments

Comments
 (0)