feat: ✨ LiteLLM multi-provider model selection + catalog-driven UI - #111
Merged
Conversation
Replace the hand-rolled OpenAICompatClient with a LiteLLM-backed LiteLLMClient for openai/openrouter/ollama, behind the unchanged LLMClient.complete() interface. Makes OpenRouter models like openrouter/z-ai/glm-5.2 reliably selectable. - llm/litellm_client.py: forces the single expected tool; falls back to JSON-from-text and returns a text block (never raises) so the Gatekeeper prose fallback is preserved; real per-call cost via LiteLLM with a catalog fallback and cost_source tracking. - catalog.py: single source of cost truth (estimate_cost_cents, get_all_models, get_model_by_full_id); refreshed to current models (Opus 4.8 / Sonnet 5 / Haiku 4.5 / Fable 5) + verified OpenRouter models incl. z-ai/glm-5.2. - client.py (Anthropic) sources cost from the catalog (fixes $0 cost for new model IDs); claude-cli reports cost_source="subscription" ($0). - parsing.extract_json_object: shared boundary JSON parser (promoted from a private claude_cli helper that pulled in a Unix-only import). - web: GET /api/models endpoint + OpenRouter key validation; the settings Models section is now catalog-driven (provider-grouped dropdown with cost/context) instead of a hardcoded preset list. - litellm added as an optional dep (multi-provider extra, pinned >=1.91.0). Verified: ruff + mypy clean, tests green, live openrouter/z-ai/glm-5.2 call (real tool-call + cost source=provider), and a browser UI round-trip that persists the selection to config.yaml. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jul 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Replaces the hand-rolled
OpenAICompatClientwith a LiteLLM-backedLiteLLMClient(openai/openrouter/ollama) behind the unchangedLLMClient.complete()interface, and makes the model catalog the single source of truth wired to the settings UI. Goal: select OpenRouter models likeopenrouter/z-ai/glm-5.2reliably, with real per-call cost tracking.Adopts the proven pattern from mirra's PromptLab (LiteLLM + registry-as-cost-truth), adapted to this project's tool-calling contract.
Key changes
llm/litellm_client.py(new): forces the single expected tool; on a tool-less reply, parses JSON from the message body, else returns a text block (never raises) so the Gatekeeper's prose fallback is preserved; real cost via LiteLLMresponse_cost→ catalog fallback → honestunknown, with acost_sourcefield.catalog.py: single source of cost truth (estimate_cost_cents,get_all_models,get_model_by_full_id); refreshed to current models (Opus 4.8 / Sonnet 5 / Haiku 4.5 / Fable 5) + verified OpenRouter models incl.z-ai/glm-5.2.client.py(Anthropic, same transport) now sources cost from the catalog — fixes silent$0cost for current model IDs.claude-clireportscost_source="subscription"($0), isolated for easy change if subscription calls ever bill.parsing.extract_json_object: shared boundary JSON parser (promoted from a privateclaude_clihelper that dragged a Unix-onlypwdimport into the OpenRouter path).GET /api/modelsendpoint + OpenRouter key validation; the settings Models section is now catalog-driven (provider-grouped dropdown showing cost/context) instead of three drifting hardcoded lists. Free-textprovider/model-idretained.ClaudeClient/ClaudeCliClientuntouched (Anthropic SDK +claude -psubscription path stay as-is); litellm is an optional dep (multi-providerextra, pinned>=1.91.0).Verification
test_litellm_client.py; updated catalog/factory/llm_client/claude_cli;test_openai_compat_client.pyremoved).openrouter/z-ai/glm-5.2call through the factory: real tool call + cost0.0505¢,source=provider.config.yaml.Notes
pwdimport leak, and a Gatekeeper regression — all fixed here).developHEAD, so it is isolated from unrelated in-progress media/buttons work in the local tree.REUSABILITY.md,docs/CLAUDE.md) live in the separate docs repo and are committed there, not in this PR.🤖 Generated with Claude Code