Skip to content

feat: add LiteLLM as unified LLM provider#136

Merged
brokermr810 merged 2 commits into
brokermr810:mainfrom
RheagalFire:feat/add-litellm-provider
Jun 3, 2026
Merged

feat: add LiteLLM as unified LLM provider#136
brokermr810 merged 2 commits into
brokermr810:mainfrom
RheagalFire:feat/add-litellm-provider

Conversation

@RheagalFire
Copy link
Copy Markdown
Contributor

Summary

Adds LiteLLM as a new LLM provider option (LLM_PROVIDER=litellm), giving QuantDinger users access to 100+ LLM providers (Anthropic, Azure, Bedrock, Ollama, Groq, Together, Fireworks, etc.) through litellm.completion() as an SDK dependency. Additive only - all existing providers untouched.

Changes

  • backend_api_python/app/services/llm.py: added LITELLM = "litellm" to LLMProvider enum, _call_litellm() method with drop_params=True, dispatch in call_llm_api()
  • backend_api_python/app/config/api_keys.py: added LITELLM_API_KEY property (optional)
  • backend_api_python/app/routes/settings.py: added LiteLLM to provider dropdown and settings group (API Key, Model, Base URL)
  • backend_api_python/requirements.txt: added litellm>=1.80,<1.87

Test plan

  • Tested locally with docker compose up -d --build
  • Backend logs show no errors
  • Relevant pytest tests pass (691 passed, 5 skipped, 0 failed)

API documentation (if routes/schemas changed)

  • No routes/schemas changed - LiteLLM added as backend provider option only
  • No /api/agent/v1 routes changed
  • No breaking API changes

Screenshots (if UI change)

NA

@RheagalFire
Copy link
Copy Markdown
Contributor Author

cc @brokermr810

@brokermr810 brokermr810 merged commit 5722bd0 into brokermr810:main Jun 3, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants