Releases: BerriAI/litellm
v1.66.3-nightly
What's Changed
- Add aggregate spend by tag by @krrishdholakia in #10071
- Add OpenAI o3 & o4-mini by @PeterDaveHello in #10065
- Add new
/tag/daily/activity
endpoint + Add tag dashboard to UI by @krrishdholakia in #10073 - Add team based usage dashboard at 1m+ spend logs (+ new
/team/daily/activity
API) by @krrishdholakia in #10081 - [Feat SSO] Add LiteLLM SCIM Integration for Team and User management by @ishaan-jaff in #10072
- Virtual Keys: Filter by key alias (#10035) by @ishaan-jaff in #10085
- Add new
/vertex_ai/discovery
route - enables calling AgentBuilder API routes by @krrishdholakia in #10084 - fix(o_series_transformation.py): correctly map o4 to openai o_series … by @krrishdholakia in #10079
Full Changelog: v1.66.2-nightly...v1.66.3-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.66.3-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Failed ❌ | 250.0 | 302.3290337319068 | 6.097097387542003 | 0.04679789661490572 | 1824 | 14 | 218.4401190000358 | 5459.562037000012 |
Aggregated | Failed ❌ | 250.0 | 302.3290337319068 | 6.097097387542003 | 0.04679789661490572 | 1824 | 14 | 218.4401190000358 | 5459.562037000012 |
v1.66.2.dev1
What's Changed
- Add aggregate spend by tag by @krrishdholakia in #10071
- Add OpenAI o3 & o4-mini by @PeterDaveHello in #10065
- Add new
/tag/daily/activity
endpoint + Add tag dashboard to UI by @krrishdholakia in #10073 - Add team based usage dashboard at 1m+ spend logs (+ new
/team/daily/activity
API) by @krrishdholakia in #10081 - [Feat SSO] Add LiteLLM SCIM Integration for Team and User management by @ishaan-jaff in #10072
- Virtual Keys: Filter by key alias (#10035) by @ishaan-jaff in #10085
- Add new
/vertex_ai/discovery
route - enables calling AgentBuilder API routes by @krrishdholakia in #10084 - fix(o_series_transformation.py): correctly map o4 to openai o_series … by @krrishdholakia in #10079
Full Changelog: v1.66.2-nightly...v1.66.2.dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.66.2.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed ✅ | 200.0 | 242.7078390639904 | 6.1689738182726535 | 0.0 | 1844 | 0 | 181.44264199997906 | 6553.659710999966 |
Aggregated | Passed ✅ | 200.0 | 242.7078390639904 | 6.1689738182726535 | 0.0 | 1844 | 0 | 181.44264199997906 | 6553.659710999966 |
v1.66.2-nightly
What's Changed
- Fix azure tenant id check from env var + response_format check on api_version 2025+ by @krrishdholakia in #9993
- Add
/vllm
and/mistral
passthrough endpoints by @krrishdholakia in #10002 - CI/CD fix mock tests by @ishaan-jaff in #10003
- Setting
litellm.modify_params
via environment variables by @Eoous in #9964 - Support checking provider
/models
endpoints on proxy/v1/models
endpoint by @krrishdholakia in #9958 - Update AWS bedrock regions by @Schnitzel in #9430
- Fix case where only system messages are passed to Gemini by @NolanTrem in #9992
- Revert "Fix case where only system messages are passed to Gemini" by @krrishdholakia in #10027
- chore(docs): Update logging.md by @mrlorentx in #10006
- build(deps): bump @babel/runtime from 7.23.9 to 7.27.0 in /ui/litellm-dashboard by @dependabot in #10001
- Fix typo: Entrata -> Entra in code by @msabramo in #9922
- Retain schema field ordering for google gemini and vertex by @adrianlyjak in #9828
- Revert "Retain schema field ordering for google gemini and vertex" by @krrishdholakia in #10038
- Add aggregate team based usage logging by @krrishdholakia in #10039
- [UI Polish] UI fixes for cache control injection settings by @ishaan-jaff in #10031
- [UI] Bug Fix - Show created_at and updated_at for Users Page by @ishaan-jaff in #10033
- [Feat - Cost Tracking improvement] Track prompt caching metrics in DailyUserSpendTransactions by @ishaan-jaff in #10029
- Fix gcs pub sub logging with env var GCS_PROJECT_ID by @krrishdholakia in #10042
- Add property ordering for vertex ai schema (#9828) + Fix combining multiple tool calls by @krrishdholakia in #10040
- [Docs] Auto prompt caching by @ishaan-jaff in #10044
- Add litellm call id passing to Aim guardrails on pre and post-hooks calls by @hxmichael in #10021
- /utils/token_counter: get model_info from deployment directly by @chaofuyang in #10047
- [Bug Fix] Azure Blob Storage fixes by @ishaan-jaff in #10059
- build(deps): bump http-proxy-middleware from 2.0.7 to 2.0.9 in /docs/my-website by @dependabot in #10064
- fix(stream_chunk_builder_utils.py): don't set index on modelresponse by @krrishdholakia in #10063
- fix(llm_http_handler.py): fix fake streaming by @krrishdholakia in #10061
New Contributors
- @Eoous made their first contribution in #9964
- @mrlorentx made their first contribution in #10006
- @hxmichael made their first contribution in #10021
- @chaofuyang made their first contribution in #10047
Full Changelog: v1.66.1-nightly...v1.66.2-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.66.2-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed ✅ | 190.0 | 244.45508035967268 | 6.136194497326665 | 0.0 | 1835 | 0 | 169.77143499997283 | 8723.871383000016 |
Aggregated | Passed ✅ | 190.0 | 244.45508035967268 | 6.136194497326665 | 0.0 | 1835 | 0 | 169.77143499997283 | 8723.871383000016 |
v1.66.1-nightly
What's Changed
- build(model_prices_and_context_window.json): add gpt-4.1 pricing by @krrishdholakia in #9990
- [Fixes/QA] For gpt-4.1 costs by @ishaan-jaff in #9991
- Fix cost for Phi-4-multimodal output tokens by @emerzon in #9880
- chore(docs): update ordering of logging & observability docs by @marcklingen in #9994
- Updated cohere v2 passthrough by @krrishdholakia in #9997
- [Feat] Add support for
cache_control_injection_points
for Anthropic API, Bedrock API by @ishaan-jaff in #9996 - [UI] Allow setting prompt
cache_control_injection_points
by @ishaan-jaff in #10000
Full Changelog: v1.66.0-nightly...v1.66.1-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.66.1-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed ✅ | 220.0 | 243.74385918230334 | 6.268015361621096 | 0.0 | 1876 | 0 | 197.45038600001408 | 3855.600032000012 |
Aggregated | Passed ✅ | 220.0 | 243.74385918230334 | 6.268015361621096 | 0.0 | 1876 | 0 | 197.45038600001408 | 3855.600032000012 |
v1.66.0-stable
What's Changed
- build(deps): bump @babel/runtime from 7.26.0 to 7.27.0 in /docs/my-website by @dependabot in #9934
- fix: correct the cost for 'gemini/gemini-2.5-pro-preview-03-25' by @n1lanjan in #9896
- Litellm add managed files db by @krrishdholakia in #9930
- [DB / Infra] Add new column team_member_permissions by @ishaan-jaff in #9941
- fix(factory.py): correct indentation for message index increment in ollama, This fixes bug #9822 by @djshaw01 in #9943
- fix(litellm_proxy_extras): add baselining db script by @krrishdholakia in #9942
- [Team Member permissions] - Fixes by @ishaan-jaff in #9945
- Litellm managed files docs by @krrishdholakia in #9948
- [v1.66.0-stable] Release notes by @ishaan-jaff in #9952
- [Docs] v1.66.0-stable fixes by @ishaan-jaff in #9953
- stable release note fixes by @ishaan-jaff in #9954
- Fix filtering litellm-dashboard keys for internal users + prevent flooding spend logs with admin endpoint errors by @krrishdholakia in #9955
- [UI QA checklist] by @ishaan-jaff in #9957
New Contributors
Full Changelog: v1.65.8-nightly...v1.66.0-stable
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.66.0-stable
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed ✅ | 250.0 | 282.9933559715544 | 5.995117478456652 | 0.0 | 1793 | 0 | 223.97943800001485 | 5176.803935999998 |
Aggregated | Passed ✅ | 250.0 | 282.9933559715544 | 5.995117478456652 | 0.0 | 1793 | 0 | 223.97943800001485 | 5176.803935999998 |
v1.66.0-nightly
What's Changed
- build(deps): bump @babel/runtime from 7.26.0 to 7.27.0 in /docs/my-website by @dependabot in #9934
- fix: correct the cost for 'gemini/gemini-2.5-pro-preview-03-25' by @n1lanjan in #9896
- Litellm add managed files db by @krrishdholakia in #9930
- [DB / Infra] Add new column team_member_permissions by @ishaan-jaff in #9941
- fix(factory.py): correct indentation for message index increment in ollama, This fixes bug #9822 by @djshaw01 in #9943
- fix(litellm_proxy_extras): add baselining db script by @krrishdholakia in #9942
- [Team Member permissions] - Fixes by @ishaan-jaff in #9945
- Litellm managed files docs by @krrishdholakia in #9948
- [v1.66.0-stable] Release notes by @ishaan-jaff in #9952
- [Docs] v1.66.0-stable fixes by @ishaan-jaff in #9953
- stable release note fixes by @ishaan-jaff in #9954
- Fix filtering litellm-dashboard keys for internal users + prevent flooding spend logs with admin endpoint errors by @krrishdholakia in #9955
- [UI QA checklist] by @ishaan-jaff in #9957
New Contributors
Full Changelog: v1.65.8-nightly...v1.66.0-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.66.0-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed ✅ | 230.0 | 252.49209995793416 | 6.279241190720279 | 0.0 | 1878 | 0 | 200.85592700002053 | 5135.250711999987 |
Aggregated | Passed ✅ | 230.0 | 252.49209995793416 | 6.279241190720279 | 0.0 | 1878 | 0 | 200.85592700002053 | 5135.250711999987 |
v1.65.8-nightly
What's Changed
- Revert avglogprobs change + Add azure/gpt-4o-realtime-audio cost tracking by @krrishdholakia in #9893
- Realtime API: Support 'base_model' cost tracking + show response in spend logs (if enabled) by @krrishdholakia in #9897
- Simplify calling gemini models w/ file id by @krrishdholakia in #9903
- feat: add extraEnvVars to the helm deployment by @mknet3 in #9292
- [Feat - UI] - Allow setting Default Team setting when LiteLLM SSO auto creates teams by @ishaan-jaff in #9918
- Fix typo: Entrata -> Entra in docs by @msabramo in #9921
- [Feat - PR1] Add xAI grok-3 models to LiteLLM by @ishaan-jaff in #9920
- [Feat - Team Member Permissions] - CRUD Endpoints for managing team member permissions by @ishaan-jaff in #9919
- [Feat] Add litellm.supports_reasoning() util to track if an llm supports reasoning by @ishaan-jaff in #9923
- [Feat] Add reasoning_effort support for
xai/grok-3-mini-beta
model family by @ishaan-jaff in #9932 - [UI] Render Reasoning content, ttft, usage metrics on test key page by @ishaan-jaff in #9931
- [UI] - Add Managing Team Member permissions on UI by @ishaan-jaff in #9927
- [UI] Linting fixes by @ishaan-jaff in #9933
- Support CRUD endpoints for Managed Files by @krrishdholakia in #9924
- fix(databricks/common_utils.py): fix custom endpoint check by @krrishdholakia in #9925
- fix(transformation.py): correctly translate 'thinking' param for lite… by @krrishdholakia in #9904
Full Changelog: v1.65.7-nightly...v1.65.8-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.65.8-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed ✅ | 220.0 | 248.0753682003237 | 6.194614175051195 | 0.0 | 1852 | 0 | 194.34754100001328 | 4413.887686999999 |
Aggregated | Passed ✅ | 220.0 | 248.0753682003237 | 6.194614175051195 | 0.0 | 1852 | 0 | 194.34754100001328 | 4413.887686999999 |
v1.65.7-nightly
What's Changed
- [Feat SSO] - Allow admins to set
default_team_params
to have default params for when litellm SSO creates default teams by @ishaan-jaff in #9895 - [Feat] Emit Key, Team Budget metrics on a cron job schedule by @ishaan-jaff in #9528
- [Bug Fix MSFT SSO] Use correct field for user email when using MSFT SSO by @ishaan-jaff in #9886
- [Docs] Tutorial using MSFT auto team assignment with LiteLLM by @ishaan-jaff in #9898
Full Changelog: v1.65.6-nightly...v1.65.7-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.65.7-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed ✅ | 240.0 | 261.69840098746454 | 6.131387078558505 | 0.0 | 1835 | 0 | 214.285206999989 | 3626.6518760000395 |
Aggregated | Passed ✅ | 240.0 | 261.69840098746454 | 6.131387078558505 | 0.0 | 1835 | 0 | 214.285206999989 | 3626.6518760000395 |
v1.65.6-nightly
What's Changed
- Fix anthropic prompt caching cost calc + trim logged message in db by @krrishdholakia in #9838
- feat(realtime/): add token tracking + log usage object in spend logs … by @krrishdholakia in #9843
- fix(cost_calculator.py): handle custom pricing at deployment level fo… by @krrishdholakia in #9855
Full Changelog: v1.65.5-nightly...v1.65.6-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.65.6-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed ✅ | 190.0 | 209.99145276997868 | 6.188819872716192 | 0.0 | 1852 | 0 | 167.33176299999286 | 4428.401366999992 |
Aggregated | Passed ✅ | 190.0 | 209.99145276997868 | 6.188819872716192 | 0.0 | 1852 | 0 | 167.33176299999286 | 4428.401366999992 |
v1.65.5-nightly
What's Changed
- build: bump litellm-proxy-extras version by @krrishdholakia in #9771
- Update model_prices by @aoaim in #9768
- Move daily user transaction logging outside of 'disable_spend_logs' flag - different tables by @krrishdholakia in #9772
- Add inference providers support for Hugging Face (#8258) (#9738) by @krrishdholakia in #9773
- [UI Bug fix] Don't show duplicate models on Team Admin models page by @ishaan-jaff in #9775
- [UI QA/Bug Fix] - Don't change team, key, org, model values on scroll by @ishaan-jaff in #9776
- [UI Polish] - Polish login screen by @ishaan-jaff in #9778
- Litellm 04 05 2025 release notes by @krrishdholakia in #9785
- feat: add offline swagger docs by @devdev999 in #7653
- fix(gemini/transformation.py): handle file_data being passed in by @krrishdholakia in #9786
- Realtime API Cost tracking by @krrishdholakia in #9795
- fix(vertex_ai.py): move to only passing in accepted keys by vertex ai response schema by @krrishdholakia in #8992
- fix(databricks/chat/transformation.py): remove reasoning_effort from … by @krrishdholakia in #9811
- Handle pydantic base model in message tool calls + Handle tools = [] + handle fireworks ai w/ 'strict' param in function call + support fake streaming on tool calls for meta.llama3-3-70b-instruct-v1:0 by @krrishdholakia in #9774
- Allow passing
thinking
param to litellm proxy via client sdk + Code QA Refactor on get_optional_params (get correct values) by @krrishdholakia in #9386 - [Feat] LiteLLM Tag/Policy Management by @ishaan-jaff in #9813
- Remove redundant
apk update
in Dockerfiles (cc #5016) by @PeterDaveHello in #9055 - [Security fix - CVE-2025-0330] - Leakage of Langfuse API keys in team exception handling by @ishaan-jaff in #9830
- [Security Fix CVE-2024-6825] Fix remote code execution in post call rules by @ishaan-jaff in #9826
- Bump next from 14.2.25 to 14.2.26 in /ui/litellm-dashboard by @dependabot in #9716
- fix: claude haiku cache read pricing per token by @hewliyang in #9834
- Add service annotations to litellm-helm chart by @mlhynfield in #9840
- Reflect key and team update in UI by @crisshaker in #9825
- Add user alias to API endpoint by @Jacobh2 in #9859
- Update Azure Phi-4 pricing by @emerzon in #9862
- feat: add enterpriseWebSearch tool for vertex-ai by @qvalentin in #9856
- VertexAI non-jsonl file storage support by @krrishdholakia in #9781
- [Bug Fix] Add support for UploadFile on LLM Pass through endpoints (OpenAI, Azure etc) by @ishaan-jaff in #9853
- [Feat SSO] Debug route - allow admins to debug SSO JWT fields by @ishaan-jaff in #9835
- [Feat] - SSO - Use MSFT Graph API to assign users to teams by @ishaan-jaff in #9865
- Cost tracking for
gemini-2.5-pro
by @krrishdholakia in #9837 - [SSO] Connect LiteLLM to Azure Entra ID Enterprise Application by @ishaan-jaff in #9872
New Contributors
- @aoaim made their first contribution in #9768
- @hewliyang made their first contribution in #9834
- @mlhynfield made their first contribution in #9840
- @crisshaker made their first contribution in #9825
- @qvalentin made their first contribution in #9856
Full Changelog: v1.65.4-nightly...v1.65.5-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.65.5-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed ✅ | 240.0 | 281.015845046053 | 6.098441913282575 | 0.0 | 1824 | 0 | 213.96507000002885 | 5930.206827000006 |
Aggregated | Passed ✅ | 240.0 | 281.015845046053 | 6.098441913282575 | 0.0 | 1824 | 0 | 213.96507000002885 | 5930.206827000006 |