Skip to content

v1.37.16

Choose a tag to compare

View all tags
@github-actions github-actions released this 18 May 07:31
v1.37.16
8d25a7b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's Changed

  • fix - allow non master key to access llm_utils_routes by @ishaan-jaff in #3710
  • fix(bedrock_httpx.py): move anthropic bedrock calls to httpx by @krrishdholakia in #3708
  • [Feat] Admin UI - use base_model for Slack Alerts by @ishaan-jaff in #3713
  • [Admin UI] show max input tokens on UI by @ishaan-jaff in #3714
  • fix(proxy_server.py): fix setting model id for db models by @krrishdholakia in #3715

Full Changelog: v1.37.14...v1.37.16

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.37.16

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name Status Median Response Time (ms) Average Response Time (ms) Requests/s Failures/s Request Count Failure Count Min Response Time (ms) Max Response Time (ms)
/chat/completions Failed ❌ 9 10.2880556709407 1.5629325106711098 1.5629325106711098 468 468 7.436624999968444 83.99098699999286
/health/liveliness Failed ❌ 8 10.80103857402248 15.632664706092875 15.632664706092875 4681 4681 6.298579000031168 1272.475381999982
/health/readiness Failed ❌ 8 10.780497224867714 15.712815091255495 15.712815091255495 4705 4705 6.286180000017794 650.4576310000232
Aggregated Failed ❌ 8 10.766867369799249 32.90841230801948 32.90841230801948 9854 9854 6.286180000017794 1272.475381999982

Contributors

ishaan-jaff
Assets 4
Loading