[Feature]: Custom Routing in Proxy (Parity with SDK)

[enniomaldonado-ai]

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

The Feature

We need a supported way to supply our own routing strategy, the same kind of flexibility we get when using the SDK custom routing, without being limited to built-in algorithms.

Motivation, pitch

We run LLM traffic through litemm and rely on its simple shuffle routing for the common case. For some workloads we need routing policy that is explicit, composite, and under our control, the same flexibility we already have when calling providers via an SDK, where we can implement custom selection, fallbacks, backoff, and experiments in code. Today, if that logic does not map cleanly onto litemm’s built-in strategies, we either duplicate routing outside the gateway (split clients, inconsistent behavior) or give up on policies we consider important for reliability and cost.

What part of LiteLLM is this about?

Proxy

LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

Twitter / LinkedIn details

No response

[enniomaldonado-ai]

Is there a roadmap of upcoming featues maybe there is something planned and we can help push it forward.