Best practice for remote APIs that require a proxy

[sahuguet]

In some corporate environment, there is no direct access to remote API.

One option is provide support for proxies (see #518), e.g. by having llm let specify an alternate endpoint. When adding a new model via a plugin, this would mean asking for the api key AND an optional endpoint.

Another option is to create a new plugin for each model (or family of models) that requires a proxy access, and the Python code of the plugin would point to the proxy instead of the default endpoint.

@simonw
Any recommendation about the best way to proceed?

[flexchar]

Could you use something like LiteLLM running alongside that you perform all types of complex config and use with LLM?

[AlexanderYastrebov]

See also related #382

[AlexanderYastrebov]

Could you use something like LiteLLM running alongside that you perform all types of complex config and use with LLM?

Yes, see #518 (comment)