Feature Request: Custom API host for all providers

[longseespace]

Overview

Basically the title. Ideally I should be able to configure the Base URL and other parameters of the AI service.

For example, for OpenAI I really need to be able to set Base URL and Organization ID.

[lzell]

@longseespace Do you mean for the AIProxy.openAIDirectService call? Org ID I can def add in as an optional argument. LMK what your goal is with the base URL and I can make that an optional param too.

If you are thinking of using the openai service for other "compatible" services I can def make that happen, but I should warn you that some services don't really implement the contract well. There are holes between providers, and some providers have arguments that you'll miss out on by going for the common denominator chat completion contract.

Now, that said, if you are just using main trunk chat completion features without fiddling a bunch of other arguments I can see how swapping the baseURL is easier than using a different type with AIProxy.<Name>DirectService

To give just one example, one that I've run into personally: The Groq API technically has a timestamp_granularities param for compatibility with OpenAI. So if you were to build a whisper client and then switched the base URL for groq, you may expect that your users can happily switch between providers with the same feature set (after all the contracts are the same). However, to their surprise and maybe to the developer's too, the timestamp_granularities argument is a stub that doesn't actually control anything on Groq's side!

[mmm117]

@lzell I have the same request, I am using a service similar to Openrouter, which unify a lot of API interfaces. Their API is compatible to OpenAI API. Obviously I need to protect the API key issued by this service, so I want to use your AI Proxy service.
Hope you can support it. Thank you.

[yujinqiu]

@lzell Same request, it's quiet useful, if we want to use AI service, but want to through a proxy instead of direct connect.

[lzell]

Copy that, I will think more on a common interface. Swapping a baseURL would be a straightforward change for the conformers of DirectService, e.g. the instance you get from AIProxy.openAIDirectService. It is more complicated for conformers of ProxiedService, which you get back when you call AIProxy.openAIService.

A few more questions for any of you @yujinqiu @mmm117 @longseespace:

1. Are you all looking to swap the baseURL only for the direct-to-service case (for example if your customers bring their own API key)?
2. Do you need tool calls or structured outputs?
3. Is it a viable option to use us with OpenRouter? We do have support for them: https://github.com/lzell/AIProxySwift?tab=readme-ov-file#openrouter (see the OpenRouter header if GH doesn't link you to the right spot).

The reason I ask about 1, in case it's not clear, is because we do have a way for you to reuse the OpenAI client for other URLs, but it's only for calls that route through AIProxy. You achieve this by creating a new service in the AIProxy dashboard and specifying the baseURL of your choosing for the 'base proxy URL'. So instead of api.openai.com you would enter my-great-service.com. Then, you use the AIProxy.openaiService client just like you would for OpenAI.

[lzell]

I posted some starter code to this related issue, which may be a helpful starting point: #107

[everettjf]

Overview

Basically the title. Ideally I should be able to configure the Base URL and other parameters of the AI service.

For example, for OpenAI I really need to be able to set Base URL and Organization ID.

Maybe you are looking for this https://github.com/lzell/AIProxySwift/pull/111/files

[mmm117]

Copy that, I will think more on a common interface. Swapping a baseURL would be a straightforward change for the conformers of DirectService, e.g. the instance you get from AIProxy.openAIDirectService. It is more complicated for conformers of ProxiedService, which you get back when you call AIProxy.openAIService.

A few more questions for any of you @yujinqiu @mmm117 @longseespace:

1. Are you all looking to swap the baseURL only for the direct-to-service case (for example if your customers bring their own API key)?
2. Do you need tool calls or structured outputs?
3. Is it a viable option to use us with OpenRouter? We do have support for them: https://github.com/lzell/AIProxySwift?tab=readme-ov-file#openrouter (see the OpenRouter header if GH doesn't link you to the right spot).

The reason I ask about 1, in case it's not clear, is because we do have a way for you to reuse the OpenAI client for other URLs, but it's only for calls that route through AIProxy. You achieve this by creating a new service in the AIProxy dashboard and specifying the baseURL of your choosing for the 'base proxy URL'. So instead of api.openai.com you would enter my-great-service.com. Then, you use the AIProxy.openaiService client just like you would for OpenAI.

Hi @lzell , I am sorry I forgot to reply you.
Yes, as you said, just specifying another baseURL in the AIProxy dashboard does work for me. So, maybe you can just add some explanation in the dashboard, that will be fine. Thank you!!