Support non-OpenAI APIs via request/response interception

[the-gigi]

simple-openai at its core supports very strictly the official OpenAI API, including its endpoints and its semantics (assumes a stable base URL + API key). It also supports multiple non-OpenAI providers that provide OpenAI-compatible API. Some of these providers have different endpoint URLs (besides the base URL) or otherwise behave differently. Some of them even have bugs like Gemini.

simple-openai provides request and response interceptors that allow a provider implementation to modify the request and/or the response and allow simple-openai to operate smoothly with these slight variation from the formal supported OpenAI API + behavior.

But, we can theoretically take it further and support completely non-OpenAI APIs too. As long as conceptually the input and output of a particular API have some OpenAI equivalent then it should be possible to take an OpenAI request object sent to simple-openai modify the request to comply with some non-OpenAI API request, send it to that API and then on the way back modify the non-OpenAI response to an OpenAI-compatible JSON that will be serialized by simple-openai to its response object.

Thoughts?

[sashirestela]

Here we have a problem, if we are going to support non-OpenAI APIs then we will have to rename the library to simple-non-openai. Well, ironies aside, I don't think it's a good idea to go that route. Simple-OpenAI was created to support the OpenAI API and we went further when we extended it to support OpenAI compatible APIs and that was fine. Moreover, your proposal would be going against the current trend, because most LLM vendors are working on exposing their APIs to be OpenAI compatible.

Maybe you are having trouble with the latest Gemini Vertex integration? Please let me know if that's the case and we could find a creative approach without warping the essence of simple-openai.

In the past, I thought about a general library to handle any API and created Jamaica, but with my limited time I couldn't work on it any further and decided to focus on simple-openai.

[the-gigi]

I understand you point of view. Let me just list a few points:

1. simple-openai doesn't expose directly the OpenAI API, which is a REST API. It exposes a Java API that is mapped to the OpenAI API, but the same Java API can be mapped to to other providers. It already is modified slightly to match other almost OpenAI-compatible providers.
2. In a perfect world, all the providers would just use the OpenAI API as their API (like DeepSeek) and then adding a new provider would be just setting a new base URL. But, in our current reality this is not the case. For example, Gemini supports many capabilities like images and audio and files in the native Gemini API, but their OpenAI compatibility layer is very limited. So, there is a lot value in building on top of their OpenAI compatibility layer and provide additional services through simple-openai.
3. simple-openai with its request and response interceptors provide a flexible platform for supporting any LLM provider that provides similar services (or a subset), but it doesn't mean that all these providers must be supported directly by simple-openai itself.
4. It is already possible to just derive a class from SimpleOpenAI and pass a custom requestInterceptor and responseInterceptor and support any API. The only reason to bring it back to simple-openai itself is to have a one place and one dependency to work with any LLM provider in Java.

Anyway, I just wanted to raise the idea. I have no immediate need. No problems with the Gemini Vertex integration.