Skip to content

LLMs: OpenAI & Compatible LLMs

Jump to bottom
Mark Hamstra edited this page Mar 2, 2025 · 4 revisions

OpenAI is the main and original integration through the Chat Completions API.

Required configuration for OpenAI:

  • aikit.model: modmore\AIKit\LLM\Models\OpenAI
  • aikit.openai_endpoint: https://api.openai.com/v1/
  • aikit.openai_api_key: your API Key from the OpenAI Platform
  • aikit.openai_model: a valid model from here - e.g. gpt-4o-mini

"OpenAI Compatible" LLMs

Many LLMs claim OpenAI compatibility, but in practice not all of that appears to be exactly true. In many cases you can try using the modmore\AIKit\LLM\Models\OpenAI model with the different endpoint and test if it works.

Some LLMs require minor changes. If sending a message results in an error and the message in the error log mentions "developer" is invalid, try switching to modmore\AIKit\LLM\Models\OpenAILegacy instead. If it still doesn't work with that, the LLM will require its own adapter.

OpenRouter.ai

OpenRouter.ai offers a variety of models following the OpenAI standard. For example, we've had successful tests using the Claude 3.7 Sonnet model.

Because OpenRouter supports a wide range of models, it's useful to experiment with.

Note that Claude 3.5 does not support function calls, limiting its capabilities in AI Kit. However, when combined with Vector Storage, it will still be knowledgeable.

Required configuration for OpenRouter:

  • aikit.model: modmore\AIKit\LLM\Models\OpenAI
  • aikit.openai_endpoint: https://openrouter.ai/api/v1/
  • aikit.openai_api_key: your API Key from the OpenRouter dashboard.
  • aikit.openai_model: a valid model from here - e.g. anthropic/claude-3.7-sonnet

Scaleway

Successful tests using Mistral via Scaleway

Clone this wiki locally