Which Ollama models are recommended for OpenRAG? #715
-
|
I'm using Ollama as my model provider. Which models work best with OpenRAG? |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments
-
|
OpenRAG isn't guaranteed to be compatible with all Ollama models. Some models might produce unexpected results (like JSON output instead of natural language) or aren't appropriate for RAG tasks. Recommended models: Language models:
Embedding models:
You can experiment with other models, but if you encounter issues that you can't resolve through RAG best practices (like context filters and prompt engineering), try switching to one of these recommended models. If you need support for a specific model, please submit a GitHub issue. |
Beta Was this translation helpful? Give feedback.
-
|
mistral-nemo:12b doesn't work. It can't retrieval_call and bring back information from the knowledge. Any other suggestions would be apprecated my test environment is a 11GB so can't run the 20b gpt. |
Beta Was this translation helpful? Give feedback.
-
|
Has anyone get Qwen3.5:9b working? gpt-oss:20b is the only local model that works for me so far. I'm running on a 16GB AMD GPU. |
Beta Was this translation helpful? Give feedback.
OpenRAG isn't guaranteed to be compatible with all Ollama models. Some models might produce unexpected results (like JSON output instead of natural language) or aren't appropriate for RAG tasks.
Recommended models:
Language models:
gpt-oss:20b(requires at least 16GB of RAM - consider using Ollama Cloud or a remote machine)mistral-nemo:12bEmbedding models:
nomic-embed-text:latestmxbai-embed-large:latestembeddinggemma:latestYou can experiment with other models, but if you encounter issues that you can't resolve through RAG best practices (like context filters and prompt engineering), try switching to one of these recommended models.
If you need support for a specific model, please…