Feature Request: Add OpenSearch as a Vector Store Provider

[flefevre]

Motivation

OpenSearch is widely used for document search and can support RAG use cases with:

• vector search
• BM25 keyword search
• hybrid search
• metadata filtering
• scalable indexing

This would help connect LiteLLM with existing OpenSearch-based RAG infrastructures and the broader OpenRAG ecosystem.

Proposed Feature

Add an OpenSearch vector store backend that can:

• store embeddings in an OpenSearch index
• run vector search
• optionally support hybrid search: BM25 + vector
• support metadata filters
• return results using the existing LiteLLM/OpenAI-compatible vector store API

Example Use Case

A team already has documents indexed in OpenSearch and wants to use LiteLLM as a single gateway for LLMs, embeddings, and RAG retrieval without adding another vector database.

Benefit

This would make LiteLLM easier to adopt in enterprise RAG setups where OpenSearch is already used through for instance the openrag

Thanks for considering this feature.

[reallyticsai]

Totally agree—OpenSearch as a vector store backend would make LiteLLM way more flexible for RAG projects. We’ve deployed OpenSearch in production for hybrid search setups and it scales nicely, offers real-time indexing, and handles both dense (vector) and sparse (BM25) retrieval.

Key implementation points:

• You can store embeddings directly by adding a knn field to your index mapping (docs).
• Hybrid search is possible by combining vector queries with BM25 using OpenSearch’s hybrid search API.
• For metadata filtering, just use standard OpenSearch filters in the query DSL alongside your vector search.
• If you want to keep things compatible with LiteLLM’s vector store API, you’ll just need to wrap OpenSearch’s search and indexing calls. Example to store embeddings:

from opensearchpy import OpenSearch

client = OpenSearch(...)

doc = {
    "text": "foo bar",
    "embedding": [0.1, 0.2, ...],  # float32 vector
    "metadata": {"author": "alice"}
}
client.index(index="docs", body=doc)

We’ve found OpenSearch much easier to integrate than dedicated vector DBs in enterprise environments—no extra infra, just extend existing clusters. Supporting hybrid retrieval is a huge win for real-world RAG, especially when users expect both semantic and keyword relevance. Happy to share more integration details if needed.