-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Open
Description
Problem
When using OpenRouter as an LLM provider, Letta cannot discover embedding models. The OpenRouterProvider class inherits from OpenAIProvider but doesn't implement list_embedding_models_async().
Current behavior:
- Only
letta/letta-freeembedding model appears in the UI/API - OpenRouter embedding models (e.g.,
openai/text-embedding-3-small) are not discoverable - Users cannot create agents with OpenRouter embeddings through normal flows
Root cause:
OpenRouter's /models endpoint only returns LLM models, not embedding models. Unlike OpenAI's API, there's no /models response that includes embedding model metadata.
Environment
- Letta version: 0.15.1 (Docker)
- Provider: OpenRouter with
OPENROUTER_API_KEYset
Proposed Solution
Add list_embedding_models_async() method to OpenRouterProvider class in letta/schemas/providers/openrouter.py.
Since OpenRouter doesn't expose embedding models via their API, the solution is to return hardcoded EmbeddingConfig objects for the known supported models:
async def list_embedding_models_async(self) -> list[EmbeddingConfig]:
"""
Return hardcoded OpenRouter embedding models.
OpenRouter's /models endpoint doesn't list embedding models,
but they support OpenAI's embedding models via the /embeddings endpoint.
"""
return [
EmbeddingConfig(
embedding_model="openai/text-embedding-3-small",
embedding_endpoint_type="openai",
embedding_endpoint=self.base_url,
embedding_dim=1536,
embedding_chunk_size=DEFAULT_EMBEDDING_CHUNK_SIZE,
handle=self.get_handle("openai/text-embedding-3-small", is_embedding=True),
batch_size=DEFAULT_EMBEDDING_BATCH_SIZE,
),
EmbeddingConfig(
embedding_model="openai/text-embedding-3-large",
embedding_endpoint_type="openai",
embedding_endpoint=self.base_url,
embedding_dim=3072,
embedding_chunk_size=DEFAULT_EMBEDDING_CHUNK_SIZE,
handle=self.get_handle("openai/text-embedding-3-large", is_embedding=True),
batch_size=DEFAULT_EMBEDDING_BATCH_SIZE,
),
]Testing Done
Tested with a patched openrouter.py mounted via Docker volume:
| Test | Result |
|---|---|
| Embedding models appear in API | ✅ 3 models (letta-free + 2 OpenRouter) |
Agent creation with openai/text-embedding-3-small |
✅ Works |
| Archival memory insert (generates embeddings) | ✅ 1536-dim vectors stored |
| Archival memory search (semantic retrieval) | ✅ Returns ranked results |
Additional Context
- OpenRouter uses the same
/embeddingsendpoint format as OpenAI - The
embedding_endpoint_type="openai"ensures compatibility - This is a minimal change that unblocks OpenRouter users
- Happy to submit a PR if this approach is acceptable
Related Issues
- OpenRouter API support #1670 - OpenRouter API support (closed, general support added)
- Getting error when calling archival_memory_insert #1784 - archival_memory_insert error with OpenRouter (closed, was about API key config)
Metadata
Metadata
Assignees
Labels
No labels