Skip to content

OpenRouterProvider missing list_embedding_models_async() - embedding models not discovered #3086

@lagiosv

Description

@lagiosv

Problem

When using OpenRouter as an LLM provider, Letta cannot discover embedding models. The OpenRouterProvider class inherits from OpenAIProvider but doesn't implement list_embedding_models_async().

Current behavior:

  • Only letta/letta-free embedding model appears in the UI/API
  • OpenRouter embedding models (e.g., openai/text-embedding-3-small) are not discoverable
  • Users cannot create agents with OpenRouter embeddings through normal flows

Root cause:
OpenRouter's /models endpoint only returns LLM models, not embedding models. Unlike OpenAI's API, there's no /models response that includes embedding model metadata.

Environment

  • Letta version: 0.15.1 (Docker)
  • Provider: OpenRouter with OPENROUTER_API_KEY set

Proposed Solution

Add list_embedding_models_async() method to OpenRouterProvider class in letta/schemas/providers/openrouter.py.

Since OpenRouter doesn't expose embedding models via their API, the solution is to return hardcoded EmbeddingConfig objects for the known supported models:

async def list_embedding_models_async(self) -> list[EmbeddingConfig]:
    """
    Return hardcoded OpenRouter embedding models.
    OpenRouter's /models endpoint doesn't list embedding models,
    but they support OpenAI's embedding models via the /embeddings endpoint.
    """
    return [
        EmbeddingConfig(
            embedding_model="openai/text-embedding-3-small",
            embedding_endpoint_type="openai",
            embedding_endpoint=self.base_url,
            embedding_dim=1536,
            embedding_chunk_size=DEFAULT_EMBEDDING_CHUNK_SIZE,
            handle=self.get_handle("openai/text-embedding-3-small", is_embedding=True),
            batch_size=DEFAULT_EMBEDDING_BATCH_SIZE,
        ),
        EmbeddingConfig(
            embedding_model="openai/text-embedding-3-large",
            embedding_endpoint_type="openai",
            embedding_endpoint=self.base_url,
            embedding_dim=3072,
            embedding_chunk_size=DEFAULT_EMBEDDING_CHUNK_SIZE,
            handle=self.get_handle("openai/text-embedding-3-large", is_embedding=True),
            batch_size=DEFAULT_EMBEDDING_BATCH_SIZE,
        ),
    ]

Testing Done

Tested with a patched openrouter.py mounted via Docker volume:

Test Result
Embedding models appear in API ✅ 3 models (letta-free + 2 OpenRouter)
Agent creation with openai/text-embedding-3-small ✅ Works
Archival memory insert (generates embeddings) ✅ 1536-dim vectors stored
Archival memory search (semantic retrieval) ✅ Returns ranked results

Additional Context

  • OpenRouter uses the same /embeddings endpoint format as OpenAI
  • The embedding_endpoint_type="openai" ensures compatibility
  • This is a minimal change that unblocks OpenRouter users
  • Happy to submit a PR if this approach is acceptable

Related Issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions