Skip to content

Configurable embedding function (custom / OpenAI-compatible endpoint) #3029

Description

@Nelie-Taylor

Problem

  • claude-mem has no way to configure the embedding model. It's hardcoded to the local ONNX default (all-MiniLM-L6-v2, 384-dim), which is weak for non-English text (e.g. Vietnamese) and lower quality than modern models.

Current behavior
Two layers block customization:

  1. worker-service.cjs calls chroma_create_collection({collection_name}) with no embedding_function_name, hardcodes --with ["onnxruntime>=1.20","protobuf<7"] (no openai), and exposes no CLAUDE_MEM_EMBED env.
  2. chroma-mcp 0.2.6 instantiates the EF with no args and never passes an EF to get_collection, so everything defaults to ONNX. (chromadb 1.0.16 HttpClient also doesn't rehydrate a persisted custom EF on read, so pre-creating collections doesn't help.)

Proposed solution
Add env-driven embedding config, e.g.:

 CLAUDE_MEM_EMBED_PROVIDER=openai        # default | openai | voyageai | ...
 CLAUDE_MEM_EMBED_MODEL=text-embedding-3-large
 CLAUDE_MEM_EMBED_API_BASE=https://my-proxy/v1   # OpenAI-compatible
 CLAUDE_MEM_EMBED_API_KEY=...
 CLAUDE_MEM_EMBED_DIMS=3072

When set, the worker should: add the provider package to the uvx --with list, pass embedding_function_name to chroma_create_collection, and ensure the same EF (with base/model/dims) is applied on read paths.

Workaround (current)
Patching worker-service.cjs (--with openai + --dotenv-path) and chroma-mcp's server.py (force the OpenAI EF on all collection ops). Works but is lost on every plugin update and uv cache clean.

Benefit
Much better multilingual + higher-quality recall, opt-in, no breaking change to the ONNX default.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions