Skip to content

feat: custom OpenAI-compatible embedding providers#229

Open
long2ice wants to merge 2 commits into
tashfeenahmed:mainfrom
long2ice:feat/custom-embedding-api
Open

feat: custom OpenAI-compatible embedding providers#229
long2ice wants to merge 2 commits into
tashfeenahmed:mainfrom
long2ice:feat/custom-embedding-api

Conversation

@long2ice

@long2ice long2ice commented Jun 5, 2026

Copy link
Copy Markdown

What

Extends the custom-provider mechanism (#117/#212) to embeddings, so any OpenAI-compatible /embeddings endpoint (self-hosted vLLM / Ollama / LM Studio, or a remote gateway) can be added and routed — and unifies how custom chat + embedding models are managed.

Why

Custom providers previously only worked for chat models (models table + chat fallback chain). Embeddings had no custom support at all — callProvider's switch(platform) only covered the 7 built-in platforms. This adds parity.

Changes

Server

  • embedding_models gains a key_id column binding a custom row to its endpoint's api_keys row (migrateEmbeddingsV2Custom); built-ins stay NULL and resolve by platform as before.
  • Embeddings service resolves a custom row's key + base_url and calls the endpoint via the existing OpenAI-style adapter; adds probeEmbeddingDimensions.
  • POST /api/embeddings/custom — probes the endpoint to auto-detect the vector dimension, then upserts. Joining an existing family requires a matching dimension (a family is one vector space) and lands at the back of that family's chain.
  • DELETE /api/embeddings/custom/:id and DELETE /api/models/custom/:id — remove a single custom model while keeping the shared endpoint key.
  • Adding a model to an existing endpoint updates key/label independently and only when supplied, so adding a second model with a blank key no longer clobbers the key the endpoint's other models depend on.

Client

  • Keys page: the two separate custom forms are merged into one "Add a custom model" form with a Chat/Embedding toggle. Several chat + embedding models can share one endpoint and key. Embedding mode auto-detects the dimension and suggests a family by model id (overridable; the dimension check still guards).
  • Embeddings page and Models (fallback) page: per-row delete for custom providers (built-ins remain non-deletable).

Routing semantics

Embeddings still only fail over within a family (same model, another provider) — vectors from different models are incompatible. A custom endpoint either becomes its own family (default) or joins an existing one for cross-provider redundancy when dimensions match.

Tests

345 passing. New coverage: custom embedding routing + within-family failover, dimension probe + family-join dimension guard, blank-key reuse of an existing endpoint key, and single custom-model deletion (keeps the shared key; 404 on built-ins).

long2ice added 2 commits June 5, 2026 23:40
Extend the custom-provider mechanism (tashfeenahmed#117/tashfeenahmed#212) to embeddings so any
OpenAI-compatible /embeddings endpoint (self-hosted vLLM/Ollama/LM Studio
or a remote gateway) can be added and routed.

Server:
- embedding_models gains a key_id column binding a custom row to its
  endpoint's api_keys row (migrateEmbeddingsV2Custom); built-ins stay NULL
- embeddings service resolves a custom row's key + base_url and calls the
  endpoint via the OpenAI-style adapter; adds probeEmbeddingDimensions
- POST /api/embeddings/custom: probes the endpoint to auto-detect the
  vector dimension, then upserts; joining an existing family requires a
  matching dimension and lands at the back of the chain
- DELETE /api/embeddings/custom/:id and DELETE /api/models/custom/:id
  remove a single custom model while keeping the shared endpoint key
- adding a model to an existing endpoint updates key/label independently
  and only when supplied, so a blank key no longer clobbers the shared one

Client:
- Keys page: one merged 'Add a custom model' form with a Chat/Embedding
  toggle; embedding mode auto-detects dimension and suggests a family by
  model id (overridable)
- Embeddings and Models pages: per-row delete for custom providers

Tests: custom embedding routing/failover, dimension probe + family join,
blank-key reuse, and single custom-model deletion.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant