Skip to content

feat: add OrcaRouter support for language models#3680

Open
xilema2 wants to merge 1 commit into
QuivrHQ:mainfrom
xilema2:feat/orcarouter-support
Open

feat: add OrcaRouter support for language models#3680
xilema2 wants to merge 1 commit into
QuivrHQ:mainfrom
xilema2:feat/orcarouter-support

Conversation

@xilema2

@xilema2 xilema2 commented May 26, 2026

Copy link
Copy Markdown

PR Description

# Description

This PR adds support for **OrcaRouter** (https://www.orcarouter.ai) as a new
LLM supplier in `quivr-core`. OrcaRouter is an OpenAI-compatible multi-vendor
routing gateway that exposes 150+ models from OpenAI / Anthropic / Google /
DeepSeek / xAI / MiniMax / Qwen / Kimi / etc. under a single API key, plus a
synthetic `orcarouter/auto` router that adaptively picks an upstream per
request.

Following the same pattern as the recently-merged Groq (#3633) and Gemini
(#3632) integrations, this PR:

1. Adds `ORCAROUTER = "orcarouter"` to `DefaultModelSuppliers`.
2. Registers eight flagship model entries in `LLMModelConfig._model_defaults`
   under the new supplier (auto router + one flagship per vendor). Any other
   OrcaRouter model ID is still accepted as a free-form string — the
   `LLMEndpointConfig` defaults just fall back to the generic 20k context /
   4k output budget for unrecognized models.
3. Wires up an `ORCAROUTER` branch in `LLMEndpoint.from_config` that uses
   `ChatOpenAI` against `https://api.orcarouter.ai/v1` (the SDK base URL —
   override via `LLMEndpointConfig.llm_base_url` if you point at a
   self-hosted instance). Reasoning models (`anthropic/claude-opus-4.7`,
   `openai/gpt-5*`, `deepseek-reasoner`) reject `temperature` upstream, so
   it's set to `None` for those.

### Usage

```python
from quivr_core.rag.entities.config import (
    LLMEndpointConfig,
    DefaultModelSuppliers,
)
from quivr_core.llm import LLMEndpoint

# API key from env: ORCAROUTER_API_KEY
config = LLMEndpointConfig(
    supplier=DefaultModelSuppliers.ORCAROUTER,
    model="orcarouter/auto",  # or "openai/gpt-5.5", "anthropic/claude-opus-4.7", ...
)
llm = LLMEndpoint.from_config(config)

Disclosure

I'm an engineer on the OrcaRouter team. Happy to iterate on naming, model
list, or default budgets — just let me know what fits Quivr's conventions
best.

Checklist before requesting a review

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code
  • I have commented hard-to-understand areas
  • I have ideally added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged

Summary by cubic

Add OrcaRouter as a new LLM supplier, enabling OpenAI-compatible routing to 150+ models through a single endpoint and a synthetic orcarouter/auto router. This adds default model configs, a default base URL, and small behavior tweaks for reasoning models.

  • New Features
    • Added ORCAROUTER to DefaultModelSuppliers.
    • Registered eight default model configs (including orcarouter/auto and vendor flagships); other OrcaRouter IDs still work with generic defaults.
    • Integrated via ChatOpenAI targeting https://api.orcarouter.ai/v1 by default (override with llm_base_url); sets temperature=None for reasoning models like anthropic/claude-opus-4.7, openai/gpt-5*, and deepseek-reasoner.
    • info() now reports the resolved base URL using _effective_base_url.
    • Supports ORCAROUTER_API_KEY and avoids mutating the caller’s config in from_config.

Written for commit dc0a6f5. Summary will update on new commits. Review in cubic

Add OrcaRouter (https://www.orcarouter.ai) as a new LLM supplier alongside
the existing OpenAI / Anthropic / Mistral / Gemini / Groq providers.

OrcaRouter is an OpenAI-compatible multi-vendor routing gateway: clients
talk to a single endpoint with one API key and OrcaRouter routes each
request to the appropriate upstream (or to a configurable router policy
via the synthetic 'orcarouter/auto' model).

- Add ORCAROUTER value to DefaultModelSuppliers.
- Register eight flagship model entries in LLMModelConfig._model_defaults
  (auto router + one flagship per vendor). Any other OrcaRouter model ID
  is still accepted as a free-form string.
- Wire up an ORCAROUTER branch in LLMEndpoint.from_config that uses
  ChatOpenAI against https://api.orcarouter.ai/v1 (overridable via
  llm_base_url). Reasoning models (claude-opus-4.7, gpt-5*,
  deepseek-reasoner) reject 'temperature' upstream, so it is passed as
  None for those.
- Have LLMEndpoint.info() report the resolved base URL via a small
  _effective_base_url helper, so diagnostics show the real endpoint
  instead of the generic 'openai' placeholder.
- Add unit tests covering: default base URL, reasoning-model temperature
  drop, custom base URL override, info() base URL reporting, env var
  resolution (ORCAROUTER_API_KEY), and non-mutation of the caller's
  config inside from_config.
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. area: backend Related to backend functionality or under the /backend directory labels May 26, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: backend Related to backend functionality or under the /backend directory size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant