Skip to content

Commit 8f9f96d

Browse files
committed
feat: add openrouter provider
1 parent 74feaa5 commit 8f9f96d

6 files changed

Lines changed: 298 additions & 23 deletions

File tree

README.md

Lines changed: 33 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ Check out [examples](./examples/), ready to ship.
1515

1616
### Key Features
1717

18-
- **Unified API**: One `.chat()` works seamlessly across OpenAI, Anthropic, Google, Ollama, and custom providers
18+
- **Unified API**: One `.chat()` works seamlessly across OpenAI, Anthropic, Google, OpenRouter, Ollama, and custom providers
1919
- **Built-in Resilience**: Automatic retries, exponential backoff, and circuit breakers handle failures gracefully
2020
- **Token Bucket Algorithm**: Automatically enforces provider rate limits intelligently
2121
- **Automatic Token Counting**: Accurate token estimation for every request, no manual calculation needed
@@ -33,7 +33,7 @@ npm i resilient-llm
3333
import { ResilientLLM } from 'resilient-llm';
3434

3535
const llm = new ResilientLLM({
36-
aiService: 'openai', // or 'anthropic', 'google', 'ollama'
36+
aiService: 'openai', // or 'anthropic', 'google', 'openrouter', 'ollama'
3737
model: 'gpt-5-nano',
3838
maxTokens: 2048,
3939
temperature: 0.7,
@@ -151,7 +151,7 @@ For all supported shapes (including plain schema objects) and parsing/validation
151151
152152
## Supported LLM Providers
153153
154-
ResilientLLM comes with built-in support for all text chat completiion models provided by **OpenAI**, **Anthropic**, **Google/Gemini**, **Ollama** API, etc.
154+
ResilientLLM comes with built-in support for all text chat completion models provided by **OpenAI**, **Anthropic**, **Google/Gemini**, **OpenRouter**, and **Ollama** APIs.
155155
156156
**Adding custom providers:** You can add support for other LLM providers (e.g., Together AI, Groq, self-hosted vLLM, or any OpenAI/Anthropic-compatible API) using `ProviderRegistry.configure()`. See the [Custom Provider Guide](./docs/custom-providers.md) for detailed instructions and examples.
157157
@@ -164,11 +164,41 @@ The simplest way is using environment variables:
164164
export OPENAI_API_KEY=sk-your-key-here
165165
export ANTHROPIC_API_KEY=sk-ant-your-key-here
166166
export GOOGLE_API_KEY=your-key-here
167+
export OPENROUTER_API_KEY=your-key-here
167168
export OLLAMA_API_KEY=your-key-here
168169
```
169170
170171
For more ways to configure API key, see the [API Key Configuration guide](./docs/reference.md#api-key-configuration) in the reference documentation.
171172
173+
## Free LLM API Setup
174+
175+
You can try ResilientLLM without paid provider keys using [OpenRouter](https://openrouter.ai/) free models. OpenRouter routes requests to free models (with rate limits); no credit card is required to sign up.
176+
177+
1. Sign up at [openrouter.ai](https://openrouter.ai/) (email or GitHub).
178+
2. Open **Keys** in the dashboard ([openrouter.ai/keys](https://openrouter.ai/keys)).
179+
3. Click **Create Key**, give it a name, and copy the key (starts with `sk-or-`).
180+
181+
You can then use `openrouter/free` model which randomly selects a free model.
182+
183+
184+
```javascript
185+
import { ResilientLLM } from 'resilient-llm';
186+
187+
const llm = new ResilientLLM({
188+
aiService: 'openrouter',
189+
model: 'openrouter/free',
190+
apiKey: 'sk-or-***'
191+
});
192+
193+
const { content } = await llm.chat([
194+
{ role: 'user', content: 'What is the capital of France?' }
195+
]);
196+
197+
console.log(content);
198+
```
199+
200+
The output might be extremely poor with `openrouter/free` model. If so, try a specific [free model from this list](https://openrouter.ai/models?max_price=0&output_modalities=text) or use a one of the [popular paid models](https://openrouter.ai/models?output_modalities=text&order=most-popular).
201+
172202
## Examples and Playground
173203
174204
Complete working projects using Resilient LLM as core library to call LLM APIs with resilience.

docs/custom-providers.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ ResilientLLM uses a `ProviderRegistry` to manage LLM providers. By default, it i
88
- OpenAI
99
- Anthropic (Claude)
1010
- Google (Gemini)
11+
- OpenRouter
1112
- Ollama
1213

1314
You can extend this by adding your own providers, such as:
@@ -16,6 +17,19 @@ You can extend this by adding your own providers, such as:
1617
- Custom API endpoints
1718
- Other LLM providers with compatible interfaces
1819

20+
### Using built-in OpenRouter
21+
22+
OpenRouter is supported as a built-in provider through `aiService: 'openrouter'`:
23+
24+
```javascript
25+
import { ResilientLLM } from 'resilient-llm';
26+
27+
const llm = new ResilientLLM({
28+
aiService: 'openrouter',
29+
model: 'openrouter/free' // or any OpenRouter model ID (provider/model e.g. openai/gpt-5-nano)
30+
});
31+
```
32+
1933
## Quick Start
2034

2135
The simplest way to add a custom provider is using `ProviderRegistry.configure()`:

docs/reference.md

Lines changed: 27 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ Complete technical reference for the ResilientLLM library API.
1818

1919
## ResilientLLM
2020

21-
A unified interface for interacting with multiple LLM providers (OpenAI, Anthropic, Google/Gemini, Ollama) with built-in resilience features including rate limiting, retries, circuit breakers, and error handling.
21+
A unified interface for interacting with multiple LLM providers (OpenAI, Anthropic, Google/Gemini, OpenRouter, Ollama) with built-in resilience features including rate limiting, retries, circuit breakers, and error handling.
2222

2323
### ResilientLLM Constructor
2424

@@ -39,7 +39,7 @@ new ResilientLLM(options?: ResilientLLMOptions)
3939

4040
| Property | Type | Required | Default | Description |
4141
|----------|------|----------|---------|-------------|
42-
| `aiService` | `string` | No | `process.env.PREFERRED_AI_SERVICE` or `"anthropic"` | AI service provider: `"openai"`, `"anthropic"`, `"google"`, or `"ollama"` |
42+
| `aiService` | `string` | No | `process.env.PREFERRED_AI_SERVICE` or `"anthropic"` | AI service provider: `"openai"`, `"anthropic"`, `"google"`, `"openrouter"`, or `"ollama"` |
4343
| `model` | `string` | No | `process.env.PREFERRED_AI_MODEL` or `"claude-3-5-sonnet-20240620"` | Model identifier for the selected AI service |
4444
| `temperature` | `number` | No | `process.env.AI_TEMPERATURE` or `0` | Temperature parameter (0-2) controlling randomness in responses |
4545
| `maxTokens` | `number` | No | `process.env.MAX_TOKENS` or `2048` | Maximum number of tokens in the response |
@@ -304,9 +304,21 @@ parseError(statusCode: number | null, error: Error, operationMetadata?: Operatio
304304
| `error` | `Error` | Yes | Underlying error |
305305
| `operationMetadata` | `OperationMetadata \| null` | No | Merged onto the thrown error’s `metadata` |
306306
307-
**Returns:** `never` — always throws **`ResilientLLMError`**.
307+
**Status Code Mappings:**
308308
309-
If `error` is already a **`ResilientLLMError`**, it is rethrown (metadata may be merged). Otherwise **`statusCode`** selects a **`PROVIDER_*`** code (e.g. `401``PROVIDER_UNAUTHORIZED`); `null` or unknown statuses map to **`PROVIDER_ERROR`**. See [`lib/ResilientLLMError.ts`](../lib/ResilientLLMError.ts) for the full `ResilientLLMErrorCode` union.
309+
| Status Code | Error Message |
310+
|------------|---------------|
311+
| `400` | "Bad request" |
312+
| `401` | "Invalid API Key" |
313+
| `403` | "You are not authorized to access this resource" |
314+
| `404` | "Not found" |
315+
| `429` | "Rate limit exceeded" |
316+
| `500` | "Internal server error" |
317+
| `503` | "Service unavailable" |
318+
| `529` | "API temporarily overloaded" |
319+
| Other | "Unknown error" |
320+
321+
**Note:** This method is called internally by the `chat()` method when errors occur. You typically don't need to call it directly.
310322
311323
---
312324
@@ -784,6 +796,7 @@ Set at least one API key for your chosen service:
784796
| `OPENAI_API_KEY` | OpenAI | Yes (if using OpenAI) |
785797
| `ANTHROPIC_API_KEY` | Anthropic | Yes (if using Anthropic) |
786798
| `GOOGLE_API_KEY` or `GOOGLE_GENERATIVE_AI` or `GEMINI_API_KEY` | Google | Yes (if using Google) |
799+
| `OPENROUTER_API_KEY` | OpenRouter | Yes (if using OpenRouter) |
787800
| `OLLAMA_API_KEY` | Ollama | No (optional) |
788801
789802
**Note:** For custom providers, use the environment variable names specified in `ProviderRegistry.configure()` via `envVarNames`.
@@ -800,6 +813,8 @@ Set at least one API key for your chosen service:
800813
| `MAX_INPUT_TOKENS` | `100000` | Default max input tokens |
801814
| `AI_TOP_P` | `0.95` | Default top-p value |
802815
| `OLLAMA_API_URL` | `"http://localhost:11434/api/generate"` | Ollama API URL |
816+
| `OPENROUTER_HTTP_REFERER` | `undefined` | Optional attribution header (`HTTP-Referer`) for OpenRouter |
817+
| `OPENROUTER_APP_TITLE` | `undefined` | Optional attribution header (`X-Title`) for OpenRouter |
803818
| `STORE_AI_API_CALLS` | `undefined` | Set to `"true"` to store API calls (OpenAI) |
804819
805820
---
@@ -998,6 +1013,14 @@ See [Custom Provider Guide](./custom-providers.md) for details on configuring pr
9981013
- Uses standard `Authorization: Bearer <token>` header
9991014
- Can store API calls if `STORE_AI_API_CALLS=true`
10001015
1016+
### OpenRouter
1017+
1018+
- Uses OpenAI-compatible endpoint `https://openrouter.ai/api/v1/chat/completions`
1019+
- Uses `Authorization: Bearer <token>` header
1020+
- Works with provider-prefixed model IDs (for example `openai/gpt-5-nano` or `openai/o1`)
1021+
- Choosing `openrouter/free` model will select a free model, but the quality might degrade severly
1022+
- Optional attribution headers can be set via `OPENROUTER_HTTP_REFERER` and `OPENROUTER_APP_TITLE`
1023+
10011024
### Google
10021025

10031026
- Uses OpenAI-compatible endpoint

lib/ProviderRegistry.ts

Lines changed: 37 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ export interface ConfigureInput {
6666
/**
6767
* Unified model schema (normalized across providers).
6868
* @property id - Model identifier (e.g. 'gpt-4o', 'claude-3-5-sonnet-20241022')
69-
* @property provider - Provider name ('openai', 'anthropic', 'google', 'ollama')
69+
* @property provider - Provider name ('openai', 'anthropic', 'google', 'openrouter', 'ollama')
7070
* @property name - Display name (if available from API)
7171
* @property contextWindow - Maximum tokens limit (if available from API, e.g. Gemini)
7272
* @property raw - Full raw API response for this model
@@ -91,7 +91,7 @@ class ProviderRegistry {
9191
/** Runtime API keys (stored separately from config to avoid serialization). Map<providerName, apiKey> */
9292
static #apiKeys = new Map<string, string>();
9393

94-
/** Default provider configurations (openai, anthropic, google, ollama). */
94+
/** Default provider configurations (openai, anthropic, google, openrouter, ollama). */
9595
static DEFAULT_PROVIDERS: Record<string, ProviderConfig> = {
9696
openai: {
9797
name: 'openai',
@@ -201,6 +201,41 @@ class ProviderRegistry {
201201
},
202202
active: true
203203
},
204+
openrouter: {
205+
name: 'openrouter',
206+
displayName: 'OpenRouter',
207+
chatApiUrl: 'https://openrouter.ai/api/v1/chat/completions',
208+
modelsApiUrl: 'https://openrouter.ai/api/v1/models',
209+
docsUrl: 'https://openrouter.ai/docs/api/api-reference/chat/send-chat-completion-request',
210+
envVarNames: ['OPENROUTER_API_KEY'],
211+
defaultModel: 'openrouter/free',
212+
apiVersion: null,
213+
iconUrl: null,
214+
customHeaders: {
215+
...(process.env.OPENROUTER_HTTP_REFERER ? { 'HTTP-Referer': process.env.OPENROUTER_HTTP_REFERER } : {}),
216+
...(process.env.OPENROUTER_APP_TITLE ? { 'X-Title': process.env.OPENROUTER_APP_TITLE } : {})
217+
},
218+
authConfig: {
219+
type: 'header',
220+
headerName: 'Authorization',
221+
headerFormat: 'Bearer {key}'
222+
},
223+
parseConfig: {
224+
modelsPath: 'data',
225+
idField: 'id',
226+
nameField: 'id',
227+
displayNameField: 'name',
228+
contextWindowField: 'context_length',
229+
idPrefix: null
230+
},
231+
chatConfig: {
232+
messageFormat: 'openai',
233+
responseParsePath: 'choices[0].message.content',
234+
toolSchemaType: 'openai',
235+
structuredOutputRequestField: 'response_format',
236+
},
237+
active: true
238+
},
204239
ollama: {
205240
name: 'ollama',
206241
displayName: 'Ollama',

0 commit comments

Comments
 (0)