Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,8 @@ server/data/
# Personal deployment scripts (contain keys/credentials — kept local)
deploy-pi.sh
update-hermes.sh
repo-assets/
docs/logos
docs/index.html
docs/og.png
docs/og.svg
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,8 @@ npm install
cp .env.example .env
ENCRYPTION_KEY="$(node -e 'console.log(require("crypto").randomBytes(32).toString("hex"))')"
printf "ENCRYPTION_KEY=%s\nPORT=3001\n" "$ENCRYPTION_KEY" > .env

# Start server + dashboard together
npm run dev
```

Expand Down
40 changes: 20 additions & 20 deletions client/src/components/markdown.tsx
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
import { memo } from 'react'
import { memo, type ComponentPropsWithoutRef } from 'react'
import ReactMarkdown, { type Components } from 'react-markdown'
import remarkGfm from 'remark-gfm'
import { cn } from '@/lib/utils'

const components: Components = {
p: ({ children }) => (
p: ({ children }: ComponentPropsWithoutRef<'p'>) => (
<p className="my-2 first:mt-0 last:mb-0 whitespace-pre-wrap wrap-break-word">
{children}
</p>
),
a: ({ children, href }) => (
a: ({ children, href }: ComponentPropsWithoutRef<'a'>) => (
<a
href={href}
target="_blank"
Expand All @@ -19,60 +19,60 @@ const components: Components = {
{children}
</a>
),
h1: ({ children }) => (
h1: ({ children }: ComponentPropsWithoutRef<'h1'>) => (
<h1 className="mt-4 mb-2 text-base font-semibold tracking-tight first:mt-0">{children}</h1>
),
h2: ({ children }) => (
h2: ({ children }: ComponentPropsWithoutRef<'h2'>) => (
<h2 className="mt-4 mb-2 text-sm font-semibold tracking-tight first:mt-0">{children}</h2>
),
h3: ({ children }) => (
h3: ({ children }: ComponentPropsWithoutRef<'h3'>) => (
<h3 className="mt-3 mb-1.5 text-sm font-semibold first:mt-0">{children}</h3>
),
h4: ({ children }) => (
h4: ({ children }: ComponentPropsWithoutRef<'h4'>) => (
<h4 className="mt-3 mb-1.5 text-sm font-semibold first:mt-0">{children}</h4>
),
ul: ({ children }) => (
ul: ({ children }: ComponentPropsWithoutRef<'ul'>) => (
<ul className="my-2 ml-5 list-disc space-y-1 [&_ul]:my-1 [&_ol]:my-1 first:mt-0 last:mb-0">
{children}
</ul>
),
ol: ({ children }) => (
ol: ({ children }: ComponentPropsWithoutRef<'ol'>) => (
<ol className="my-2 ml-5 list-decimal space-y-1 [&_ul]:my-1 [&_ol]:my-1 first:mt-0 last:mb-0">
{children}
</ol>
),
li: ({ children }) => <li className="leading-relaxed">{children}</li>,
blockquote: ({ children }) => (
li: ({ children }: ComponentPropsWithoutRef<'li'>) => <li className="leading-relaxed">{children}</li>,
blockquote: ({ children }: ComponentPropsWithoutRef<'blockquote'>) => (
<blockquote className="my-2 border-l-2 border-foreground/20 pl-3 italic text-foreground/80 first:mt-0 last:mb-0">
{children}
</blockquote>
),
hr: () => <hr className="my-3 border-foreground/15" />,
strong: ({ children }) => <strong className="font-semibold">{children}</strong>,
em: ({ children }) => <em className="italic">{children}</em>,
del: ({ children }) => <del className="opacity-70">{children}</del>,
strong: ({ children }: ComponentPropsWithoutRef<'strong'>) => <strong className="font-semibold">{children}</strong>,
em: ({ children }: ComponentPropsWithoutRef<'em'>) => <em className="italic">{children}</em>,
del: ({ children }: ComponentPropsWithoutRef<'del'>) => <del className="opacity-70">{children}</del>,

table: ({ children }) => (
table: ({ children }: ComponentPropsWithoutRef<'table'>) => (
<div className="my-2 overflow-x-auto first:mt-0 last:mb-0">
<table className="w-full border-collapse text-xs">{children}</table>
</div>
),
thead: ({ children }) => <thead className="bg-background/40">{children}</thead>,
th: ({ children, style }) => (
thead: ({ children }: ComponentPropsWithoutRef<'thead'>) => <thead className="bg-background/40">{children}</thead>,
th: ({ children, style }: ComponentPropsWithoutRef<'th'>) => (
<th
style={style}
className="border border-foreground/15 px-2 py-1 text-left font-semibold"
>
{children}
</th>
),
td: ({ children, style }) => (
td: ({ children, style }: ComponentPropsWithoutRef<'td'>) => (
<td style={style} className="border border-foreground/15 px-2 py-1 align-top">
{children}
</td>
),

code: ({ className, children, ...props }) => {
code: ({ className, children, ...props }: ComponentPropsWithoutRef<'code'>) => {
const isBlock = /language-/.test(className ?? '')
if (isBlock) {
return (
Expand All @@ -90,7 +90,7 @@ const components: Components = {
</code>
)
},
pre: ({ children }) => (
pre: ({ children }: ComponentPropsWithoutRef<'pre'>) => (
<pre className="my-2 overflow-x-auto rounded-md border bg-background/60 p-3 first:mt-0 last:mb-0">
{children}
</pre>
Expand Down
146 changes: 146 additions & 0 deletions docs/api-contracts-server.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# FreeLLMAPI — API Contracts

**Generated:** 2026-05-31 | **Part:** server

## OpenAI-Compatible Proxy

All proxy endpoints are mounted at `/v1` and accept standard OpenAI request/response formats.

### POST `/v1/chat/completions`

The primary proxy endpoint. Routes requests through the intelligent fallback chain.

**Request Body** (`ChatCompletionRequest`):
```json
{
"model": "auto",
"messages": [{"role": "user", "content": "Hello"}],
"temperature": 0.7,
"max_tokens": 1000,
"stream": false,
"tools": [{"type": "function", "function": {"name": "...", "parameters": {}}}],
"tool_choice": "auto"
}
```

**Model Resolution:**
- `"auto"` or omitted → sticky routing (same model for 30-min window)
- Specific model ID → exact catalog match with `400 model_not_found` on invalid/disabled
- Partial match supported: `"claude-3"` matches `"anthropic/claude-3"`

**Response** (non-streaming):
```json
{
"id": "chatcmpl-...",
"object": "chat.completion",
"model": "gemini-2.0-flash",
"choices": [{"index": 0, "message": {...}, "finish_reason": "stop"}],
"usage": {"prompt_tokens": 10, "completion_tokens": 20, "total_tokens": 30},
"_routed_via": {"platform": "google", "model": "gemini-3.1-flash"}
}
```

**Streaming:** When `stream: true`, returns SSE chunks (`chat.completion.chunk`).

**Retry Logic:** Up to 3 attempts across fallback chain. On 429/5xx from upstream, the model is penalized and the next model in the chain is tried.

### GET `/v1/models`

Returns the model catalog in OpenAI-compatible format.

---

## Dashboard API

All dashboard endpoints are mounted at `/api`.

### Keys Management — `/api/keys`

| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/keys` | List all keys (masked) |
| POST | `/api/keys` | Add a new API key |
| DELETE | `/api/keys/:id` | Delete a key |
| PATCH | `/api/keys/:id` | Toggle key enabled/disabled |
| PATCH | `/api/keys/platform/:platform` | Toggle all keys for a platform |

**POST body** (Zod-validated):
```json
{
"platform": "google",
"key": "AIza...",
"label": "My Google Key"
}
```

Valid platforms: `google`, `groq`, `cerebras`, `sambanova`, `nvidia`, `mistral`, `openrouter`, `github`, `cohere`, `cloudflare`, `zhipu`, `ollama`, `kilo`, `pollinations`, `llm7`, `huggingface`, `memos`

### Models — `/api/models`

| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/models` | List all models with status |
| PATCH | `/api/models/:id` | Toggle model enabled/disabled |

### Fallback Configuration — `/api/fallback`

| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/fallback` | Get fallback chain (ordered by priority) |
| PUT | `/api/fallback` | Reorder fallback chain |
| PATCH | `/api/fallback/:modelDbId` | Toggle model in fallback chain |

### Analytics — `/api/analytics`

| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/analytics/summary?range=7d` | Summary stats (requests, success rate, tokens, cost savings) |
| GET | `/api/analytics/by-model?range=7d` | Stats grouped by model |
| GET | `/api/analytics/by-platform?range=7d` | Stats grouped by platform |
| GET | `/api/analytics/timeline?range=7d&interval=day` | Timeline data points |
| GET | `/api/analytics/error-distribution?range=7d` | Error distribution by category/platform |
| GET | `/api/analytics/errors?range=7d` | Recent error logs (limit 50) |

Query params: `range` = `24h` | `7d` | `30d`; `interval` = `hour` | `day`

### Health — `/api/health`

| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/health` | System health with rate-limit status per model |
| POST | `/api/health/check` | Trigger manual health check of all keys |
| POST | `/api/health/check/:keyId` | Check a specific key |

### Settings — `/api/settings`

| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/settings/unified-key` | Get the unified API key |
| POST | `/api/settings/regenerate-key` | Regenerate the unified API key |

### Ping

| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/ping` | Simple health check (`{ status: "ok" }`) |

---

## Error Format

All errors follow the OpenAI standard envelope:

```json
{
"error": {
"message": "Detailed context message",
"type": "invalid_request_error",
"code": "model_not_found"
}
}
```

## Authentication

- **Proxy endpoints** (`/v1/*`): Authenticated via `Authorization: Bearer <unified-key>` header. The unified key is auto-generated on first boot and stored in SQLite.
- **Dashboard endpoints** (`/api/*`): No authentication (localhost-only deployment model). CORS restricts browser access to configured `DASHBOARD_ORIGINS`.
Loading