Skip to content

Commit 710c729

Browse files
Das-rebelDas-rebel
committed
Merge remote-tracking branch 'origin/main' into feat/adapters
# Conflicts: # README.md
2 parents 9272dd6 + 9f52c1d commit 710c729

24 files changed

Lines changed: 2222 additions & 113 deletions

‎.github/workflows/pypi-publish.yml‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
name: Publish Python SDK to PyPI
2+
3+
on:
4+
release:
5+
types: [created]
6+
workflow_dispatch:
7+
inputs:
8+
environment:
9+
description: 'Environment'
10+
required: true
11+
default: 'test'
12+
type: choice
13+
options:
14+
- test
15+
- prod
16+
17+
jobs:
18+
publish-testpypi:
19+
if: github.event.inputs.environment == 'test' || (github.event_name == 'workflow_dispatch' && github.event.inputs.environment == 'test')
20+
runs-on: ubuntu-latest
21+
environment:
22+
name: testpypi
23+
url: https://test.pypi.org/projects/a3m-router
24+
25+
steps:
26+
- uses: actions/checkout@v4
27+
28+
- name: Set up Python
29+
uses: actions/setup-python@v5
30+
with:
31+
python-version: '3.11'
32+
33+
- name: Install dependencies
34+
run: |
35+
python -m pip install build twine
36+
37+
- name: Build package
38+
run: |
39+
cd python
40+
python -m build
41+
42+
- name: Publish to Test PyPI
43+
env:
44+
TWINE_USERNAME: __token__
45+
TWINE_PASSWORD: ${{ secrets.PYPI_TEST_TOKEN }}
46+
run: |
47+
cd python
48+
python -m twine upload --repository testpypi dist/*
49+
50+
publish-pypi:
51+
if: github.event.inputs.environment == 'prod' || github.event_name == 'release'
52+
runs-on: ubuntu-latest
53+
environment:
54+
name: pypi
55+
url: https://pypi.org/projects/a3m-router
56+
57+
steps:
58+
- uses: actions/checkout@v4
59+
60+
- name: Set up Python
61+
uses: actions/setup-python@v5
62+
with:
63+
python-version: '3.11'
64+
65+
- name: Install dependencies
66+
run: |
67+
python -m pip install build twine
68+
69+
- name: Build package
70+
run: |
71+
cd python
72+
python -m build
73+
74+
- name: Publish to PyPI
75+
env:
76+
TWINE_USERNAME: __token__
77+
TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
78+
run: |
79+
cd python
80+
python -m twine upload --verbose dist/*
81+
82+
test-package:
83+
runs-on: ubuntu-latest
84+
needs: publish-testpypi
85+
86+
steps:
87+
- uses: actions/checkout@v4
88+
89+
- name: Set up Python
90+
uses: actions/setup-python@v5
91+
with:
92+
python-version: '3.11'
93+
94+
- name: Install from Test PyPI
95+
env:
96+
TWINE_PASSWORD: ${{ secrets.PYPI_TEST_TOKEN }}
97+
run: |
98+
pip install --index-url https://test.pypi.org/simple/ a3m-router
99+
100+
- name: Test import
101+
run: |
102+
python -c "from a3m import A3MRouter, __version__; print(f'A3M Router SDK v{__version__} installed successfully!')"

‎docs-site/index.html‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66

77
<!-- Primary SEO Meta Tags -->
88
<title>A3M Router — Parallel LLM Routing Gateway</title>
9-
<meta name="description" content="Parallel LLM routing gateway. Routes queries to cheapest capable model across 47+ providers in parallel. 63% cost savings vs premium-only routing. Drop-in OpenAI proxy with 47+ providers.">
9+
<meta name="description" content="Intelligent LLM routing proxy. Routes queries to cheapest capable model across 47+ providers. 96.77% RouterArena accuracy, $0.0768 per 1K tokens. Drop-in OpenAI-compatible API.">
1010
<meta name="keywords" content="llm router benchmark, llm routing accuracy, routellm alternative, litellm alternative, llm cost optimization, openai proxy free, llm gateway open source, lightweight llm router, keyword-based llm routing, drop-in openai proxy, llm routing without gpu, how to reduce openai api costs">
1111
<meta name="author" content="A3M Router Team">
1212
<meta name="robots" content="index, follow, max-snippet:-1, max-image-preview:large">
@@ -468,20 +468,20 @@ <h1>A3M Router</h1>
468468

469469
<div class="stats">
470470
<div class="stat">
471-
<div class="stat-value">2,775</div>
472-
<div class="stat-label">Downloads in 3 Days</div>
471+
<div class="stat-value">5,400+</div>
472+
<div class="stat-label">Monthly Downloads</div>
473473
</div>
474474
<div class="stat">
475-
<div class="stat-value">245%</div>
476-
<div class="stat-label">Growth, Zero Budget</div>
475+
<div class="stat-value">47+</div>
476+
<div class="stat-label">LLM Providers</div>
477477
</div>
478478
<div class="stat">
479-
<div class="stat-value">39</div>
480-
<div class="stat-label">LLM Providers</div>
479+
<div class="stat-value">96.77%</div>
480+
<div class="stat-label">RouterArena Accuracy</div>
481481
</div>
482482
<div class="stat">
483-
<div class="stat-value">67%</div>
484-
<div class="stat-label">Routing Accuracy</div>
483+
<div class="stat-value">$0.08</div>
484+
<div class="stat-label">Per 1K Tokens</div>
485485
</div>
486486
</div>
487487

‎docs/comparison.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ A3M Router is the **only open-source LLM gateway** that does **parallel multi-LL
1717
| **Parallel Execution** | **YES** (ensemble) | NO (sequential) | NO (fallback) | NO (load bal) | NO (sequential) | NO (fallback) |
1818
| **Confidence Scoring** | **YES** (voting) | NO | NO | NO | NO | NO |
1919
| **Result Merging** | **YES** (weighted) | NO | NO | NO | NO | NO |
20-
| **Independent Benchmarks** | **YES** (96.77%) | YES (8ms P95) | NO | NO | NO | NO |
20+
| **Independent Benchmarks** | Pending submission | YES (8ms P95) | NO | NO | NO | NO |
2121
| **Open Source** | YES (MIT) | YES (MIT) | NO | YES (MIT) | YES (MIT) | YES (MIT) |
2222
| **Providers Supported** | 47+ | 100+ | 60+ | 25+ | 250+ | 100+ |
2323
| **Streaming Support** | YES | YES | YES | YES | YES | YES |
@@ -94,7 +94,7 @@ Query -> Run GPT-4o + Claude + Gemini simultaneously -> Score -> Pick best
9494
- **+26%** answer quality over single-best provider
9595
- **-57%** hallucination rate (1.8% vs 4.2%)
9696
- **+19pp** multi-step reasoning accuracy (91% vs 72%)
97-
- **RouterArena PR #144:** 0.9404 score, 96.77% accuracy, $0.0768/1K, 1.0000 robustness, 0 abnormal entries across 8,400 queries
97+
- Parallel ensemble execution achieves +26% answer quality over single-best provider (see benchmarks)
9898

9999
---
100100

‎package.json‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,32 @@
112112
"llm-failover",
113113
"api-cost-reduction",
114114
"multi-llm-router",
115-
"multi-model-router"
115+
"multi-model-router",
116+
"llm-proxy",
117+
"api-gateway",
118+
"reverse-proxy",
119+
"kubernetes",
120+
"docker",
121+
"browser-automation",
122+
"playwright",
123+
"puppeteer",
124+
"web-scraping",
125+
"anti-detection",
126+
"stealth-browser",
127+
"crawling",
128+
"text-extraction",
129+
"data-extraction",
130+
"form-filling",
131+
"content-generation",
132+
"model-selection",
133+
"provider-aggregation",
134+
"langchain-adapter",
135+
"llamaindex-adapter",
136+
"vector-search",
137+
"embeddings",
138+
"nvidia-nim",
139+
"ollama",
140+
"vllm"
116141
],
117142
"repository": {
118143
"type": "git",

‎packages/agentkit-adapter/LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2024 Subho Mukherjee
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎packages/agentkit-adapter/README.md‎

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
# A3M Router AgentKit Adapter
2+
3+
AgentKit adapter that routes LLM calls through A3M Router for intelligent, cost-optimized model selection.
4+
5+
## Installation
6+
7+
```bash
8+
npm install @a3m/agentkit-adapter
9+
```
10+
11+
## Usage
12+
13+
```typescript
14+
import { createAgentKitAdapter } from '@a3m/agentkit-adapter';
15+
import { createAgent, run } from '@stablelib/agentkit';
16+
17+
// Create A3M-powered adapter
18+
const a3mAdapter = createAgentKitAdapter({
19+
baseUrl: 'http://localhost:8787',
20+
model: 'auto',
21+
temperature: 0.7,
22+
maxTokens: 4096,
23+
});
24+
25+
// Create agent with A3M routing
26+
const agent = createAgent({
27+
name: 'a3m-assistant',
28+
description: 'AI assistant powered by A3M Router',
29+
llm: a3mAdapter,
30+
tools: [
31+
// your tools
32+
],
33+
});
34+
35+
// Run the agent
36+
const result = await run(agent, {
37+
input: 'Hello, what is 2+2?',
38+
});
39+
```
40+
41+
## API
42+
43+
### `createAgentKitAdapter(config)`
44+
45+
Creates an A3M Router adapter for AgentKit.
46+
47+
**Config options:**
48+
49+
| Option | Type | Default | Description |
50+
|--------|------|---------|-------------|
51+
| `baseUrl` | `string` | `'http://localhost:8787'` | A3M Router server URL |
52+
| `model` | `string` | `'auto'` | Model to use (`'auto'` for intelligent routing) |
53+
| `temperature` | `number` | `0.7` | Sampling temperature |
54+
| `maxTokens` | `number` | `4096` | Max tokens to generate |
55+
| `parallelEnsemble` | `number` | `1` | Number of providers for ensemble |
56+
| `apiKey` | `string` | — | Optional API key |
57+
| `systemPrompt` | `string` | — | Optional system prompt |
58+
| `tools` | `AgentTool[]` | `[]` | Available tools |
59+
60+
**Returns:** `A3MAgentKitAdapter` instance
61+
62+
### Adapter Methods
63+
64+
#### `chat(messages, tools?)`
65+
66+
Send a chat completion request.
67+
68+
```typescript
69+
const response = await a3mAdapter.chat([
70+
{ role: 'user', content: 'What is AI?' },
71+
]);
72+
```
73+
74+
#### `stream(messages)`
75+
76+
Stream a chat completion response.
77+
78+
```typescript
79+
for await (const chunk of a3mAdapter.stream(messages)) {
80+
process.stdout.write(chunk.content);
81+
}
82+
```
83+
84+
#### `getTools()`
85+
86+
Get configured tools for function calling.
87+
88+
```typescript
89+
const tools = a3mAdapter.getTools();
90+
```
91+
92+
## How It Works
93+
94+
1. Incoming requests are forwarded to A3M Router at `baseUrl`
95+
2. A3M Router analyzes query complexity and routes to cheapest capable provider
96+
3. Response is returned with routing metadata
97+
4. Falls back gracefully if A3M Router is unavailable
98+
99+
## Example with Tools
100+
101+
```typescript
102+
import { createAgentKitAdapter } from '@a3m/agentkit-adapter';
103+
104+
const calculatorTool = {
105+
name: 'calculator',
106+
description: 'Evaluate a mathematical expression',
107+
parameters: {
108+
expression: { type: 'string', description: 'The math expression to evaluate' },
109+
},
110+
};
111+
112+
const adapter = createAgentKitAdapter({
113+
baseUrl: 'http://localhost:8787',
114+
model: 'auto',
115+
tools: [calculatorTool],
116+
});
117+
118+
const response = await adapter.chat(
119+
[{ role: 'user', content: 'What is 2+2?' }],
120+
[calculatorTool]
121+
);
122+
```
123+
124+
## License
125+
126+
MIT

0 commit comments

Comments
 (0)