[Feature] OpenRouter Provider Routing Configuration #887
Description
VSCode Version: 1.99.30044
Void Version: 1.4.9
Commit: b5a4184
Date: 2025-06-23T08:09:51.384Z
Electron: 34.3.2
ElectronBuildId: undefined
Chromium: 132.0.6834.210
Node.js: 20.18.3
V8: 13.2.152.41-electron.0
OS: Darwin arm64 24.6.0
Void currently uses OpenRouter's default auto-routing with cost prio and load balancing. This can result in requests being sent to slow or suboptimal providers (FP4 quantization, small context windows, soso response times).
OpenRouter exposes provider routing controls in their API, but Void doesn't fully utilize them.
Proposal
Support for configuring OpenRouter provider preferences per model.
A simple approach would be extending the existing advanced settings JSON, something like:
{
"contextWindow": 163840,
"reservedOutputTokenSpace": 4096,
"providerPreferences": {
"order": ["deepinfra/turbo", "together", "fireworks"],
"sort": "throughput",
"allowFallbacks": true,
"quantizations": ["fp16", "bf16"]
}
}Then pass this to OpenRouter's API in the provider field.
Features
Provider Ordering: Specify preferred provider order. Use exact provider slugs from OpenRouter's model pages.
Sort Strategy: Override default price sorting with:
"throughput"- prioritize speed"latency"- prioritize low latency"price"- prioritize cost
Quantization: Filter providers on quantization (e.g., exclude FP4, only allow FP16/BF16).
Fallback: whether to fall back to auto-routing when preferred providers fail, or error out.
Price: Set acceptable max pricing per million tokens.
Thoughts
- Backwards compatible - no config means current behavior
- When preferred provider fails, seleectable; either log and fallback to auto-routing or error out (same for invalid names)
A GUI settings panel could potentially be added later, but the JSON approach feels like a quick and acceptable way to do it.