fix: multiply monthly token budget by key count, with proxy toggle#240
fix: multiply monthly token budget by key count, with proxy toggle#240Slyker wants to merge 3 commits into
Conversation
Each API key carries its own independent free-tier quota. The /token-usage endpoint now multiplies parseBudget() by the number of enabled keys for that platform so the stacked bar reflects total available capacity. Optimal with a proxy (some providers rate-limit by key + IP). IDEMPOTENCY: 378/378, ts-0
?proxy=true → budget × key count (multi-IP deployment) ?proxy=false → raw per-key budget (single IP, default) Always filters platforms with zero keys, regardless of mode. Response includes 'proxy' boolean so the client can reflect state. IDEMPOTENCY: 378/378, ts-0
Adds a Switch toggle in the monthly token budget section so users can choose between: - Per provider (default): raw per-key quota, correct for single-IP deployments - Per key multi-IP: budget x key count, for multi-IP proxy setups The toggle passes ?proxy=true/false to /api/fallback/token-usage. React-query key includes proxyMode so data refetches on toggle. IDEMPOTENCY: 378/378, ts-0
|
Thanks for the work here, the implementation itself is clean. But I'm declining this one on policy grounds rather than code quality. Multiplying the displayed budget by key count assumes each key carries its own independent free-tier quota. In practice that means multiple accounts per provider, which is exactly what most provider ToS forbid and what the README's "one account per provider" rule of thumb exists to prevent. A first-class toggle that presents multi-account stacking as a supported deployment mode (multi-IP via proxy, per the description) is a line I don't want the project to cross. The proxy feature in #231 is for reachability, not quota multiplication, and I want to keep that distinction sharp. For the legitimate multi-key case (several keys issued against one shared account quota), the current single-key number is the correct display and multiplying would overstate it. So, closing. Sorry for the redone work, and to be clear this doesn't affect #241 or the #231 rework, both of which I want. |
What
Adds a proxy-aware monthly token budget calculation with a UI toggle.
Backend
/api/fallback/token-usagenow accepts?proxy=trueor?proxy=false(default):?proxy=false(default): raw per-key budget — platforms with multiple keys still show the single-key quota. Correct for single-IP deployments where providers pool rate limits.?proxy=true: multiplies each model's budget by the number of enabled API keys for its platform. Correct for multi-IP proxy setups where each key gets its own independent rate-limit pool.Both modes filter out platforms with zero enabled keys. The response includes a
proxyboolean.Frontend
A toggle switch is added to the "Monthly token budget" bar. It passes
?proxy=true/falseto the API and react-query refetches on toggle. The label reads "Per key · multi-IP" when active, "Per provider" when off.Why
Each API key carries its own independent free-tier quota — but only when deployed behind separate IPs (via a proxy). Without a proxy, providers may pool rate limits by IP, making multiple keys on the same IP count as one. This PR exposes both views so users can choose based on their deployment.
Changes
server/src/routes/fallback.ts:/token-usageaccepts?proxyquery param, multiplies by key count whentrueclient/src/pages/FallbackPage.tsx: proxy mode toggle in the TokenUsageBar with react-query refetch