Skip to content

fix: multiply monthly token budget by key count, with proxy toggle#240

Closed
Slyker wants to merge 3 commits into
tashfeenahmed:mainfrom
Slyker:feat/token-budget-per-key
Closed

fix: multiply monthly token budget by key count, with proxy toggle#240
Slyker wants to merge 3 commits into
tashfeenahmed:mainfrom
Slyker:feat/token-budget-per-key

Conversation

@Slyker

@Slyker Slyker commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

What

Adds a proxy-aware monthly token budget calculation with a UI toggle.

Backend

/api/fallback/token-usage now accepts ?proxy=true or ?proxy=false (default):

  • ?proxy=false (default): raw per-key budget — platforms with multiple keys still show the single-key quota. Correct for single-IP deployments where providers pool rate limits.
  • ?proxy=true: multiplies each model's budget by the number of enabled API keys for its platform. Correct for multi-IP proxy setups where each key gets its own independent rate-limit pool.

Both modes filter out platforms with zero enabled keys. The response includes a proxy boolean.

Frontend

A toggle switch is added to the "Monthly token budget" bar. It passes ?proxy=true/false to the API and react-query refetches on toggle. The label reads "Per key · multi-IP" when active, "Per provider" when off.

Why

Each API key carries its own independent free-tier quota — but only when deployed behind separate IPs (via a proxy). Without a proxy, providers may pool rate limits by IP, making multiple keys on the same IP count as one. This PR exposes both views so users can choose based on their deployment.

Changes

  • server/src/routes/fallback.ts: /token-usage accepts ?proxy query param, multiplies by key count when true
  • client/src/pages/FallbackPage.tsx: proxy mode toggle in the TokenUsageBar with react-query refetch

Slyker added 3 commits June 6, 2026 06:34
Each API key carries its own independent free-tier quota. The
/token-usage endpoint now multiplies parseBudget() by the number of
enabled keys for that platform so the stacked bar reflects total
available capacity.

Optimal with a proxy (some providers rate-limit by key + IP).

IDEMPOTENCY: 378/378, ts-0
?proxy=true  → budget × key count (multi-IP deployment)
?proxy=false → raw per-key budget (single IP, default)

Always filters platforms with zero keys, regardless of mode.
Response includes 'proxy' boolean so the client can reflect state.

IDEMPOTENCY: 378/378, ts-0
Adds a Switch toggle in the monthly token budget section so users can
choose between:
- Per provider (default): raw per-key quota, correct for single-IP deployments
- Per key multi-IP: budget x key count, for multi-IP proxy setups

The toggle passes ?proxy=true/false to /api/fallback/token-usage.
React-query key includes proxyMode so data refetches on toggle.

IDEMPOTENCY: 378/378, ts-0
@Slyker Slyker changed the title fix: multiply monthly token budget by key count per platform fix: multiply monthly token budget by key count, with proxy toggle Jun 6, 2026
@tashfeenahmed

Copy link
Copy Markdown
Owner

Thanks for the work here, the implementation itself is clean. But I'm declining this one on policy grounds rather than code quality.

Multiplying the displayed budget by key count assumes each key carries its own independent free-tier quota. In practice that means multiple accounts per provider, which is exactly what most provider ToS forbid and what the README's "one account per provider" rule of thumb exists to prevent. A first-class toggle that presents multi-account stacking as a supported deployment mode (multi-IP via proxy, per the description) is a line I don't want the project to cross. The proxy feature in #231 is for reachability, not quota multiplication, and I want to keep that distinction sharp.

For the legitimate multi-key case (several keys issued against one shared account quota), the current single-key number is the correct display and multiplying would overstate it.

So, closing. Sorry for the redone work, and to be clear this doesn't affect #241 or the #231 rework, both of which I want.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants