Skip to content

fix(proxy): apply temp_budget_increase to multi-window budget_limits - #35265

Open
a-korath wants to merge 2 commits into
BerriAI:litellm_internal_stagingfrom
a-korath:fix/temp-budget-increase-multi-window
Open

fix(proxy): apply temp_budget_increase to multi-window budget_limits#35265
a-korath wants to merge 2 commits into
BerriAI:litellm_internal_stagingfrom
a-korath:fix/temp-budget-increase-multi-window

Conversation

@a-korath

@a-korath a-korath commented Jul 30, 2026

Copy link
Copy Markdown

TLDR

Problem this solves:

  • temp_budget_increase is silently ignored on budget_limits keys
  • The temp-boost feature is a complete no-op for multi-window budgets, across both the auth check and the pre-call reservation gate

How it solves it:

  • Both multi-window enforcement paths (auth-time check and pre-call reservation) now read the active boost
  • Each finite window's effective limit is raised by the boost through one shared helper so the two paths cannot diverge

Relevant issues

Fixes #35247

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Reproduction below uses tiny cent-scale windows so a real over-budget state is reached in a couple of live chat calls rather than needing to burn $500. Captured at commit c8b55a0 against a live proxy hitting a real provider

  1. Create a key with only a multi-window budget_limits (no top-level max_budget), a 24h window capped at $0.01
curl -s http://localhost:4000/key/generate -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"budget_limits":[{"budget_duration":"24h","max_budget":0.01}]}' | jq '{key, budget_limits}'
  1. Send chat calls with that key until the 24h window is exceeded; the request fails with BudgetExceededError: Key over 24h budget
for i in 1 2 3 4 5; do
  curl -s http://localhost:4000/v1/chat/completions -H "Authorization: Bearer <key_from_step_1>" -H "Content-Type: application/json" \
    -d '{"model":"gpt-4o","messages":[{"role":"user","content":"write a 200 word story"}]}' | jq -r '.error.message // "ok"'
done
  1. Grant a temp boost via /key/update
curl -s http://localhost:4000/key/update -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d "{\"key\":\"<key_from_step_1>\",\"temp_budget_increase\":0.05,\"temp_budget_expiry\":\"$(python -c 'import datetime;print((datetime.datetime.now(datetime.timezone.utc)+datetime.timedelta(hours=2)).isoformat())')\"}" | jq '.metadata'
  1. Retry the request from step 2; before this PR it kept returning the same BudgetExceededError, after this PR it succeeds until the boost expires or the boosted ceiling ($0.06) is hit. Before the reservation-path part of this fix the boosted key cleared the auth check but still blocked at the reservation gate with a distinct Budget has been exceeded! Key=...:24h error, so the boost was only fully honored once both paths were patched

Type

🐛 Bug Fix

Changes

temp_budget_increase / temp_budget_expiry (set via /key/update) were only ever applied to a key's top-level max_budget. _update_key_budget_with_temp_budget_increase returns early when max_budget is None, which is exactly the case for a key configured purely through budget_limits. Multi-window key budgets are enforced by two independent paths and neither read the boost: the auth-time check (_virtual_key_multi_budget_check) and the pre-call reservation (_get_budget_counters in budget_reservation.py). So for any multi-window key the boost saved to metadata, showed up in /key/info, and did nothing to request authorization; patching only the auth check still left the reservation gate blocking a boosted key

Both paths now raise each finite window's effective limit by the active boost, read through the existing _get_temp_budget_increase helper, and route through a single apply_temp_budget_increase helper so the two enforcement sites cannot drift. Infinite windows are left untouched since a boost on inf is still inf, and expired boosts return None so they change nothing. The boost is key-level metadata, so team, user, and org window builders intentionally do not receive it. On the auth path the boosted ceiling also flows into get_current_spend so its stale-counter re-check uses the same threshold, and the raised BudgetExceededError reports the effective limit

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

temp_budget_increase / temp_budget_expiry were silently ignored for keys
whose budget is configured via budget_limits (multi-window) instead of a
top-level max_budget. _update_key_budget_with_temp_budget_increase returns
early when max_budget is None, and _virtual_key_multi_budget_check never
read the boost, so the feature was a complete no-op for such keys.

_virtual_key_multi_budget_check now reads the active boost via
_get_temp_budget_increase and raises each finite window's effective limit
by that amount, mirroring the single-max_budget behaviour. Expired boosts
are ignored.

Fixes BerriAI#35247
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Comment thread litellm/proxy/auth/auth_checks.py Outdated
Comment on lines +3660 to +3661
if math.isfinite(effective_max_budget):
effective_max_budget += temp_budget_increase

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 security Non-finite boosts disable enforcement

When a key update supplies Infinity or NaN with a future expiry, this addition makes the effective limit non-finite, causing the subsequent math.isfinite condition to skip enforcement of the otherwise finite budget window until expiry. Reject non-finite increases at ingestion or fail closed here. How this was verified: The accepted update value flows unchanged from persisted token metadata through _get_temp_budget_increase into this addition and the subsequent finiteness guard.

Rule Used: What: Fail any PR which may contains a security in... (source)

Knowledge Base Used: Proxy Authentication and Authorization

@greptile-apps

greptile-apps Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR applies active temporary budget increases to finite multi-window key budgets.

  • Computes an effective boosted limit for each finite budget window.
  • Uses the effective limit for spend rechecks, enforcement, and error reporting.
  • Adds tests for active, exceeded, and expired temporary increases.

Confidence Score: 4/5

The PR should not merge until non-finite temporary increases are prevented from disabling multi-window budget enforcement.

The new arithmetic correctly handles ordinary finite boosts, but an accepted non-finite value makes the effective limit fail the enforcement guard and leaves the affected key window uncapped until expiry.

Files Needing Attention: litellm/proxy/auth/auth_checks.py

Security Review

A non-finite temporary increase can make the new effective multi-window limit non-finite, causing enforcement of that window to be skipped until expiry.

Important Files Changed

Filename Overview
litellm/proxy/auth/auth_checks.py Applies temporary increases consistently to multi-window checks, but does not fail closed when the resulting limit is non-finite.
tests/test_litellm/proxy/auth/test_multi_budget_windows.py Adds focused coverage for ordinary active and expired boosts, but does not cover non-finite boost values.

Reviews (1): Last reviewed commit: "fix(proxy): apply temp_budget_increase t..." | Re-trigger Greptile

from litellm.proxy.auth.user_api_key_auth import _get_temp_budget_increase
from litellm.proxy.proxy_server import get_current_spend

temp_budget_increase = _get_temp_budget_increase(valid_token) or 0.0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: Unprivileged users can raise their own budget

temp_budget_increase is trusted here, but /key/update does not classify either the dedicated temporary-grant fields or the equivalent raw metadata keys as a budget change. A key owner or team member with /key/update can therefore set a future expiry and an arbitrarily large increase, which this change now applies to every multi-window limit. Require the same admin authorization as max_budget and budget_limits, validate that the increase is finite and non-negative, and reject these reserved keys when supplied through raw metadata.

@veria-ai

veria-ai Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

PR overview

This pull request updates proxy authorization checks so temporary budget increases are applied to multi-window budget limits.

One security issue remains open: users with key or team update access can grant themselves an arbitrarily large temporary budget increase, bypassing the intended budget controls across all configured windows. No reported issues have yet been addressed, so authorization and input validation for these temporary grant fields still need to be enforced.

Open issues (1)

Fixed/addressed: 0 · PR risk: 6/10

@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing a-korath:fix/temp-budget-increase-multi-window (e91036e) with litellm_internal_staging (4eecf7a)

Open in CodSpeed

Issue BerriAI#35247 has two enforcement paths for multi-window key budgets: the
auth-time check and the pre-call reservation. The prior commit only patched
the auth check, so a boosted key still blocked at the reservation gate.

Thread the boost through _get_budget_limit_counters and route both paths
through a shared apply_temp_budget_increase helper so they cannot diverge.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: temp_budget_increase has no effect on keys using budget_limits (multi-window budgets)

3 participants