Skip to content

feat(BA-7365): add a per-user rate limit middleware to the web server - #13771

Draft
jopemachine wants to merge 3 commits into
feature/BA-7363-publish-user-rate-limitfrom
feature/BA-7365-webserver-user-rate-limit-middleware
Draft

feat(BA-7365): add a per-user rate limit middleware to the web server#13771
jopemachine wants to merge 3 commits into
feature/BA-7363-publish-user-rate-limitfrom
feature/BA-7365-webserver-user-rate-limit-middleware

Conversation

@jopemachine

@jopemachine jopemachine commented Aug 13, 2026

Copy link
Copy Markdown
Member

📚 Stacked PRs

This PR is part of a 5-PR stack. Merge in order:

  1. ⬇️ feat(BA-7364): deliver the login user id in the auth response #13777feat(BA-7364): deliver the login user id in the auth response
  2. ⬇️ feat(BA-7365): key the rate limit counter by user id #13778feat(BA-7365): key the rate limit counter by user id
  3. ⬇️ feat(BA-7362): move the per-user API rate limit to the user resource policy #13779feat(BA-7362): move the per-user API rate limit to the user resource policy
  4. ⬇️ feat(BA-7363): publish the per-user rate limit to the shared Redis DB #13784feat(BA-7363): publish the per-user rate limit to the shared Redis DB
  5. 👉 feat(BA-7365): add a per-user rate limit middleware to the web server #13771feat(BA-7365): add a per-user rate limit middleware to the web server ← you are here

Only the final tip (#13771) is guaranteed to build / pass CI; intermediate PRs are logical slices for reviewability.

Summary

  • Add a web server middleware that counts requests proxied to the manager (/func/*) against the login user's rolling counter and rejects over-limit requests with HTTP 429, so floods no longer consume manager resources (auth DB lookup, Valkey ops).
  • The limit is the keypair rate_limit carried in the session; None means counted but never rejected. Unauthenticated requests and sessions from managers that do not send the user id yet pass through unchanged.
  • The web server is the primary limiter; the manager-side middleware stays as the fallback for clients that still reach the manager directly. While both paths exist, a proxied request increments the shared counter twice — a transitional cost that disappears once direct manager access is closed off.

Verified against a live cluster (manager + web server on this branch, halfstack Valkey/PostgreSQL): the counter is a single user.<uuid> key; 429 once the counter passes the limit; rate_limit = NULL counted but never rejected; unauthenticated /func/*, non-/func paths and sessions without user_id pass through uncounted; the manager-only path allows exactly rate_limit requests.

Resolves BA-7365.

@github-actions github-actions Bot added size:L 100~500 LoC comp:webserver Related to Web Server component labels Aug 13, 2026
jopemachine added a commit that referenced this pull request Aug 13, 2026
@github-actions github-actions Bot added comp:manager Related to Manager component comp:common Related to Common component labels Aug 13, 2026
@jopemachine
jopemachine changed the base branch from feature/BA-7364-deliver-user-id-to-webserver-session to main August 13, 2026 09:19
@jopemachine
jopemachine force-pushed the feature/BA-7365-webserver-user-rate-limit-middleware branch from 6f6de7b to 276aea8 Compare August 14, 2026 05:57
@jopemachine
jopemachine changed the base branch from main to feature/BA-7365-user-keyed-rate-limit-counter August 14, 2026 05:57
@jopemachine
jopemachine force-pushed the feature/BA-7365-user-keyed-rate-limit-counter branch from 3f23516 to ef4b9b6 Compare August 14, 2026 05:58
@jopemachine
jopemachine force-pushed the feature/BA-7365-webserver-user-rate-limit-middleware branch from 276aea8 to 435c734 Compare August 14, 2026 05:58
@jopemachine jopemachine changed the title feat(BA-7365): add a per-user rate limit middleware to the webserver feat(BA-7365): add a per-user rate limit middleware to the web server Aug 14, 2026
@jopemachine
jopemachine force-pushed the feature/BA-7365-user-keyed-rate-limit-counter branch 3 times, most recently from 1b8bfd3 to ecf3a77 Compare August 14, 2026 14:08
@jopemachine
jopemachine force-pushed the feature/BA-7365-webserver-user-rate-limit-middleware branch from 435c734 to 9429438 Compare August 14, 2026 14:20
@github-actions github-actions Bot added size:XL 500~ LoC comp:client Related to Client component comp:cli Related to CLI component require:db-migration Automatically set when alembic migrations are added or updated and removed size:L 100~500 LoC labels Aug 14, 2026
jopemachine and others added 2 commits August 14, 2026 23:20
Requests proxied to the manager (`/func/*`) are counted against the login
user's rolling counter and rejected with HTTP 429 once the session's rate
limit is passed, so floods no longer reach the manager. The middleware is
registered after `setup_session()` because it reads the session storage the
session middleware installs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The limit is no longer carried in the session token, so the middleware reads
the value the manager publishes for the user. A user with no published limit
passes through: the manager republishes on every authorized request, so the
value is missing only before a user's first proxied request or after a window
of inactivity, and the manager-side limiter covers both.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jopemachine
jopemachine force-pushed the feature/BA-7365-webserver-user-rate-limit-middleware branch from 9429438 to 8c8b746 Compare August 14, 2026 14:20
@jopemachine
jopemachine changed the base branch from feature/BA-7365-user-keyed-rate-limit-counter to feature/BA-7363-publish-user-rate-limit August 14, 2026 14:20
Co-authored-by: octodog <mu001@lablup.com>
@github-actions github-actions Bot added the area:docs Documentations label Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:docs Documentations comp:cli Related to CLI component comp:client Related to Client component comp:common Related to Common component comp:manager Related to Manager component comp:webserver Related to Web Server component require:db-migration Automatically set when alembic migrations are added or updated size:L 100~500 LoC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant