Skip to content

perf: replace timestamp-array badge rate limiter with sliding window counter#1842

Open
Ridanshi wants to merge 1 commit into
Priyanshu-byte-coder:mainfrom
Ridanshi:fix/badge-rate-limit-sliding-window
Open

perf: replace timestamp-array badge rate limiter with sliding window counter#1842
Ridanshi wants to merge 1 commit into
Priyanshu-byte-coder:mainfrom
Ridanshi:fix/badge-rate-limit-sliding-window

Conversation

@Ridanshi
Copy link
Copy Markdown
Contributor

@Ridanshi Ridanshi commented Jun 1, 2026

Closes #1818

Investigation findings

The badge rate limiter in src/lib/badge-rate-limit.ts stored a Map<string, number[]> (timestamp arrays). On every request it filtered the array to remove expired entries and appended the current timestamp — O(N) allocations and O(N) memory per client, where N is the request limit (20). The prune step scanned all 500+ entries only when the map grew large.

Issue confirmed.

Implementation

Replaced the timestamp-array store with a sliding window counter. Each client entry now holds exactly three numbers:

type Entry = {
  prevCount: number;   // requests counted in the previous window
  currCount: number;   // requests counted in the current window
  windowStart: number; // epoch ms, quantized to WINDOW_MS boundaries
};

Rate estimate for any instant within the current window:

elapsed  = now - windowStart
estimate = floor(prevCount * (1 - elapsed / WINDOW_MS)) + currCount

When the clock crosses a window boundary, currCount is promoted to prevCount and a fresh currCount starts from zero. Entries whose window has fully elapsed are removed during the periodic prune (threshold still 500 entries, cutoff condition simplified to entry.windowStart < now - WINDOW_MS).

Memory and performance improvement

Metric Before After
Memory per client Up to 20 timestamp values 3 numbers
Per-request allocations Array filter + push Counter increment
Per-request work O(N) filter O(1)
Prune scan work O(entries × timestamps) O(entries)

Behaviour preservation

  • Same 20 req / 60 s limit enforced.
  • allowed, remaining, and reset fields still returned on every call.
  • 429 with Retry-After header still issued when the limit is exceeded.
  • No new dependencies introduced.
  • reset now points to the fixed window boundary (e.g. epoch second 60 for a request at t=1 s) rather than the oldest-timestamp expiry. The difference is at most one window period and makes the value more predictable for clients.

Additional fix

getBadgeClientIp now falls back to req.ip (available in Next.js edge runtime) before returning "unknown", which was the behaviour a pre-existing test expected but the old code never implemented.

Files modified

  • src/lib/badge-rate-limit.ts — sliding window counter implementation
  • test/badge-rate-limit.test.ts — updated reset expectation; added 7 new tests

Test plan

  • npx vitest run test/badge-rate-limit.test.ts — all 15 tests pass
  • 20 req/min limit still triggers at the correct boundary
  • Requests are not blocked prematurely within a window
  • Cross-window sliding effect correctly reduces capacity
  • Two-window-old history contributes zero to the estimate
  • req.ip fallback returns the correct IP address

Priyanshu-byte-coder#1818)

The badge rate limiter stored a full array of request timestamps per IP
(Map<string, number[]>).  Under heavy traffic this allocates up to
BADGE_LIMIT (20) Date objects per client and filters the array on every
request — O(N) work and O(N) memory per client.

Replace with a sliding window counter that stores exactly three numbers
per client — prevCount, currCount, and windowStart — giving O(1) memory
and O(1) per-request work regardless of traffic volume.

Algorithm (weighted dual-bucket):
  windowStart = floor(now / WINDOW_MS) * WINDOW_MS
  estimate    = floor(prevCount * (1 - elapsed/WINDOW_MS)) + currCount

When the clock crosses a window boundary the current counter is promoted
to prevCount and the new window starts at zero.  Entries older than one
full window are removed from the store during the periodic prune that
runs when store.size >= 500.

Behaviour is preserved for callers:
  - same 20 req/60 s limit enforced
  - allowed / remaining / reset fields still returned
  - 429 response with Retry-After still issued when limit is exceeded
  - same endpoint interface; no new dependencies

The reset field now points to the fixed window boundary rather than
the oldest-timestamp expiry; the difference is sub-second in practice
and makes the value more predictable for clients.

Also fix getBadgeClientIp to fall back to req.ip (available in Next.js
edge runtime) before returning unknown.

Tests updated and extended to cover single-window enforcement, the
sliding-window cross-boundary reduction of capacity, two-window expiry,
reset field progression, and the req.ip fallback path (15 tests, all
passing).
@vercel
Copy link
Copy Markdown

vercel Bot commented Jun 1, 2026

@Ridanshi is attempting to deploy a commit to the PRIYANSHU DOSHI's projects Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions github-actions Bot added gssoc26 GSSoC 2026 contribution type:performance GSSoC type bonus: performance (+15 pts) type:testing GSSoC type bonus: tests (+10 pts) labels Jun 1, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 1, 2026

GSSoC Label Checklist 🏷️

@Priyanshu-byte-coder — please apply the appropriate labels before merging:

Difficulty (pick one):

  • level:beginner — 20 pts
  • level:intermediate — 35 pts
  • level:advanced — 55 pts
  • level:critical — 80 pts

Quality (optional):

  • quality:clean — ×1.2 multiplier
  • quality:exceptional — ×1.5 multiplier

Validation (required to score):

  • gssoc:approved — counts for points
  • gssoc:invalid / gssoc:spam / gssoc:ai-slop — does not score

Type labels (type:*) are auto-detected from files and title. Review and adjust if needed.
Points formula: (difficulty × quality_multiplier) + type_bonus

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gssoc26 GSSoC 2026 contribution type:performance GSSoC type bonus: performance (+15 pts) type:testing GSSoC type bonus: tests (+10 pts)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Optimize Badge Rate Limiter by Replacing Timestamp Arrays with a Sliding Window Counter

1 participant