Warn when packument ref fan-out nears the subrequest cap#43
Merged
Conversation
A packument rebuild issues ~3 + one-per-ref R2 reads; a cold request with a large active-ref set could approach Cloudflare's per-request subrequest cap (50 on most plans). Log a warning past ~40 refs so the fan-out (a per-package meta aggregate) can be addressed before it becomes a hard failure. Rebuilds are rare (the F5 output cache), so this stays quiet until the ref set is genuinely large.
fengmk2
added a commit
that referenced
this pull request
Jul 1, 2026
…o ref limit) (#44) Removes the effective limit on the number of refs. Closes the F1 fan-out. ## Problem The packument rebuild read one `metaKey` per configured ref (K R2 reads). A cold rebuild's subrequest count grew with the ref set, so continuous releases would eventually push it past Cloudflare's per-request subrequest cap , a hard ceiling on how many refs could accumulate. (The `#43` tripwire only warned about this; it didn't remove it.) ## Fix A per-package aggregate the packument reads in one shot: - `/-/publish` upserts each package's meta into `meta-index/<name>` (`version -> meta`), a CAS read-modify-write like the refs index, pruning expired entries on write. - The packument reads that **one** object instead of K per-version keys. Subrequests per rebuild are now flat , `refs-index + meta-index + npm base + time` , **regardless of ref count**. - A ref not yet in the aggregate (published before this deploys) falls back to its per-version `metaKey`; that path fades as refs republish or expire (90-day TTL). Platform packages that were never published for a given ref still fall back to name-derived metadata. - `/-/purge` removes from the aggregate too, so a purged build stops being served. Extracted the R2 etag-CAS loop into `casR2Json`, now shared by the refs index and the meta index. Removed the `#43` subrequest-cap tripwire , the limit is gone. ## Trade-off (a soft cost, not a cap) Each publish rewrites the aggregate (O(K) write), the same shape the refs index already has, and bounded by the 90-day TTL (steady-state K = release-rate × 90 days, not unbounded). The packument body still scales with K (inherent to any registry) but is cached (F5) + gzipped. ## Verify `tsc` clean, 75 tests pass, action bundle unaffected. New tests: the packument injects a version with its per-version key **deleted** (proving it reads the aggregate), and purge drops the version from the aggregate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A cheap early-warning guard (the one thing worth watching from the perf review, not fixing yet).
A packument rebuild issues ~3 + one-per-ref R2 reads. A cold request with a large active-ref set could approach Cloudflare's per-request subrequest cap (50 on most plans, 1000 unbound). This logs a
console.warnonce the ref count crosses ~40, so the fan-out (a per-package meta aggregate, deferred as "F1") can be addressed before it could ever hit the cap.Rebuilds are rare (the F5 output cache serves hits without touching refs), so this stays silent until the active-ref set is genuinely large , it's a tripwire, not a hot-path cost.
tscclean, 73 tests pass (no behavior change; K is small in tests so it never fires).