Daily cron: sweep expired preview artifacts from R2#45
Merged
Conversation
Refs self-expire from the indexes (skipped on read, pruned on the next write), but their per-version R2 objects (meta/<name>/<version>.json and the tarball) were never auto-deleted, only on explicit purge, so R2 storage grew with every ref ever published. Add cleanupExpiredArtifacts: it lists the meta/ and tarball/ prefixes and bulk-deletes objects whose uploaded age exceeds REF_TTL_MS. Each object is re-uploaded on republish, so uploaded age tracks the ref's age and the cutoff maps exactly to the ref TTL, an active ref is never touched, and the live indexes (refs/, meta-index/) are under different prefixes and left alone. A Void cron (crons/cleanup-expired.ts, daily) runs it; the generated Worker already dispatches scheduled events, so this composes with the routes/ app entry.
The expiry sweep hardcoded ['meta/', 'tarball/'] kept in sync with metaKey/ tarballKey only by a 'must match' comment; a prefix rename would silently desync the cron (it would stop matching and R2 would leak with no error). Export the two prefixes from r2Cache, use them in both the key builders and the sweep.
fengmk2
force-pushed
the
cron-cleanup-expired
branch
from
2495fdd to
3dfe6d5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the scheduled cleanup you asked for.
Why
Refs self-expire from the indexes (skipped on read, pruned on the next write), but their per-version R2 objects ,
meta/<name>/<version>.jsonand the tarball , were only ever deleted on explicit/-/purge. So R2 storage grew with every ref ever published, unbounded under continuous releases.What
cleanupExpiredArtifacts(src/preview/cleanupExpired.ts): lists themeta/andtarball/prefixes and bulk-deletes objects whoseuploadedage exceedsREF_TTL_MS. Since a ref's objects are re-uploaded on each publish,uploadedage tracks the ref's age and the cutoff maps exactly to the ref TTL , an active ref is never touched, and the live indexes (refs/,meta-index/) sit under different prefixes and are left alone.crons/cleanup-expired.ts: a Void cron (export const cron = '37 3 * * *', daily) that runs it. Confirmed viable ,.void/entry.tsalready exports{ fetch, scheduled }and dispatches cron jobs, so this composes with theroutes/app entry; no external scheduler.Verify
tscclean (addedcronsto the tsconfig include), 77 tests pass , new tests confirm it deletes past-TTL objects, keeps in-TTL ones, and never touches therefs//meta-index/objects. Bundle unaffected. The staging deploy will exercise the cron-trigger provisioning on the real Void runtime.