Skip to content

feat: Persistent Buildkit Daemon for build host#5452

Merged
junpuf merged 9 commits intomainfrom
buildkitd
Nov 6, 2025
Merged

feat: Persistent Buildkit Daemon for build host#5452
junpuf merged 9 commits intomainfrom
buildkitd

Conversation

@junpuf
Copy link
Contributor

@junpuf junpuf commented Nov 6, 2025

This PR adds a bootstrap script and systemd service that provision a long-running BuildKit daemon on each build host. Which will dramatically reduces rebuild time by reusing cached layers across jobs.

Highlights

Persistent cache:

BuildKit stores layer data under the largest available volume (e.g. /buildkit), surviving across builds and reboots.

Dynamic GC policy:

Cache size automatically scales to ~80 % of available disk space, with recency-based pruning (configurable keep window).

Idempotent startup:

The bootstrap script installs BuildKit if missing, generates config only when needed, detects drift via diff, and restarts the daemon only when configuration changes.

Systemd integration:

Managed as buildkitd.service—auto-starts on boot, restarts on failure, no manual process management required.

@junpuf junpuf changed the title install buildkit Persistent Buildkit Daemon for build host Nov 6, 2025
@aws-deep-learning-containers-ci aws-deep-learning-containers-ci bot added the Size:XL Determines the size of the PR label Nov 6, 2025
@junpuf junpuf changed the title Persistent Buildkit Daemon for build host feat: Persistent Buildkit Daemon for build host Nov 6, 2025
@junpuf junpuf merged commit a9538c4 into main Nov 6, 2025
8 checks passed
@junpuf junpuf deleted the buildkitd branch November 6, 2025 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

authorized Size:L Size:M Size:XL Determines the size of the PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants