nodeup: stream verified image bytes into ctr import#18278
Merged
Conversation
k8s-ci-robot
added
cncf-cla: yes
hakman
force-pushed
the
stream-container-images
branch
from
2dde5f0 to
f349d34
Compare
Replace the download → ungzip-to-disk → reference-by-path flow with a single streaming pipeline: download to a hash-named cache file with inline hash verification, then re-open and stream (transparently gunzipping via magic-byte detection) straight into ctr images import over stdin. This avoids materializing an extracted tar on disk and prevents hash-mismatched bytes from ever reaching containerd's image store, since ctr atomically commits image references from the archive's manifest at import time. fi.DownloadURL now streams through a MultiWriter hasher with atomic rename, and a new fi.OpenURL helper exposes the hardened HTTP GET stream that the image import path reuses. - Reject non-2xx HTTP responses - Wait for containerd readiness via TryAgainLaterError, bounded by a 5s probe - Skip the global download backoff for ctr-side failures - Detect gzip by magic bytes rather than file suffix - Key LoadImageTask by source filename and reject sources-less images at config load Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
hakman
force-pushed
the
stream-container-images
branch
from
f349d34 to
30d83e0
Compare
k8s-ci-robot
added
size/L
Member
|
/lgtm |
k8s-ci-robot
added
the
lgtm
rifelpet
approved these changes
May 6, 2026
Contributor
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: rifelpet The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace the download → ungzip-to-disk → reference-by-path flow with a single streaming pipeline: download to a hash-named cache file with inline hash verification, then re-open and stream (transparently gunzipping via magic-byte detection) straight into ctr images import over stdin. This avoids materializing an extracted tar on disk and prevents hash-mismatched bytes from ever reaching containerd's image store, since ctr atomically commits image references from the archive's manifest at import time.
fi.DownloadURL now streams through a MultiWriter hasher with atomic rename, and a new fi.OpenURL helper exposes the hardened HTTP GET stream that the image import path reuses.
Assisted by Claude Opus
/cc @ameukam @rifelpet