cache/s3: only request required upload checksums#6970
Open
crazy-max wants to merge 2 commits into
Open
Conversation
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
crazy-max
force-pushed
the
fix-s3-cache-aws-chunked
branch
from
July 20, 2026 10:04
6732e81 to
5efadf0
Compare
crazy-max
marked this pull request as ready for review
July 20, 2026 10:28
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes #6969
This updates the vendored AWS SDK for Go v2 modules and fixes S3 cache exports so BuildKit no longer sends optional trailing checksum uploads that force
Content-Encoding: aws-chunked.The dependency update is needed because the old transfer manager API doesn't expose
RequestChecksumCalculation. The newer transfer manager release exposes that policy directly, so BuildKit can apply the samewhen_requiredbehavior at the upload layer that it already applies at the S3 client layer.The S3 cache client now passes
RequestChecksumCalculationWhenRequiredto the transfer manager, matching the existing S3 client configuration and allowing the SDK to avoid optional default CRC32 upload checksums while still preserving required or explicitly requested checksums.BuildKit already configured the AWS S3 client with
RequestChecksumCalculationWhenRequired:buildkit/cache/remotecache/s3/s3.go
Line 451 in b25689e
But the S3 cache exporter uses the higher-level transfer manager for uploads. Older transfer manager releases defaulted an empty checksum algorithm to CRC32 internally, which made upload requests explicitly request a trailing checksum. For streamed uploads, that explicit trailing checksum forces
aws-chunkedrequest encoding, and some S3-compatible backends reject that encoding with501 NotImplemented.