Fix SigV4 signing failure on bulk retry after partial success - #759
Merged
lawofcycles merged 2 commits intoMay 16, 2026
Merged
Conversation
lawofcycles
force-pushed
the
fix/sigv4-bulk-retry
branch
3 times, most recently
from
May 12, 2026 16:24
08a1f2b to
4f42b9a
Compare
Xtansia
approved these changes
May 12, 2026
TrackingBytesArray.toInputStream() returned the entire underlying byte array including bytes of removed entries, while writeTo() correctly skipped them. This caused a mismatch between the x-amz-content-sha256 header (computed from toInputStream()) and the actual HTTP body (sent via writeTo()) when retrying a bulk request after partial success. Changed toInputStream() to build its result using the same entry traversal logic as writeTo(). Signed-off-by: Sotaro Hikita <bering1814@gmail.com>
lawofcycles
force-pushed
the
fix/sigv4-bulk-retry
branch
from
May 14, 2026 22:42
4f42b9a to
dae4eee
Compare
Signed-off-by: Sotaro Hikita <bering1814@gmail.com>
Collaborator
Author
|
@Xtansia Thank you for the review! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fix SigV4 signing failure that occurs when a bulk request is retried after partial success.
TrackingBytesArray.toInputStream()returned the entire underlying byte array including bytes of removed entries, whilewriteTo()correctly skipped them. After a partial bulk success,BulkProcessorremoves successfully indexed entries from the buffer and retries the rest. On retry,AwsV4SignerSupportcomputes the content SHA256 fromtoInputStream()(stale data), but the HTTP body is sent viawriteTo()(correct data). The mismatch causes the server to reject the request with403: Request header x-amz-content-sha256 invalid.This fix changes
toInputStream()to use the same entry traversal logic aswriteTo(), ensuring both methods always return the same byte sequence.Issues Resolved
Closes #758
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.