flex checksum validated result + response extensions - #4693
Draft
aajtodd wants to merge 3 commits into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
The SDK validates response checksums, but a caller has no way to learn whether validation actually happened or which algorithm was used. Validation runs lazily inside the response body as it is consumed: a match completes silently at end-of-stream, and a mismatch surfaces as a body error. There is no caller-observable signal for "this response was validated with CRC32," and "the body reached EOF cleanly" is ambiguous — it means either validated-and-matched or never-validated (no checksum header, or a part-level checksum the SDK skips).
This change makes the validation outcome observable on the operation output. It also introduces a generic mechanism — an
Extensionstype-map on every output, populated from the config bag during deserialization — that future features (for example per-request telemetry) can reuse to attach auxiliary, non-modeled data to an output.Description
A caller reads the outcome from the output's extensions after consuming the response body:
The design separates a generic extensions mechanism from the checksum-specific consumer.
Generic:
Extensionson every output. A newaws_smithy_types::extensions::Extensionsis a type-erased map (over the existingTypeErasedBox) exposed through aProvideExtensionstrait. Every generated operation output carries one, in an inlinedEqIgnorewrapper so it does not participate in the output's derivedPartialEq(Extensionshas no meaningful equality) while the output keeps its blanket derives. Population is generic: interceptors accumulate anExtensionsin the config bag, and the generated response deserializer lifts it onto the output via a synthetic_set_extensionsbefore the output is type-erased. This lift is rendered uniformly across every deserializer path and is not checksum-aware.Consumer: response checksum validation.
aws-smithy-checksumsgainsValidationOutcome,NotValidatedReason, andResponseChecksumValidationResult— a cheaply-cloneable handle over a shared cell.ChecksumBodyrecordsValidated { algorithm }into the handle on a clean end-of-stream (a mismatch stays a body error and leaves the cell unset). The response checksum interceptor creates the handle, wraps the body with it, and inserts it into the config-bagExtensions; on the paths where it does not wrap the body it recordsNotValidated { NoChecksum }orNotValidated { PartLevelChecksum }directly.Because validation is consume-time, the timing of the observable outcome follows the body:
send()returns.GetObject) the body is handed to the caller, so the outcome resolves once the caller drains it.How to review
Commit-by-commit:
define extensions type— the generic substrate:Extensions+ProvideExtensionsinaws-smithy-types, the inlinedEqIgnorewrapper, andOutputExtensionsDecoratoradding the field, accessor, and_set_extensionsto every output. Standalone codegen test proves an output exposesextensions().wire checksum validation into output extensions and add tests— the consumer and the lift: the validation-outcome types andChecksumBodycell-write inaws-smithy-checksums, the interceptor writing the handle into the config-bagExtensions,Storable for Extensions, and the generic lift inResponseDeserializerGenerator.HttpChecksumTestasserts the outcome for all algorithms.add additional tests— the changelog entry and anaws-sdk-s3integration test proving the lazy streaming outcome on a realGetObject(unresolved before the body is drained,Validatedafter).Notable details
deserialize_streamingtodeserialize_streaming_with_configso the config bag is in scope for the lift. This affects every legacy-path streaming operation, which is why the change is rendered uniformly rather than gated to checksum operations.Extensionsis intentionally notPartialEq; theEqIgnorewrapper is inlined into generated crates rather than added to the stableaws-smithy-typesAPI.aws-smithy-checksumsandaws-smithy-typespatch versions are bumped.Testing
HttpChecksumTest(codegen): assertsValidated { algorithm }on the output extensions for CRC32, CRC32C, CRC64NVME, SHA1, and SHA256, plus a no-checksum-header case assertingNotValidated { NoChecksum }. The existing mismatch test (surfacing as an error) is unchanged.OutputExtensionsDecoratorTest(codegen): a generated output exposesextensions().aws-smithy-checksumsunit tests: the handle isValidatedon a clean read and left unset on a mismatch.aws-sdk-s3integration test (test_validation_outcome_recorded_on_streaming_output): on a streamingGetObject, the outcome isNonebefore the body is drained andValidated { Crc32 }after.Checklist
.changelogdirectory, specifying "client," "server," or both in theapplies_tokey..changelogdirectory, specifying "aws-sdk-rust" in theapplies_tokey.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.