Skip to content

flex checksum validated result + response extensions - #4693

Draft
aajtodd wants to merge 3 commits into
mainfrom
aajtodd/checksum-result
Draft

flex checksum validated result + response extensions#4693
aajtodd wants to merge 3 commits into
mainfrom
aajtodd/checksum-result

Conversation

@aajtodd

@aajtodd aajtodd commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Motivation and Context

The SDK validates response checksums, but a caller has no way to learn whether validation actually happened or which algorithm was used. Validation runs lazily inside the response body as it is consumed: a match completes silently at end-of-stream, and a mismatch surfaces as a body error. There is no caller-observable signal for "this response was validated with CRC32," and "the body reached EOF cleanly" is ambiguous — it means either validated-and-matched or never-validated (no checksum header, or a part-level checksum the SDK skips).

This change makes the validation outcome observable on the operation output. It also introduces a generic mechanism — an Extensions type-map on every output, populated from the config bag during deserialization — that future features (for example per-request telemetry) can reuse to attach auxiliary, non-modeled data to an output.

Description

A caller reads the outcome from the output's extensions after consuming the response body:

use aws_sdk_s3::operation::ProvideExtensions;
use aws_smithy_checksums::body::validate::{ResponseChecksumValidationResult, ValidationOutcome};

let output = client.get_object().bucket(b).key(k).checksum_mode(ChecksumMode::Enabled).send().await?;
let validation = output.extensions().get::<ResponseChecksumValidationResult>().cloned();

// drive validation to completion by consuming the body
while let Some(chunk) = output.body.next().await { /* ... */ }

match validation.and_then(|v| v.outcome()) {
    Some(ValidationOutcome::Validated { algorithm }) => { /* validated with `algorithm` */ }
    Some(ValidationOutcome::NotValidated { reason }) => { /* not validated, see `reason` */ }
    None => { /* body not yet fully consumed */ }
}

The design separates a generic extensions mechanism from the checksum-specific consumer.

Generic: Extensions on every output. A new aws_smithy_types::extensions::Extensions is a type-erased map (over the existing TypeErasedBox) exposed through a ProvideExtensions trait. Every generated operation output carries one, in an inlined EqIgnore wrapper so it does not participate in the output's derived PartialEq (Extensions has no meaningful equality) while the output keeps its blanket derives. Population is generic: interceptors accumulate an Extensions in the config bag, and the generated response deserializer lifts it onto the output via a synthetic _set_extensions before the output is type-erased. This lift is rendered uniformly across every deserializer path and is not checksum-aware.

Consumer: response checksum validation. aws-smithy-checksums gains ValidationOutcome, NotValidatedReason, and ResponseChecksumValidationResult — a cheaply-cloneable handle over a shared cell. ChecksumBody records Validated { algorithm } into the handle on a clean end-of-stream (a mismatch stays a body error and leaves the cell unset). The response checksum interceptor creates the handle, wraps the body with it, and inserts it into the config-bag Extensions; on the paths where it does not wrap the body it records NotValidated { NoChecksum } or NotValidated { PartLevelChecksum } directly.

Because validation is consume-time, the timing of the observable outcome follows the body:

  • For non-streaming operations the orchestrator reads the body to completion before deserialization, so the outcome is resolved by the time send() returns.
  • For streaming operations (for example GetObject) the body is handed to the caller, so the outcome resolves once the caller drains it.

How to review

Commit-by-commit:

  1. define extensions type — the generic substrate: Extensions + ProvideExtensions in aws-smithy-types, the inlined EqIgnore wrapper, and OutputExtensionsDecorator adding the field, accessor, and _set_extensions to every output. Standalone codegen test proves an output exposes extensions().
  2. wire checksum validation into output extensions and add tests — the consumer and the lift: the validation-outcome types and ChecksumBody cell-write in aws-smithy-checksums, the interceptor writing the handle into the config-bag Extensions, Storable for Extensions, and the generic lift in ResponseDeserializerGenerator. HttpChecksumTest asserts the outcome for all algorithms.
  3. add additional tests — the changelog entry and an aws-sdk-s3 integration test proving the lazy streaming outcome on a real GetObject (unresolved before the body is drained, Validated after).

Notable details

  • The legacy streaming deserializer was switched from the deprecated deserialize_streaming to deserialize_streaming_with_config so the config bag is in scope for the lift. This affects every legacy-path streaming operation, which is why the change is rendered uniformly rather than gated to checksum operations.
  • Extensions is intentionally not PartialEq; the EqIgnore wrapper is inlined into generated crates rather than added to the stable aws-smithy-types API.
  • aws-smithy-checksums and aws-smithy-types patch versions are bumped.

Testing

  • HttpChecksumTest (codegen): asserts Validated { algorithm } on the output extensions for CRC32, CRC32C, CRC64NVME, SHA1, and SHA256, plus a no-checksum-header case asserting NotValidated { NoChecksum }. The existing mismatch test (surfacing as an error) is unchanged.
  • OutputExtensionsDecoratorTest (codegen): a generated output exposes extensions().
  • aws-smithy-checksums unit tests: the handle is Validated on a clean read and left unset on a mismatch.
  • aws-sdk-s3 integration test (test_validation_outcome_recorded_on_streaming_output): on a streaming GetObject, the outcome is None before the body is drained and Validated { Crc32 } after.

Checklist

  • For changes to the smithy-rs codegen or runtime crates, I have created a changelog entry Markdown file in the .changelog directory, specifying "client," "server," or both in the applies_to key.
  • For changes to the AWS SDK, generated SDK code, or SDK runtime crates, I have created a changelog entry Markdown file in the .changelog directory, specifying "aws-sdk-rust" in the applies_to key.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant