Describe the bug
rust-s3 appears to include all headers present in the request HeaderMap when building the SigV4 canonical headers and SignedHeaders value.
This can cause SignatureDoesNotMatch errors with stricter S3-compatible backends such as Ceph RGW, especially after upgrading Ceph RGW to Ceph 20.2.2 Tentacle. In this environment, Ceph RGW appears to validate/canonicalize S3 requests more strictly than before.
The same endpoint, bucket, credentials, region, and path-style configuration work correctly with other S3 clients, for example the PHP AWS SDK.
To Reproduce
Observed with a bucket configured through Stalwart’s S3 blob store, which delegates S3 operations to rust-s3.
The relevant Stalwart call path is essentially:
Bucket::new(&config.bucket, region, credentials)?
.with_path_style();
bucket.put_object(&path, data).await;
bucket.get_object(&path).await;
bucket.head_object(&path).await;
bucket.delete_object(&path).await;
The same Ceph RGW endpoint, bucket, credentials, and path-style setup work with the AWS SDK.
Ceph RGW returns:
SignatureDoesNotMatch
Expected behavior
rust-s3 should avoid signing volatile HTTP/client headers by default, or provide a compatibility option for strict S3-compatible backends. For ordinary S3 object operations, the default/minimal signed-header set should ideally be limited to stable SigV4 headers, for example: host, x-amz-date, x-amz-content-sha256
and when actually required by the request:
content-md5, x-amz-security-token, explicitly configured x-amz-* headers, x-amz-meta-* headers
range Headers such as the following should not be signed unless explicitly requested by the caller:
accept, accept-encoding, user-agent, connection, content-type for bodyless requests, content-length for bodyless requests, amz-sdk-request, amz-sdk-invocation-id
A possible API-level solution could be one of the following:
bucket.set_signed_headers_mode(SignedHeadersMode::Minimal);
or:
bucket.set_signed_headers_allowlist([
"host",
"x-amz-date",
"x-amz-content-sha256",
"x-amz-security-token",
"x-amz-meta-*",
]);
or:
bucket.set_signed_headers_denylist([
"accept",
"accept-encoding",
"user-agent",
"connection",
"amz-sdk-request",
"amz-sdk-invocation-id",
]);
This would keep the existing behavior available while allowing applications to opt into safer behavior for stricter S3-compatible stores such as Ceph RGW, Cloudflare R2, MinIO, etc.
Environment
rust-s3 version: 0.37.2
S3-compatible backend: Ceph RGW Ceph version where the issue appears: 20.2.2 Tentacle
Access mode: path-style S3 access Using client: Stalwart 0.16.11 S3 blob store via rust-s3
Comparison client that works: PHP AWS SDK
Additional context
The main goal is to prevent transport/client headers from accidentally becoming part of the SigV4 canonical request.
This started occurring after upgrading Ceph RGW to Ceph 20.2.2 Tentacle. The Ceph 20.2.2 release includes RGW REST/query handling changes, and RGW appears to validate/canonicalize these S3 requests more strictly than before. This is related in spirit to #466, but it is broader than DeleteObject and Cloudflare R2. The underlying issue seems to be that the signer has no minimal/stable signed-header mode and no allowlist/denylist mechanism for S3-compatible backends with stricter SigV4 validation.
A conceptual implementation could filter the headers before building the canonical request or expose this as configurable behavior so applications can decide which headers are signed.
Describe the bug
rust-s3 appears to include all headers present in the request HeaderMap when building the SigV4 canonical headers and SignedHeaders value.
This can cause SignatureDoesNotMatch errors with stricter S3-compatible backends such as Ceph RGW, especially after upgrading Ceph RGW to Ceph 20.2.2 Tentacle. In this environment, Ceph RGW appears to validate/canonicalize S3 requests more strictly than before.
The same endpoint, bucket, credentials, region, and path-style configuration work correctly with other S3 clients, for example the PHP AWS SDK.
To Reproduce
Observed with a bucket configured through Stalwart’s S3 blob store, which delegates S3 operations to rust-s3.
The relevant Stalwart call path is essentially:
Bucket::new(&config.bucket, region, credentials)?
.with_path_style();
bucket.put_object(&path, data).await;
bucket.get_object(&path).await;
bucket.head_object(&path).await;
bucket.delete_object(&path).await;
The same Ceph RGW endpoint, bucket, credentials, and path-style setup work with the AWS SDK.
Ceph RGW returns:
SignatureDoesNotMatch
Expected behavior
rust-s3 should avoid signing volatile HTTP/client headers by default, or provide a compatibility option for strict S3-compatible backends. For ordinary S3 object operations, the default/minimal signed-header set should ideally be limited to stable SigV4 headers, for example: host, x-amz-date, x-amz-content-sha256
and when actually required by the request:
content-md5, x-amz-security-token, explicitly configured x-amz-* headers, x-amz-meta-* headers
range Headers such as the following should not be signed unless explicitly requested by the caller:
accept, accept-encoding, user-agent, connection, content-type for bodyless requests, content-length for bodyless requests, amz-sdk-request, amz-sdk-invocation-id
A possible API-level solution could be one of the following:
bucket.set_signed_headers_mode(SignedHeadersMode::Minimal);
or:
bucket.set_signed_headers_allowlist([
"host",
"x-amz-date",
"x-amz-content-sha256",
"x-amz-security-token",
"x-amz-meta-*",
]);
or:
bucket.set_signed_headers_denylist([
"accept",
"accept-encoding",
"user-agent",
"connection",
"amz-sdk-request",
"amz-sdk-invocation-id",
]);
This would keep the existing behavior available while allowing applications to opt into safer behavior for stricter S3-compatible stores such as Ceph RGW, Cloudflare R2, MinIO, etc.
Environment
rust-s3 version: 0.37.2
S3-compatible backend: Ceph RGW Ceph version where the issue appears: 20.2.2 Tentacle
Access mode: path-style S3 access Using client: Stalwart 0.16.11 S3 blob store via rust-s3
Comparison client that works: PHP AWS SDK
Additional context
The main goal is to prevent transport/client headers from accidentally becoming part of the SigV4 canonical request.
This started occurring after upgrading Ceph RGW to Ceph 20.2.2 Tentacle. The Ceph 20.2.2 release includes RGW REST/query handling changes, and RGW appears to validate/canonicalize these S3 requests more strictly than before. This is related in spirit to #466, but it is broader than DeleteObject and Cloudflare R2. The underlying issue seems to be that the signer has no minimal/stable signed-header mode and no allowlist/denylist mechanism for S3-compatible backends with stricter SigV4 validation.
A conceptual implementation could filter the headers before building the canonical request or expose this as configurable behavior so applications can decide which headers are signed.