Retry Metric Header included in S3 Presigned URL

[dstufft]

Describe the bug

When creating a Presigned URL for S3, the signed request includes the amz-sdk-request header, which requires the client to pass that header as well. This not only leaks an internal SDK header, but it breaks using the presigned URL as a plain HTTP redirect.

Expected Behavior

The amz-sdk-request header is not included in the Amz-SignedHeaders.

Current Behavior

The amz-sdk-request header is included in the Amz-SignedHeaders, requiring clients to pass amz-sdk-request: attempt=1; max=3 when making the request.

Reproduction Steps

svc := s3.NewPresignClient(s3Client)

var req *v4.PresignedHTTPRequest
req, err := svc.PresignGetObject(ctx,
	&s3.GetObjectInput{
		Bucket: &s3Bucket,
		Key:    &objectKey,
	},
	s3.WithPresignExpires(time.Minute*15),
)


fmt.Printf("%v", req)

Note that the printed output will include the amz-sdk-request header:

&{https://s3.us-east-1.amazonaws.com/bucket/key?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...&X-Amz-SignedHeaders=amz-sdk-request%3Bhost&x-id=GetObject&X-Amz-Signature=xxx GET map[Amz-Sdk-Request:[attempt=1; max=3] Host:[s3.us-east-1.amazonaws.com]]}

Possible Solution

No response

Additional Information/Context

No response

AWS Go SDK V2 Module Versions Used

```
github.com/aws/aws-sdk-go-v2 v1.24.1
github.com/aws/aws-sdk-go-v2/config v1.26.4
github.com/aws/aws-sdk-go-v2/credentials v1.16.15
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.15.8
github.com/aws/aws-sdk-go-v2/service/s3 v1.48.1
```

Compiler and Version used

go version go1.21.5 darwin/arm64

Operating System and version

macOS 14.2.1

[RanVaknin]

Hi @dstufft

This is very odd. That header should not be signed.

In my reproduction I'm not able to reproduce the reported behavior:

func presignGet() {
	cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion("us-west-2"), config.WithClientLogMode(aws.LogRequestWithBody))
	if err != nil {
		panic(err)
	}

	client := s3.NewFromConfig(cfg)

	presigner := s3.NewPresignClient(client)

	input := &s3.GetObjectInput{
		Bucket: aws.String("testbucket"),
		Key:    aws.String("sample2.txt"),
	}
	req, err := presigner.PresignGetObject(context.TODO(),
		input,
		s3.WithPresignExpires(time.Minute*15),
	)

	fmt.Printf("%v", req.URL)
}
/*
URL decoded and formatted for better readability. 

https://testbucket.s3.us-west-2.amazonaws.com/sample2.txt?
X-Amz-Algorithm=AWS4-HMAC-SHA256&
X-Amz-Credential=REDACTED/20240130/us-west-2/s3/aws4_request&
X-Amz-Date=20240130T190301Z&
X-Amz-Expires=900&
X-Amz-SignedHeaders=host&
x-id=GetObject&
X-Amz-Signature=REDACTED
*/

Can you share your complete code snippet including imports and client creation?

I can't think of a reason why this would get signed. Is there any other information you can provide us with about your application?

Thanks,
Ran~

[ysjjovo]

Hi @dstufft

This is very odd. That header should not be signed.

In my reproduction I'm not able to reproduce the reported behavior:

func presignGet() {
	cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion("us-west-2"), config.WithClientLogMode(aws.LogRequestWithBody))
	if err != nil {
		panic(err)
	}

	client := s3.NewFromConfig(cfg)

	presigner := s3.NewPresignClient(client)

	input := &s3.GetObjectInput{
		Bucket: aws.String("testbucket"),
		Key:    aws.String("sample2.txt"),
	}
	req, err := presigner.PresignGetObject(context.TODO(),
		input,
		s3.WithPresignExpires(time.Minute*15),
	)

	fmt.Printf("%v", req.URL)
}
/*
URL decoded and formatted for better readability. 

https://testbucket.s3.us-west-2.amazonaws.com/sample2.txt?
X-Amz-Algorithm=AWS4-HMAC-SHA256&
X-Amz-Credential=REDACTED/20240130/us-west-2/s3/aws4_request&
X-Amz-Date=20240130T190301Z&
X-Amz-Expires=900&
X-Amz-SignedHeaders=host&
x-id=GetObject&
X-Amz-Signature=REDACTED
*/

Can you share your complete code snippet including imports and client creation?

I can't think of a reason why this would get signed. Is there any other information you can provide us with about your application?

Thanks, Ran~
Did you use the same sdk version(1.24.1) as the author,I encountered the same problem.

[admininfon]

Signature logic:

/vendor/github.com/aws/aws-sdk-go-v2/aws/signer/v4/v4.go:377(signedRequest, err := signer.Build())
/vendor/github.com/aws/aws-sdk-go-v2/aws/signer/v4/v4.go:[170-185](s.buildCanonicalHeaders)

It should allow us to selectively filter the header data when the signature is generated.

I have the same problem, but solved it.
It should be a reference to multiple version conflicts. I deleted the v2 SDK and reloaded it, and the test call passed.

Shell:

s.1
> ls -al | grep aws-sdk-go-v2 | xargs sudo rm -r

s.2
> go mod download && go mod tidy && go mod verify && go mod vendor

Go Mod:

go 1.19

require (
	github.com/aws/aws-sdk-go-v2 v1.24.1
	github.com/aws/aws-sdk-go-v2/config v1.26.6
	github.com/aws/aws-sdk-go-v2/credentials v1.16.16
	github.com/aws/aws-sdk-go-v2/service/appconfig v1.25.3
	github.com/aws/aws-sdk-go-v2/service/iotdataplane v1.19.5
	github.com/aws/aws-sdk-go-v2/service/lambda v1.49.7
	github.com/aws/aws-sdk-go-v2/service/s3 v1.48.1
)

[lucix-aws]

This appears to be a cross-module compatibility issue.

As always ensure all modules under the SDKv2 umbrella are up-to-date:

go get -u "github.com/aws/aws-sdk-go-v2/..."