Pre-signing S3 PUT requests with object metadata causing SignatureDoesNotMatch error

[ryanmab]

Hi all!

I've noticed increased error rates from S3 when making a PUT request to a pre-signed URL generated by async-aws/s3, and I think it might've been introduced by the casing changes introduced here: #2037 (specifically, this line).

The error coming back from S3 is:

<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message></Error>

This is only a recent issue, and so I've also diffed a working URL, to one which now triggers the signature error. And, from what I can tell, the only difference is related to the casing of the S3 object's metadata query parameters. Specifically that those in SignedHeaders have different cases in the query parameters in the URL with the invalid signature.

Working (sanitised parameters for brevity):

https://<s3-url>/<object>.xml?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Security-Token=&X-Amz-Date=&X-Amz-Expires=&X-Amz-Credential=&x-amz-meta-owner=&x-amz-meta-repository=&x-amz-meta-provider=&x-amz-meta-filename=&x-amz-meta-projectroot=&x-amz-meta-tag=&x-amz-meta-commit=&x-amz-meta-parent=&x-amz-meta-ref=&x-amz-meta-pullrequest=&x-amz-meta-baseref=&x-amz-meta-basecommit=&x-amz-meta-uploadid=&x-amz-meta-projectid=&x-amz-content-sha256=UNSIGNED-PAYLOAD&X-Amz-SignedHeaders=host%3Bx-amz-meta-basecommit%3Bx-amz-meta-baseref%3Bx-amz-meta-commit%3Bx-amz-meta-filename%3Bx-amz-meta-owner%3Bx-amz-meta-parent%3Bx-amz-meta-projectid%3Bx-amz-meta-projectroot%3Bx-amz-meta-provider%3Bx-amz-meta-pullrequest%3Bx-amz-meta-ref%3Bx-amz-meta-repository%3Bx-amz-meta-tag%3Bx-amz-meta-uploadid&X-Amz-Signature=

Signature error (sanitised parameters for brevity):

https://<s3-url>/<object>.xml?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Security-Token=&X-Amz-Date=&X-Amz-Expires=&X-Amz-Credential=&X-Amz-Meta-Owner=&X-Amz-Meta-Repository=&X-Amz-Meta-Provider=&X-Amz-Meta-Filename=&X-Amz-Meta-Projectroot=&X-Amz-Meta-Tag=&X-Amz-Meta-Commit=&X-Amz-Meta-Parent=&X-Amz-Meta-Ref=&X-Amz-Meta-Pullrequest=&X-Amz-Meta-Baseref=&X-Amz-Meta-Basecommit=&X-Amz-Meta-Uploadid=&X-Amz-Meta-Projectid=&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-SignedHeaders=host%3Bx-amz-meta-basecommit%3Bx-amz-meta-baseref%3Bx-amz-meta-commit%3Bx-amz-meta-filename%3Bx-amz-meta-owner%3Bx-amz-meta-parent%3Bx-amz-meta-projectid%3Bx-amz-meta-projectroot%3Bx-amz-meta-provider%3Bx-amz-meta-pullrequest%3Bx-amz-meta-ref%3Bx-amz-meta-repository%3Bx-amz-meta-tag%3Bx-amz-meta-uploadid&X-Amz-Signature=

My initial instinct is we should match the casing between the SignedHeaders parameter, and the query parameters themselves (meaning we tweak the buildCanonicalHeaders function) - though I haven't tested this theory yet.

If this sounds right though, I'm happy to raise a PR - but do let me know if theres a different suggestion!

Thanks,

Ryan

[ryanmab]

Just an update here - I patched in a revert to the casing locally, and have confirmed the S3 PUT request's signature is validated successfully when the header's casing is left unchanged.

At the same time, I've pushed up PR (which is working if you'd like to merge!), along with a test to confirm change in behaviour.

Details on the PR: #2044