Handle Amazon header formatting so pre-signed S3 PUT URLs with metadata result in valid signatures

[ryanmab]

Description

Closes #2043.

Pre-signed S3 PUT requests (with user-defined metadata) are rejected by S3 because the signature is not valid.

This was caused by a change in the formatting of the user-defined metadata keys.

This fixes the issue by preserving the casing of S3 user-defined metadata - I've validated on my side that S3 PUT requests are now accepted.

Can we apply the case change to X-Amz-Meta- still, if the signature also includes the case change?

I tried uniformly applying formatAmazonHeader to all x-amz values in the buildCanonicalRequest too (so everything formatted as X-Amz-<something> and X-Amz-Meta-<something>), thinking it was just because the canonical request was still in lowercase and that effected the signature. However that didn't appear to make a difference - the signatures were still rejected by S3.

I haven't looked into the official AWS SDK, but my assumption from the docs is its an expectation that the user-defined metadata is all lowercase.

[ryanmab]

This is the change.

[jderusse]

thank you for this PR, I'm currently AFK, I'll have a deeper look latter.

I believe all of these issues have the same origin: We convert header's key to lowercase.. then we tried to revert the process to convert to camel case.
IMHO this is wrong. If we keep the original case all of these issues will disappear.

[ryanmab]

thank you for this PR, I'm currently AFK, I'll have a deeper look latter.

I believe all of these issues have the same origin: We convert header's key to lowercase.. then we tried to revert the process to convert to camel case. IMHO this is wrong. If we keep the original case all of these issues will disappear.

@jderusse No worries at all, just whenever you have a free moment.

Agree on the point about preserving the original case - I don't necessarily know how involved that change is at the minute, but I'd be happy to pitch in in the evenings, and help with a follow up PR to do that!

[ryanmab]

@jderusse @GrahamCampbell apologies for the chaser - I’m sure you’re super busy. I’m just keen to get this merged and pre-signing back working when we can!

Anything you need from me on this for now?