S3 PUT with V4 headers don't work

[jpswinski]

When constructing the HTTP request for S3 using the V4 headers, the request is rejected with:

"The AWS Access Key Id you provided does not exist in our records."

It appears this is related to not providing a session token correctly in the request when using temporary credentials. The AWS documentation is not specific on how to use the session token: https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html