Requests SHOULD NOT contain Content-Length: 0 for GET/HEAD/DELETE requests

[joeshaw]

Describe the bug

(Originally filed as awslabs/mountpoint-s3#1381)

I work on an S3-compatible product and we've seen compatibility issues with s3-mountpoint as a result of it sending Content-Length: 0 and including it as a signed header on HTTP requests that don't require it. Our server platform strips Content-Length for requests using an HTTP method where a request body is not expected, and subsequently our signature verification code fails because it isn't present when we calculate the signature. We've worked around it but this is a common source of HTTP client/server compatibility issues generally and I thought you might be interested in changing it for GET, HEAD, and DELETE requests.

The current HTTP RFC is RFC 9110. In Section 8.6, it says:

A user agent SHOULD send Content-Length in a request when the method defines a meaning for enclosed content and it is not sending Transfer-Encoding. For example, a user agent normally sends Content-Length in a POST request even when the value is 0 (indicating empty content). A user agent SHOULD NOT send a Content-Length header field when the request message does not contain content and the method semantics do not anticipate such data.

(emphasis mine)

Under Section 9.3 where it goes over the individual methods, for GET, HEAD and DELETE it says:

A client SHOULD NOT generate content in a GET [or HEAD or DELETE] request unless it is made directly to an origin server that has previously indicated, in or out of band, that such a request has a purpose and will be adequately supported.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

No Content-Length: 0 header sent for GET/HEAD/DELETE requests.

Current Behavior

Content-Length is sent on all requests, including Content-Length: 0 for GET/HEAD/DELETE.

Reproduction Steps

This was bought to us as an incompatibility between mountpoint-s3 and Fastly Object Storage from one of our customers.

We've worked around the issue on our end, so it no longer produces an error and can't be easily reproduced, but before that was deployed you would get a signature mismatch error.

If you are able to inspect the requests that are sent, you'll see Content-Length: 0 being provided for at least GET and DELETE requests. I am not sure about HEAD.

Possible Solution

No response

Additional Information/Context

No response

aws-c-s3 version used

Git rev 408e9c9, as imported into mountpoint-s3

Compiler and version used

N/A

Operating System and version

N/A

[TingDaoK]

Can you be more specific to what request you saw the client adding the content-length header?

I don't think we add content-length header to any of those type you mentioned.
For GET/HEAD, we do copy every header from the original request, here. However, I'd expect the original request should be responsible for the header.

For DELETE, the only request we sent with DELETE is the abort MPU, which we explicitly exclude content-length here

Can you double check which level adds the content-length:0 header? I'd assume it's some high level application who adds the header before it's passed down to the S3 client.

[TingDaoK]

aws-c-s3/source/s3_default_meta_request.c

Line 350 in f14b84d

aws_s3_message_util_assign_body(

Looks like we always try to assign the body for default type of meta request, which will add the content length to the request.

[TingDaoK]

The issue should be fixed with latest release