Transfer-Encoding with chunked as the last coding is refused with 400 since v20.70.0

[nigrosimone]

Since v20.70.0 the new check in HttpParser.h compares the whole Transfer-Encoding value against the literal string "chunked":

uWebSockets/src/HttpParser.h

Line 584 in 3ffd6f4

if (transferEncodingString != "chunked") {

    /* We only support chunked */
    if (transferEncodingString != "chunked") {
        return {HTTP_ERROR_400_BAD_REQUEST, FULLPTR};
    }

Two problems with it:

1. Transfer-Encoding: CHUNKED is refused. RFC 9112 section 7 says "All transfer-coding names are case-insensitive", so this one has to be accepted.
2. gzip, chunked gets 400. Refusing it is fine, uWS does not implement gzip as a transfer coding, but RFC 9112 section 6.1 asks for 501 (Not Implemented) here, not 400. A client now reads "your request is malformed" when the real answer is "I do not support this".

v20.69.0 answered 200 to both, and so does node's own http server, so this is also a behaviour change for anyone upgrading.

Repro below, prints one line per case.

// Transfer-Encoding handling, uWS 20.69.0 vs 20.70.0.
const uWS = require("uWebSockets.js");
const net = require("net");

const CASES = ["chunked", "CHUNKED", "gzip, chunked", "deflate, gzip, chunked", "identity, chunked"];

function ask(port, te) {
    return new Promise((resolve) => {
        const s = net.connect(port, "127.0.0.1", () =>
            s.write(`POST / HTTP/1.1\r\nHost: x\r\nTransfer-Encoding: ${te}\r\n\r\n5\r\nhello\r\n0\r\n\r\n`)
        );
        let buf = "";
        s.on("data", (d) => (buf += d));
        s.on("error", () => resolve("connection error"));
        s.on("close", () => resolve(buf.split("\r\n")[0] || "closed with no answer"));
        setTimeout(() => { s.destroy(); resolve(buf.split("\r\n")[0] || "no answer"); }, 800);
    });
}

uWS.App()
    .any("/*", (res) => {
        res.onAborted(() => {});
        res.onData(() => {});
        res.end("ok");
    })
    .listen("127.0.0.1", 0, async (token) => {
        const port = uWS.us_socket_local_port(token);
        for (const te of CASES) {
            console.log(JSON.stringify(te).padEnd(26) + (await ask(port, te)));
        }
        uWS.us_listen_socket_close(token);
    });

                          20.69.0            20.70.0
"chunked"                 200 OK             200 OK
"CHUNKED"                 200 OK             400 Bad Request
"gzip, chunked"           200 OK             400 Bad Request
"deflate, gzip, chunked"  200 OK             400 Bad Request
"identity, chunked"       200 OK             400 Bad Request

[uNetworkingAB]

Sounds like it's mostly a matter of returning 501 rather than 400, because uWS never supported anything other than chunked. So not really a "bug" more like "refinement"

[uNetworkingAB]

Basically:

If "chunked" then 200
Else If ends with "chunked" then 501
Else 400

[uNetworkingAB]

Ah shit, wait. I see the problem. Before we just let the app handle the gzip and so on. Now we just refuse it. Okay.

So the fix should be more like, require it to end with "chunked", else 400. But let the app handle the unzipping.

[uNetworkingAB]

// lowercase transferEncodingString

                /* The app can support any framing including gzip, deflate, but it always needs to end with chunked */
                if (!transferEncodingString.ends_with("chunked")) {
                    return {HTTP_ERROR_400_BAD_REQUEST, FULLPTR};
                }