Skip to content

fix: parse charset when Content-Type has whitespace after ';' - #2130

Open
jaideeppyne wants to merge 1 commit into
spec-first:mainfrom
jaideeppyne:fix/split-content-type-charset
Open

fix: parse charset when Content-Type has whitespace after ';'#2130
jaideeppyne wants to merge 1 commit into
spec-first:mainfrom
jaideeppyne:fix/split-content-type-charset

Conversation

@jaideeppyne

Copy link
Copy Markdown

Fixes # .

split_content_type() failed to pick up the charset when the Content-Type header has a space after the ;, e.g. application/json; charset=utf-8. It matched parameters with startswith("charset=") on the raw semicolon-split parts, without trimming the optional whitespace that RFC 7231 allows. So the common spaced form returned encoding=None, and the request/response validators silently fell back to utf-8 and mis-decoded bodies sent in another charset.

Reproduction on current main:

'application/json;charset=utf-8'   -> ('application/json', 'utf-8')   # worked
'application/json; charset=utf-8'  -> ('application/json', None)      # bug (expected 'utf-8')
'text/html; charset=UTF-8'         -> ('text/html', None)             # bug

Changes proposed in this pull request:

  • Strip optional whitespace around the media type and each parameter in split_content_type before matching the charset parameter.
  • Add a regression test covering the no-parameter, unspaced-charset, spaced-charset, and charset-after-another-parameter cases.

split_content_type() looked for the charset parameter with
str.startswith('charset='), but iterated over the raw semicolon-split
parts without trimming the optional whitespace allowed by RFC 7231.
As a result the common 'application/json; charset=utf-8' form (space
after the separator) failed to detect the encoding and callers silently
fell back to utf-8, mis-decoding request/response bodies sent in another
charset. Strip surrounding whitespace from the media type and each
parameter before matching.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant