Fix append_query_params corrupting values with percent sequences#229
Open
phani69015 wants to merge 1 commit into
Open
Fix append_query_params corrupting values with percent sequences#229phani69015 wants to merge 1 commit into
phani69015 wants to merge 1 commit into
Conversation
append_query_params called unquote() on every value before re-encoding, which assumed all values were already percent-encoded. Callers pass raw values, so this silently decoded them: - A file literally named 'a%2Bb.csv' was decoded to 'a+b.csv', colliding with a genuinely different file named 'a+b.csv' and producing the same download URL. Any value containing a percent sequence was corrupted. - Values were encoded with the default safe set, so '=' inside values was turned into '%3D', changing 'downloadRange=bytes=0-9' into 'bytes%3D0-9' on the range-download path. Values are now treated as raw strings and percent-encoded exactly once, and '=' is kept in the safe set so range requests are transmitted as the API expects. The URL-encoding behaviour was introduced in v4.0.3 (jpmorganchase#216). Add regression tests for percent-sequence values, '+' vs '%2B' distinctness, '=' preservation in downloadRange, and query-separator encoding.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
append_query_params called unquote() on every value before re-encoding, which assumed all values were already percent-encoded. Callers pass raw values, so this silently decoded them:
Values are now treated as raw strings and percent-encoded exactly once, and '=' is kept in the safe set so range requests are transmitted as the API expects. The URL-encoding behaviour was introduced in v4.0.3 (#216).
Add regression tests for percent-sequence values, '+' vs '%2B' distinctness, '=' preservation in downloadRange, and query-separator encoding.