Fix JSON::GeneratorError masking errors for malformed UTF-8 request bodies - #1206
Open
grk wants to merge 1 commit into
Open
Fix JSON::GeneratorError masking errors for malformed UTF-8 request bodies#1206grk wants to merge 1 commit into
grk wants to merge 1 commit into
Conversation
…odies extract_request_data_from_rack serializes the parsed raw body params to JSON while building the :body field, and this happens before the payload-wide UTF-8 normalization pass in Rollbar::Item. When a request body contains invalid UTF-8 (e.g. a malformed request that Rails rejects with ActionController::BadRequest), that serialization raises JSON::GeneratorError, which replaces the original exception being reported with a failsafe report. Normalize the parsed body params with Rollbar::Util.enforce_valid_utf8 in mergeable_raw_body_params, before they are scrubbed and serialized, so the original error is reported correctly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Rollbar::RequestDataExtractor#extract_request_data_from_rackserializes the parsed raw body params to JSON while building the:bodyfield:This serialization happens before the payload-wide UTF-8 normalization pass that
Rollbar::Itemruns (Rollbar::Util.enforce_valid_utf8). When a request body contains invalid UTF-8 — for example a malformed request that Rails rejects withActionController::BadRequest—Rollbar::JSON.dumpraisesJSON::GeneratorError: source sequence is illegal/malformed utf-8.The result is that the original exception being reported gets replaced with a failsafe report, masking the real error (
ActionController::BadRequest) from users.Fix
Normalize the parsed body params with
Rollbar::Util.enforce_valid_utf8insidemergeable_raw_body_params, before they are scrubbed and serialized. This mirrors the existing payload-wide UTF-8 handling and ensures the:bodyfield can always be serialized, so the original error is reported correctly.The change is limited to error-report payload content; request parsing/validation is unaffected.
Test
Added a spec that feeds a JSON
POSTbody with an invalid UTF-8 byte throughextract_request_data_from_rack. Without the fix it raisesJSON::GeneratorError; with the fix it returns a valid, UTF-8:bodystring.