Guard JSON serialization against json 3.0 encoding errors - #3075
Draft
sentry-junior[bot] wants to merge 5 commits into
Draft
Guard JSON serialization against json 3.0 encoding errors#3075sentry-junior[bot] wants to merge 5 commits into
sentry-junior[bot] wants to merge 5 commits into
Conversation
Member
|
I don't want the deep encode, we should clean up possible encoding issues wherever we fill up the data |
Member
|
rebase latest master |
sl0thentr0py
force-pushed
the
fix/json-3-encoding
branch
2 times, most recently
from
September 7, 2026 12:44
dea181c to
c40f83d
Compare
json 3.0 turns the json 2.8+ deprecation warning (UTF-8 string passed as BINARY) into a hard raise (Encoding::UndefinedConversionError) when JSON.generate/JSON.dump encounters a String tagged with a non-UTF-8 encoding that contains invalid bytes for the target encoding. Add Utils::EncodingHelper.deep_encode_utf_8 to recursively sanitize Hash/Array/String payloads, use it to proactively protect the JSON generation call sites that were not already guarded: - Envelope::Item#to_s / #lost_event_byte_size / #size_breakdown - Transport#serialize_envelope (envelope headers) - Event#to_json_compatible - TelemetryEventBuffer#add_item / #send_items - DebugTransport#log_envelope, DebugStructuredLogger#capture_log_event - Breadcrumb#serialized_data, Utils::TelemetryAttributes#attribute_hash Also add a last-resort rescue for EncodingError/JSON::GeneratorError in Transport#send_envelope so a single unrecoverable event is recorded as a lost event (like SizeExceededError) instead of crashing the background worker. Fixes #2462 Co-Authored-By: Neel Shah <6536764+sl0thentr0py@users.noreply.github.com>
Transport::Configuration has no #debug attribute; the debug flag lives on the top-level Configuration. Store it on the Transport instance (mirroring how @sdk_logger/@dsn are cached) and use that instead. Caught by transport_spec.rb's new encoding-error test.
… retries Per review feedback: drop the reactive 'catch JSON.generate, deep-encode, and retry' pattern from serialization exit points (Envelope::Item, Transport#serialize_envelope headers, Event#to_json_compatible, TelemetryEventBuffer, DebugTransport, DebugStructuredLogger). Those are reverted to plain JSON.generate/JSON.dump. Instead, sanitize encoding proactively wherever the affected data actually gets filled in: - Breadcrumb#data= now force-encodes/scrubs strings in the arbitrary user-supplied data hash to valid UTF-8 (both on construction and on later assignment), matching the existing pattern used for Breadcrumb#message=, SingleException#value, and RequestInterface's body/headers. - Utils::TelemetryAttributes#attribute_hash now sanitizes String attribute values directly, and sanitizes non-scalar (Hash/Array) attribute values before JSON.generate rather than reacting to a raised error. Transport#send_envelope keeps its rescue for EncodingError/ JSON::GeneratorError as a last-resort safety net (log + record lost event) for any case not already covered by the above, without attempting to fix and retry. Updated specs accordingly.
sl0thentr0py
force-pushed
the
fix/json-3-encoding
branch
from
September 7, 2026 12:44
c40f83d to
d6dc3c8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
json 3.0.0 (released today) turns the json 2.8+ deprecation warning ("UTF-8 string passed as BINARY, this will raise an encoding error in json 3.0") into a hard
Encoding::UndefinedConversionErrorraise wheneverJSON.generate/JSON.dumpencounters a String tagged with a non-UTF-8 encoding (e.g.ASCII-8BIT/BINARY) containing bytes invalid for the target encoding. This matches the stack trace in #2462, which points straight atenvelope/item.rb#to_s→#serialize→transport.rb#serialize_envelope.Places identified and protected
Unguarded
JSON.generate/JSON.dumpcall sites that could receive attacker/user-controlled strings with unexpected encodings:Envelope::Item#to_s,#lost_event_byte_size,#size_breakdown(the exact reported crash path)Transport#serialize_envelope(envelope headers)Event#to_json_compatibleTelemetryEventBuffer#add_item,#send_itemsTransport::DebugTransport#log_envelope,DebugStructuredLogger#capture_log_event(debug-only, still hardened)Breadcrumb#serialized_data,Utils::TelemetryAttributes#attribute_hash(already rescued broadly, but previously discarded the whole payload instead of sanitizing it)Fix
Sentry::Utils::EncodingHelper.deep_encode_utf_8, which recursively walks Hash/Array/String structures and force-encodes/scrubs any String to valid UTF-8.deep_encode_utf_8-sanitized input ifJSON.generate/JSON.dumpraisesEncodingError/JSON::GeneratorError, so valid data is preserved instead of being dropped.rescue EncodingError, JSON::GeneratorErrorinTransport#send_envelope, mirroring the existingSizeExceededErrorhandling, so if per-item sanitization still can't produce valid JSON, the event is recorded as a lost event (:send_error) instead of crashing the background worker.Fixes #2462
Testing
spec/sentry/utils/encoding_helper_spec.rbfordeep_encode_utf_8.envelope/item_spec.rb,breadcrumb_spec.rb, andtransport_spec.rbthat stubJSON.generateto raiseEncodingError(simulating json 3.0 behavior regardless of the json gem version actually loaded in CI) and assert recovery/no-crash behavior.Requested by neel.
--
View Junior Session [Sentry]