[Bug]: OOM from pending-event storage — full-map re-serialization per event + runBlocking writes (reproduced on 40.1.1 and 43.0.0)

[dlkreamandroid]

Braze Android SDK Version

40.1.1 (also reproduced on 43.0.0)

Steps To Reproduce

1. Log custom events faster than dispatch drains them — e.g. ~10-15 events/sec while scrolling a feed with impression tracking. Dispatch drains at most 32 events per request (sustained ~1 request/30s under the server-delivered rate limit), so the pending-event map grows without bound (verified past 2MB / 8,000+ events — no cap observed). It persists across sessions.
2. Keep logging once the map is ~1.5MB+: the heap fills with dozens of concurrent full-copy serializations of the map (each a map-sized char[]), each pinned by a parked runBlocking worker on the shared coroutine pool.
3. Restart the app with a large persisted map: the heap reaches ~106MB within 44 seconds of launch from the same mechanism.

Why this happens (verified via bytecode of 40.1.1 and 43.0.0):

• All pending events are stored under a single DataStore preferences key as one serialized JSON map ({uuid: eventJson}).
• Every logCustomEvent triggers: read whole map → add one event → re-serialize the ENTIRE map → commit via runBlocking (DataStoreProvider.write$android_sdk_base_release).

Expected Behavior

Logging an event should have O(event) cost: appending one event should not re-serialize the entire pending store, should not block shared pool threads with runBlocking, and heap usage should stay flat regardless of how many events are queued. Per-event keys or append-only storage with non-blocking commits would achieve this — similar in spirit to the 43.0.0 fix for synchronous disk I/O in Card.logImpression().

Actual Incorrect Behavior

Measured on 40.1.1: with a 1.9MB pending map, Dalvik PSS climbs to ~311MB with 71 blocked DefaultDispatcher-worker threads; ~67% of a 400MB heap dump is dozens of concurrent full copies of the map, each pinned by a parked runBlocking worker. Production apps at the 512MB largeHeap limit crash with OutOfMemoryError — worst right after launch, since the persisted map immediately re-enters the write path. UI throughput also collapses (severe jank / ANRs) because the shared Default/IO pool is starved by blocked workers.

Reproduced on 43.0.0: the pending map grew linearly without bound (16B → 1.7MB, 8,000+ events); once past ~1.5MB, PSS jumped from ~70MB to 280–315MB with 68 blocked workers, and a 30-second scroll loop took 15+ minutes to complete.

Additional note: 43.0.0 silently discarded pending events written by 40.1.1 (2,346 events dropped at first load, only 47 dispatched) — apps upgrading with a backlog lose queued analytics.

We can share heap-dump statistics on request.

Verbose Logs

V Braze .com.braze.storage.DataStoreProvider: Writing data:{"<uuid>":"{\"name\":\"ce\",...}", ...entire pending map re-serialized on every single event write...}
I Braze .h: Max number of events per dispatch reached: 32 . No more events will be included in this dispatch
D Braze .j: Adding event to storage with uid <uuid>
D Braze .j: Deleting event from storage with uid <uuid>

Additional Information

Environment: Android app with largeHeap (512MB limit), minSdk 28, Kotlin 2.4.10. Reproduced on an Android 14 emulator and observed across our production fleet (OOM rate roughly doubled after our event volume increased).

How we measured: pending-map file size sampled via adb, heap composition via hprof analysis (two dumps: 160MB at launch+44s, 412MB at peak), thread census via ps -T. Happy to share the histogram numbers, retained-size breakdowns, or reproduction scripts.

[radixdev]

1. Can you share verbose logs of a fresh repro?
2. Can you provide any configuration set via BrazeConfig and/or braze.xml?
3. Are you calling requestImmediateDataFlush during your feed scrolling? If so, how often?
4. Are you debouncing your impression events during your feed scroll?

[dlkreamandroid]

Thanks for the quick response!

1. Verbose logs
Attached a sanitized verbose capture (API key, marketing copy and deep links
redacted; structure/uuids/timestamps preserved). Context: as an emergency
mitigation we recently stopped routing these high-volume events to Braze via our
server-side event config, so this capture is from a diagnostic build of our app
that restores the original routing — event volume is identical to what
production had. The pending map was ~600 KB at capture. The log shows the
per-event pattern: Adding event to storage with uid … followed by
DataStoreProvider: Writing data:{…} where the value is the ENTIRE pending map
(logcat truncates the line, but each write logs/serializes the whole map), plus
Max number of events per dispatch reached: 32 on the drain side.
During a 90-second scroll session with the map at ~600 KB, Dalvik PSS reached
437 MB on this build.

2. Configuration
No braze.xml — everything is set via BrazeConfig.Builder:
setApiKey(…), setCustomEndpoint(…),
setIsFirebaseCloudMessagingRegistrationEnabled(true),
setFirebaseCloudMessagingSenderIdKey(…),
setHandlePushDeepLinksAutomatically(true),
setGreatNetworkDataFlushInterval(10) (value at the time of the report),
setSmallNotificationIcon(…), setDefaultNotificationAccentColor(…).

We've since shipped great=5s / good=10s flush intervals as a mitigation. That
drains a persisted backlog ~10x faster after launch (thanks to the rate-limit
burst capacity of 30 requests), but sustained drain is still ~32 events/30s due
to the server-delivered global request rate limit (capacity 30, refill ~1
request/30s), so it can't prevent accumulation during active use.

3. requestImmediateDataFlush
No — we never call it anywhere in the app. All flushing is the SDK's automatic
sync policy.

4. Debouncing
Yes, per viewport visit: an item logs once when it crosses the visibility
threshold and can't log again until it fully leaves the viewport. No additional
time-based debounce. The 7–16 events/sec peak comes from content-dense feeds
(many items crossing the threshold per second while scrolling), not repeated
fires of a single item. We're also tightening thresholds/dwell to reduce volume
— but the core issue stands: per-event logging cost currently scales with the
total size of the pending store (full-map re-serialization + runBlocking
commit), so any sustained rate above the drain rate degrades the app
superlinearly. Append-per-event storage would make the cost O(event).