Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ Releases before 2.8.1 predate these markers.
* 💥 Reading or stripping a heap dump with 8 byte identifiers that holds a heap dump info record failed with *"Unknown tag 0x00"*. That record holds an `Int` heap id and then a string id, and both the reader skipping it and the stripper copying it over treated the heap id as an id too, so with 8 byte identifiers they moved 4 bytes too far and read the middle of whatever came next as a record tag. Heap dump info records only appear in heap dumps written by Android, which always uses 4 byte identifiers — where the two sizes are the same and the bug can't show — so this is about a heap dump written by `HprofWriter`.
* ✨ `shark-cli strip-hprof` and `HprofPrimitiveArrayStripper.stripPrimitiveArrays(File)` now handle gzip on both ends: a heap dump whose content is gzipped is read gzipped whatever it's named, and the output is written gzipped when its name ends with ".gz", which is what the default output name of a ".hprof.gz" input already gave you — so "app.hprof.gz" strips to a gzipped "app-stripped.hprof.gz". The Android heap dumps in our test resources compress 3.9x to 4.4x, and 5.6x to 5.8x once stripped, so a heap dump that's been shared is usually gzipped by the time you get it, and stripping it meant gunzipping it first and gzipping the result again.
* ✨ `StreamingSourceProvider.gunzipIfGzipped()` and `StreamingSinkProvider.gzip()` are what do that. They compose onto any source or sink, so a caller of the `stripPrimitiveArrays` overload that takes a source and a sink can opt into the same behavior, or into only one half of it.
* 🔨 `HprofPrimitiveArrayStripper` allocated a boxed `Long` for every instance in the heap dump. Whether an instance wraps a primitive is a lookup by class id, and that lookup went to a `Map<Long, …>`, so hashing the key boxed the id of every instance dump, `LOAD_CLASS` and `CLASS_DUMP` record: 24 bytes each, which is 27.8 MB of the 33.9 MB stripping a 294 MB heap dump of 1154587 instances allocated, and 431 MB of the 437 MB on a 1.4 GB one of 17961453 instances. There are only ever 8 primitive wrapper classes, so the class ids found so far are now held in a `LongArray` and scanned rather than hashed, and what stripping allocates no longer grows with the heap dump: 5.8 MB whether the heap dump is 25 MB, 294 MB or 1.4 GB, where it used to be 9.8 MB, 33.9 MB and 437 MB. Wall clock is unchanged — the boxes were cheap to allocate and died immediately — so this is about the garbage collector, and it matters most on Android, where `HeapAnalysisConfig(stripHeapDump = true)` strips inside the app whose heap was just dumped.
* ✨ `HprofPrimitiveArrayStripper` and `shark-cli strip-hprof` now say what stripping leaves behind. Everything that isn't a primitive array or a wrapped primitive is copied over unchanged, and that includes the string records holding the class, field and method names the rest of the heap dump refers to. Those hold no runtime data in a heap dump from Android, but a heap dump from a JVM also holds every string constant of every loaded class in them, so stripping a JVM heap dump leaves the constants written in the code behind.

## Version 3.0 Alpha 9 (2026-06-25)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package shark
import java.io.File
import okio.Buffer
import org.assertj.core.api.Assertions.assertThat
import org.assertj.core.api.Assertions.assertThatThrownBy
import org.junit.Rule
import org.junit.Test
import org.junit.rules.TemporaryFolder
Expand Down Expand Up @@ -142,6 +143,45 @@ class HprofPrimitiveArrayStripperTest {
assertThat(strippedSecretListArray).isEqualTo(arrayOfZeros)
}

@Test
fun `heap dump naming a primitive wrapper class twice fails loudly`() {
val wrapperClassNames =
listOf(
"java.lang.Boolean",
"java.lang.Character",
"java.lang.Float",
"java.lang.Double",
"java.lang.Byte",
"java.lang.Short",
"java.lang.Integer",
"java.lang.Long",
)
val sourceByteArray =
Buffer()
.apply {
HprofWriter.openWriterFor(
this,
hprofHeader = HprofHeader(version = ANDROID, identifierByteSize = 4),
)
.use { writer ->
// One string record per primitive wrapper class, then one naming the last of them
// again: one id more than there are wrapper classes for the stripper to hold.
(wrapperClassNames + wrapperClassNames.last()).forEachIndexed { index, className ->
writer.write(StringRecord(id = index + 1L, string = className))
}
}
}
.readByteArray()

assertThatThrownBy {
HprofPrimitiveArrayStripper()
.stripPrimitiveArrays(ByteArraySourceProvider(sourceByteArray), { Buffer() })
}
.isInstanceOf(IllegalStateException::class.java)
.hasMessageContaining("Found 9 ids for the primitive wrapper classes")
.hasMessageContaining("at most 8 of them to find")
}

private fun Buffer.writeRawTestHprof(
secretLongArray: LongArray,
secretCharArray: CharArray,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -171,8 +171,8 @@ class HprofPrimitiveArrayStripper {
// Local ref optimizations
val intByteSize = INT.byteSize

val primitiveWrapperClassesByNameStringId = mutableMapOf<Long, PrimitiveWrapperClass>()
val primitiveWrapperClassesByClassId = mutableMapOf<Long, PrimitiveWrapperClass>()
val primitiveWrapperClassesByNameStringId = PrimitiveWrapperClassesById()
val primitiveWrapperClassesByClassId = PrimitiveWrapperClassesById()
var startedReadingHeapDump = false

// Arrays are replaced by repeating one of these over their content, so that replacing an array
Expand Down Expand Up @@ -437,6 +437,47 @@ class HprofPrimitiveArrayStripper {
val className: String,
val valueType: PrimitiveType
)

/**
* The primitive wrapper classes found so far, looked up by an id read from the heap dump.
*
* Scanning an array of ids instead of hashing them matters here: looking a class id up in a [Map]
* keyed by [Long] boxes the class id of every instance in the heap dump, which is hundreds of
* megabytes of garbage on a large one. There's one primitive wrapper class per [PrimitiveType],
* so a heap dump holds 8 ids to scan and they fit in a single cache line.
*/
private class PrimitiveWrapperClassesById {
private val ids = LongArray(PrimitiveType.values().size)
private val wrapperClasses = arrayOfNulls<PrimitiveWrapperClass>(ids.size)
private var size = 0

operator fun set(
id: Long,
wrapperClass: PrimitiveWrapperClass
) {
check(size < ids.size) {
"Found ${size + 1} ids for the primitive wrapper classes, the last of them $id for " +
"${wrapperClass.className}, when there is one wrapper class per primitive type and " +
"therefore at most ${ids.size} of them to find. Getting here takes a heap dump that " +
"holds the name of a wrapper class in two string records, or that loads one of those " +
"classes twice, and no runtime writes either: they dedupe the strings they dump, and " +
"these classes are loaded by the bootstrap class loader, once. Please report this heap " +
"dump to https://github.com/square/leakcanary/issues"
}
ids[size] = id
wrapperClasses[size] = wrapperClass
size++
}

operator fun get(id: Long): PrimitiveWrapperClass? {
for (index in 0 until size) {
if (ids[index] == id) {
return wrapperClasses[index]
}
}
return null
}
}
}

/**
Expand Down