Strip heap dumps that Android wrote, arrays too big for an Int, and gzipped ones

[pyricau]

Five things found while comparing HprofPrimitiveArrayStripper against
hprof-redact, one commit each.

Wrapped primitives were left alone on Android

Stripping is meant to zero the value every primitive wrapper wraps, on top of zeroing primitive
arrays. It learned where that value sits from the CLASS_DUMP record of the wrapper class, so it
could only zero the instances dumped after it. HotSpot writes every class dump ahead of any instance,
which is why the test covering this — built from a JVM heap dump — passed. ART walks the heap in
memory order, so most Integer instances come before the class dump of java.lang.Integer:

heap dump	wrapper instances	kept their value
leak_asynctask_o.hprof	1291	1175 (91%)
leak_asynctask_m.hprof	3365	3289 (98%)

Nothing about a wrapper needs learning from the class dump: it declares exactly one instance field,
and a class declares its own fields ahead of the ones it inherits, so the value is at the start of the
instance field values. The class id now comes from LOAD_CLASS, which both ART and HotSpot write
before the heap dump starts, and the class dump is read to check that layout instead of to derive
it. Two new tests: one against leak_asynctask_o.hprof asserting every wrapper wraps zero, one
writing the wrapper's class dump after its instances, both verified to fail without the fix.

Writing over an array allocated a copy of it

The replacement bytes were built as a byte array the size of the array's content, one per array.
That's the remaining half of #2777: sized in Int arithmetic, so a long[268435456] asked for an
array of -2147483648 bytes and stripping died with a NegativeArraySizeException. Three patterns are
now built per heap dump and repeated over each array's content instead — a constant 24 KB, whatever
the heap dump holds.

	before	after
2.15 GB JVM heap dump with a long[268435456]	NegativeArraySizeException	strips in 3 s
smallest heap for a 110 MB dump with a byte[104857600]	216 MB	8 MB
bytes allocated stripping a 294 MB dump from a real app	374 MB	34 MB
time for that same heap dump	~480 ms	~450 ms

Output is byte for byte identical on the heap dumps in our test resources and on that 294 MB one.
CopyingSource.overwriteRepeating is the new public method; overwrite is untouched.

Heap dump info records were skipped by the wrong size

HEAP_DUMP_INFO holds an Int heap id and then a string id. HprofRecordReader.readHeapDumpInfoRecord
reads it that way and HprofWriter writes it that way, but the skip in HprofRecordReader and the
copy in the stripper both counted the heap id as an object id. With the 4 byte identifiers Android
always uses those are the same size, which is why nothing noticed; with 8 byte identifiers both move
4 bytes too far and read the middle of the next sub-record as a tag, failing with "Unknown tag
0x00".

Transparent gzip

A heap dump that's been shared has almost always been gzipped on the way, so stripping one meant
gunzipping it and gzipping the result by hand. strip-hprof and the file overload now read gzipped
content gzipped — decided from the first two bytes, not from the name — and write gzipped output when
the output name ends with .gz, which is what the default output name of a .hprof.gz input already
gave you. StreamingSourceProvider.gunzipIfGzipped() and StreamingSinkProvider.gzip() are what do
it and compose onto anything.

The Android heap dumps in our test resources compress 3.9x to 4.4x, and 5.6x to 5.8x once stripped.

Saying what stripping doesn't cover

Stripping replaces primitive arrays and wrapped primitives and copies everything else over unchanged,
which includes the string records holding class, field and method names. An Android heap dump has
nothing but names in there. HotSpot writes its whole symbol table out, so a JVM heap dump also carries
every string constant of every loaded class — no user data, but the constants written in the code. The
KDoc and the CLI help now say so.

Left for later

A primitive array of more than 2^31 elements still can't be stripped: the length is read as a signed
Int, in the stripper and in HprofRecordReader alike, so it wraps negative. Reaching that needs a
16 GB long array, and fixing it is a change to how both read the length, not to how they size the
skip.

ABI

shark-hprof gains CopyingSource.overwriteRepeating, StreamingSourceProvider.gunzipIfGzipped()
and StreamingSinkProvider.gzip(). Nothing removed, nothing changed.

🤖 Generated with Claude Code