Skip to content

perf: Improved decoding from InputStreams - #224

Closed
bogovicj wants to merge 5 commits into
developmentfrom
perf/inputstream-limit
Closed

perf: Improved decoding from InputStreams#224
bogovicj wants to merge 5 commits into
developmentfrom
perf/inputstream-limit

Conversation

@bogovicj

Copy link
Copy Markdown
Contributor

This PR improves decoding performance by using the known decoded size of a block in bytes. The primary changes are:

  • Calling limit on the ReadData used in FlatArrayCodec
  • Overriding limit in InputStreamReadData so that it can have an expected size without materializing

We call org.apache.commons.io.IOUtils.toByteArray(...) to materialize InputStreamReadData
of unknown size, and have previously identified that this call is much slower than the alternative path with known size.

An InputStreamReadData can arise during de-coding, for example during ZStandard decompression.

See also #161
@tpietzsch @cmhulbert

@bogovicj bogovicj closed this May 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant