Skip to content

Commit 17dea01

Browse files
imikejacksonclaude
andcommitted
PERF: Enlarge zlib read buffer in NIfTI reader for NAS performance
The NIfTI reader opened files with a bare gzopen(), leaving zlib at its 8 KB default internal buffer. zlib refills from the underlying file in buffer-sized chunks, so a multi-GB .nii.gz triggered hundreds of thousands of small reads. On network attached storage each refill is a latency-bound round-trip, making reads disproportionately slow compared to a local disk. Call gzbuffer() with a 4 MiB buffer (new shared constant nifti::k_GzReadBufferSize) immediately after every gzopen(), before the first read/seek. This cuts the number of NAS round-trips by ~512x on a 2 GB file while costing only a few MiB of working memory. Larger buffers yield diminishing returns once they exceed the link's bandwidth-delay product and client read-ahead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent b2f17cc commit 17dea01

3 files changed

Lines changed: 28 additions & 0 deletions

File tree

src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ReadNIfTIFile.cpp

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -378,6 +378,12 @@ Result<> ReadNIfTIFile::operator()()
378378
return MakeErrorResult(-34720, fmt::format("Could not open NIfTI file for reading: '{}'", m_InputValues->InputFilePath.string()));
379379
}
380380

381+
// Enlarge zlib's internal buffer before the first read/seek so the file is
382+
// refilled in large chunks instead of zlib's 8 KB default. This drastically
383+
// reduces the number of latency-bound network round-trips when the input
384+
// lives on a NAS. See nifti::k_GzReadBufferSize for the rationale.
385+
gzbuffer(gz, static_cast<unsigned int>(nx::core::nifti::k_GzReadBufferSize));
386+
381387
const z_off_t targetOffset = static_cast<z_off_t>(md.voxOffset);
382388
if(gzseek(gz, targetOffset, SEEK_SET) != targetOffset)
383389
{

src/Plugins/SimplnxCore/src/SimplnxCore/utils/NiftiUtilities.cpp

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,11 @@ Result<NiftiMetadata> ReadNiftiHeader(const std::filesystem::path& filePath, boo
119119
return MakeErrorResult<NiftiMetadata>(-34700, fmt::format("Could not open NIfTI file for reading: '{}'", pathStr));
120120
}
121121

122+
// Match the voxel reader's enlarged zlib buffer so the (single) header read
123+
// is served by one large refill rather than zlib's 8 KB default. Harmless
124+
// here since only 348 bytes are read, but keeps the two open sites consistent.
125+
gzbuffer(gz, static_cast<unsigned int>(k_GzReadBufferSize));
126+
122127
nifti_1_header hdr{};
123128
int bytesRead = gzread(gz, &hdr, static_cast<unsigned int>(k_HeaderSize));
124129
gzclose(gz);

src/Plugins/SimplnxCore/src/SimplnxCore/utils/NiftiUtilities.hpp

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,23 @@ inline constexpr usize k_HeaderSize = 348;
4646
*/
4747
inline constexpr usize k_MinVoxOffset = 352;
4848

49+
/**
50+
* @brief Size (in bytes) of zlib's internal input/output buffer, set via
51+
* `gzbuffer()` immediately after every `gzopen()`.
52+
*
53+
* zlib defaults to an 8 KB internal buffer, which means it refills from
54+
* the underlying file in 8 KB chunks of compressed data — one `read()`
55+
* syscall each. On a local disk that is negligible, but on network
56+
* attached storage (NAS) every refill is a separate network round-trip,
57+
* so a multi-GB `.nii.gz` triggers hundreds of thousands of small,
58+
* latency-bound reads. Raising the buffer to 4 MiB cuts the number of
59+
* round-trips by ~512x while costing only a few × this size in working
60+
* memory (negligible next to the voxel volume). Larger values yield
61+
* diminishing returns once the buffer exceeds the link's
62+
* bandwidth-delay product and the client's own read-ahead.
63+
*/
64+
inline constexpr usize k_GzReadBufferSize = 1u << 22; // 4 MiB
65+
4966
/**
5067
* @brief Magic bytes that identify the single-file `.nii` / `.nii.gz`
5168
* format. Trailing null is included per the spec.

0 commit comments

Comments
 (0)