Skip to content

MRIFiles: Faster ISMRMRD read via single bulk HDF5 call - #291

Open
hakkelt wants to merge 1 commit into
MagneticResonanceImaging:masterfrom
hakkelt:hakkelt/ismrmrd-bulk-read
Open

MRIFiles: Faster ISMRMRD read via single bulk HDF5 call#291
hakkelt wants to merge 1 commit into
MagneticResonanceImaging:masterfrom
hakkelt:hakkelt/ismrmrd-bulk-read

Conversation

@hakkelt

@hakkelt hakkelt commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Summary

  • In the unfiltered case (no slice/repetition/contrast filter), replace the per-profile element reads with a single read(dset) call that loads the entire dataset at once.
  • Remove the now-redundant separate header-only bulk read that preceded the loop.
  • The filtered path is unchanged: it still bulk-reads headers to decide which profiles to keep, then fetches only matching profiles individually.

Why this helps

The ISMRMRD dataset is written with chunkshape = [1] (one HDF5 chunk per profile). The previous code called h["…/data"][m] inside the loop, issuing one HDF5 element read per profile — O(N) separate chunk lookups, each with its own dataspace selection and variable-length heap traversal. On a 186 MB file with 3952 profiles this was the dominant cost.

Benchmark (186 MB OCMR file, 3952 profiles, 3 runs each)

Path Time (min) Allocations Speedup
Original — per-element reads 3.14 s 436 MiB
New — single bulk read 1.26 s 395 MiB 2.5×
Filtered worst case — original 2.71 s 436 MiB
Filtered worst case — new 2.48 s 436 MiB ~1.1×

The speedup grows with profile count; larger cardiac/fMRI raw datasets (10k–200k profiles) will see a proportionally larger improvement.

Correctness

All 3952 profiles match the original read bit-for-bit (data and trajectory arrays) in both the unfiltered and filtered paths.

🤖 Generated with Claude Code

The ISMRMRD dataset uses chunk size 1 (one chunk per profile), so the
previous code issued O(N) separate HDF5 element reads inside the loop.
On a 186 MB file with 3952 profiles this took ~3 s; the bulk read drops
it to ~1.3 s (2.5× faster, ~40 MiB fewer allocations).

Two paths:
- No filter (common case): single `read(dset)` loads everything at once;
  the separate header-only bulk read is no longer needed and is removed.
- Filtered (slice/repetition/contrast): unchanged — bulk-reads headers to
  decide which profiles to keep, then fetches only matching profiles.

Correctness verified: all profiles match the original read bit-for-bit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@hakkelt hakkelt changed the title MRIFiles: 2.5× faster ISMRMRD read via single bulk HDF5 call MRIFiles: Faster ISMRMRD read via single bulk HDF5 call Jun 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant