Skip to content

Releases: scigolib/hdf5

v0.13.12

14 Mar 13:11
c055c0c

Choose a tag to compare

Add VLenUint8 Datatype (Issue #30)

Added VLenUint8 (Datatype = 507, Go type [][]byte) for variable-length byte arrays. Completes the VLen type family — all other numeric variants were already supported.

Use case

Storing serialized binary data (protobuf messages, ROS serialized messages) compatible with C++ H5::VarLenType(H5::PredType::STD_U8LE).

Example

ds, err := fw.CreateDataset("/messages", hdf5.VLenUint8, []uint64{3})
ds.Write([][]byte{
    {0x12, 0x34, 0x56},
    {0xAB, 0xCD, 0xEF, 0x00},
    {0x42},
})

Requested by @zhoujun24.

Full Changelog: https://github.com/scigolib/hdf5/blob/main/CHANGELOG.md

v0.13.11

14 Mar 12:14
910cbeb

Choose a tag to compare

Bug Fix: Write Interoperability (Issue #28, #29)

Fixed 3 bugs that caused files with groups + attributes or multiple root-level datasets to be unreadable by h5dump, h5ls, and h5py.

Changes

  • Allocator EOA: update allocator after object header grows from attribute addition, so superblock EOA covers entire file
  • B-tree v1 right key: use lexicographically largest name (strcmp per H5Gnode.c), not numerically largest heap offset
  • SNOD sorting: sort symbol table entries by name after insertion (per H5G__node_insert)
  • ObjectHeaderSizeFromParsed: now supports both v1 and v2 headers
  • Lint: fix pre-existing gosec G115 and unused nolint directives
  • 4 regression tests added

Reported by

@vrv-bit in #28 and #29

Full Changelog: https://github.com/scigolib/hdf5/blob/main/CHANGELOG.md

v0.13.10

06 Mar 11:35
2f92dd0

Choose a tag to compare

What's Changed

Fixed critical interoperability bug — HDF5 files written by the Go library can now be read by h5ls, h5dump, h5py, and h5wasm (#24).

Bug Fixes

  • V2 Object Header: chunk_size now excludes checksum per C reference (H5Ocache.c)
  • Local heap: offset 0 contains required empty string for root group name (H5Gstab.c)
  • Datatype encoding: correct integer and float property layouts per H5Odtype.c
  • V0 allocator: root group space reservation prevents dataset overlap; B-tree right key updated correctly

Other

  • Renamed internal test files to enterprise-quality names
  • Updated CHANGELOG.md and ROADMAP.md

Full Changelog: v0.13.9...v0.13.10

v0.13.9

04 Mar 15:30
7748ffb

Choose a tag to compare

Bug Fix

V2 Object Header Checksum (Issue #24)

Fixed a critical compatibility issue where V2 object headers were written without the mandatory 4-byte Jenkins lookup3 checksum. HDF5 readers (h5py, h5wasm, h5dump) validate this checksum on every object header read, causing incorrect metadata checksum after all read attempts errors.

Changes:

  • Added Jenkins lookup3 checksum computation to V2 object header writing
  • Added support for variable chunk size field width (1/2/4/8 bytes) per HDF5 spec
  • Unified all header size calculations via ObjectHeaderWriter.Size()
  • Fixed CI double-trigger: push events now only fire on main branch
  • Verified against HDF5 C reference implementation (H5Ocache.c)

Impact: Files created with v0.11.0–v0.13.8 have invalid V2 object header checksums. Files created with v0.13.9+ are fully compatible with h5py, h5wasm, h5dump, and the HDF5 C library.

Reported by @vrv-bit. Fixes #24.

v0.13.8

04 Mar 06:35
23809a8

Choose a tag to compare

Bug Fixes

Superblock EOA Compatibility (Issue #22)

Fixed a critical compatibility issue where HDF5 files created by this library could not be read by h5py, h5wasm, or other HDF5 readers.

Root cause: The superblock's End-of-File Address (EOA) was written once at file creation time but never updated as datasets, attributes, and groups were added.

Symptoms:

  • h5py: KeyError: 'Unable to synchronously open object (actual len exceeds EOA)'
  • h5wasm: actual_len exceeds EOA

Fix: FileWriter.Close() now rewrites the superblock with the final EOA before closing.

FileWriter resource leak on Windows

FileWriter.Close() now also closes the read-side file handle opened by OpenForWrite(), preventing file locking errors on Windows.


Impact: Files created with v0.11.0–v0.13.7 may have incorrect EOA. Upgrade to v0.13.8+ for full compatibility with h5py, h5wasm, h5dump, and the HDF5 C library.

Reported by @vrv-bit — thank you!

v0.13.7 — Test Coverage Boost

27 Feb 13:04
5748315

Choose a tag to compare

Test Coverage Boost

Major test coverage improvement across all library packages — ~250 new test functions, +15,000 lines of tests.

Coverage by package

Package Before After Delta
Root (hdf5) 75.8% 82.4% +6.6%
internal/core 76.0% 87.9% +11.9%
internal/structures 77.1% 91.5% +14.4%
internal/writer 88.2%
internal/rebalancing 95.0%
internal/utils 96.2%

Key areas covered

  • Public API: ReadStrings, ReadCompound, loadObject, loadChildren, NamedDatatype
  • Hyperslab: compact, float32/int64, 3D/strided, chunked layouts
  • Write paths: all integer types, string datasets, compound datasets, chunked+filters
  • Dense attributes: fractal heap + B-tree v2 full pipeline
  • Filter pipeline: BZIP2, LZF decompression, filter dispatch
  • Internal structures: incremental/lazy rebalancing, fractal heap ops, B-tree v2

Maintenance

  • Fixed all gosec G602 slice bounds warnings
  • Fixed data race in incremental rebalancer tests
  • 0 lint issues across 34+ linters

v0.13.6

24 Feb 16:18
4a922d7

Choose a tag to compare

What's New

Extended Slice Attribute Types (TASK-040)

All integer slice types now supported for attribute writing:
[]int8, []int16, []uint8, []uint16, []uint32, []uint64

Previously only []int32, []int64, []float32, []float64 were supported. Scalar attributes already supported all sizes — this closes the gap for slices.

Inspired by PR #19 from @CWBudde (MeKo-Christian).

Lint Cleanup

Fixed all 70 golangci-lint issues across 24 files:

  • Removed 39 stale //nolint:gosec directives
  • Added targeted nolint for 19 new gosec warnings
  • Replaced 11 WriteString(Sprintf) with fmt.Fprintf

Documentation Fix

  • Fixed CHANGELOG: ChunkIterator (TASK-031) and compression filters (TASK-027) now correctly listed under v0.13.5
  • Made README.md version-agnostic

Full Changelog: https://github.com/scigolib/hdf5/blob/main/CHANGELOG.md

v0.13.5 - Jenkins Lookup3 Checksum Fix

02 Feb 11:43
01a277d

Choose a tag to compare

v0.13.5 - Jenkins Lookup3 Checksum Fix

Critical Bug Fix

Issue #17: Files created with Superblock V2/V3 could not be opened by h5dump, h5py, or the HDF5 C library due to incorrect checksum algorithm.

What Changed

  • Fixed: Replaced CRC32 IEEE with Jenkins lookup3 hash for all metadata checksums
  • Added: JenkinsChecksum() function - direct port of H5_checksum_lookup3() from HDF5 C library

Affected Components

  • Superblock V2/V3 checksum
  • B-tree v2 header and leaf checksums
  • Fractal heap header checksum
  • Indirect block checksum

Validation

  • Validated against known HDF5 files (aggr.h5 checksum: 0xD5CB91E3)
  • All tests pass (100%)
  • Lint: 0 issues

Upgrade

All users creating files with Superblock V2/V3 should upgrade to ensure compatibility with HDF5 ecosystem tools.

go get github.com/scigolib/hdf5@v0.13.5

Full Changelog: v0.13.4...v0.13.5

v0.13.4 - Variable-length string attribute support

29 Jan 21:24
9063e5d

Choose a tag to compare

What's Changed

Bug Fixes

  • Fixed variable-length string attribute reading (Issue #14)
    • V1/V2 attribute message 8-byte alignment (H5O_ALIGN_OLD macro)
    • IsVariableString() now correctly checks ClassBitField per HDF5 Format Spec III.A.2.4.d
    • VLen string format: 4-byte length prefix + Global Heap reference

Result

Files created by h5py with variable-length string attributes now work correctly:

Group '/' has 1 attributes:
  - File Attribute = 123456

Dataset: Test Dataset
Dataset 'Test Dataset' has 2 attributes:
  - Dataset Attribute 1 = Test Attribute 1
  - Dataset Attribute 2 = Test Attribute 2

Quality

  • All tests pass ✅
  • Linter: 0 issues ✅
  • Official HDF5 Suite: 100% (378/378) ✅

Upgrade

go get github.com/scigolib/hdf5@v0.13.4

Full Changelog: v0.13.3...v0.13.4

v0.13.3 - 100% HDF5 Test Suite Pass Rate

27 Jan 21:57
86f05e0

Choose a tag to compare

What's Changed

Bug Fixes

  • V1 Object Header: Fixed end offset calculation (missing 16-byte header prefix)
  • V2 Object Header: Account for 4-byte CRC32 checksum at chunk end
  • V2 Message Headers: Use 6-byte headers when H5O_HDR_ATTR_CRT_ORDER_TRACKED is set
  • Soft Links: Lazy resolution following C library behavior
  • Named Datatypes: Added support for committed datatypes

Results

  • Official HDF5 test suite: 100% pass rate (378/378 valid files)
  • All tests pass on Linux, macOS, Windows
  • 0 lint issues

Closes

  • Issue #9: V0 superblock files showing 0 children

Full Changelog: v0.13.2...v0.13.3