Pure Go HDF5 Reader Library - Complementary Alternative for Cross-Platform Deployments

[kolkov]

Hi HDF5 community! 👋

I wanted to share a new pure Go HDF5 library that might be useful for certain use cases: scigolib/hdf5

Background

Your gonum/hdf5 library has been an excellent reference and inspiration! While it provides comprehensive HDF5 functionality through CGo bindings, we've built a pure Go read-only implementation to address specific
deployment scenarios.

Why Pure Go?

We created this library to solve challenges with:

• Cross-compilation - No C dependencies = easier builds for multiple platforms
• Containerization - Smaller Docker images without HDF5 C libraries
• Cloud deployments - Simpler deployment on AWS Lambda, Cloud Functions, etc.
• Windows/macOS - No need to install HDF5 system libraries

Current Status (v0.9.0-beta)

Supported:

• ✅ Superblock versions 0, 2, 3
• ✅ Object headers (v2)
• ✅ Groups (traditional + modern formats)
• ✅ Datasets (compact, contiguous, chunked with GZIP)
• ✅ Datatypes (integers, floats, strings, compounds)
• ✅ File traversal and navigation
• ✅ Cross-platform CI/CD (Linux, macOS, Windows)

Not Yet:

• ❌ Writing (read-only by design for safety)
• ❌ Advanced compression (SZIP, LZF, etc.)
• ❌ Some complex datatypes (arrays, enums, references)

Use Cases

Use gonum/hdf5 when you need:

• Full read/write capabilities
• Advanced HDF5 features
• Maximum compatibility

Use scigolib/hdf5 when you need:

• Pure Go (no CGo)
• Easy cross-compilation
• Lightweight deployments
• Read-only access is sufficient

Feedback Welcome!

We're actively testing against real-world HDF5 files. If anyone has:

• Edge cases or problematic files to test
• Suggestions for priority features
• Interest in collaboration

Please feel free to open issues or discussions!

Links

• Repository: https://github.com/scigolib/hdf5
• Module: github.com/scigolib/hdf5
• License: MIT

Thank you to the gonum/hdf5 maintainers for the excellent reference implementation! 🙏

---

Note: This is not meant to replace gonum/hdf5 but to complement it for specific deployment scenarios. Both libraries serve different needs.