-
-
Notifications
You must be signed in to change notification settings - Fork 326
Description
1. Problem Description
Currently, the generic interface for configuring HDF5 filters relies exclusively on cd_values, an array of unsigned int. While this works for simple integer parameters, it creates significant friction for modern filters (like H5Z-ZFP) that require floating-point parameters (rates, precision, thresholds).
Pain Points:
- Type Punning: Users must manually bit-cast
doubleorfloatvalues into integer arrays. This is error-prone, endian-sensitive, and difficult to implement in high-level languages (Python, Java, Matlab) without dedicated wrappers. - CLI Usability: Tools like
h5repackrequire users to pass "magic numbers" (e.g.,-f UD=32013,0,4,1,0,0,1074921472) instead of human-readable settings. - Opaque Configuration: It is difficult to introspect a file's creation properties and understand what compression settings were used (e.g.,
cd_values[2]is meaningless compared to"rate=4.0").
2. Proposed Solution
Introduce a mechanism to configure filters using a string-based key-value interface, mirroring modern capabilities recently added for Virtual File Drivers (VFDs) and VOL connectors.
The underlying storage format (the cd_values array stored in the object header) should remain unchanged. This feature would simply be a usability layer that allows the filter plugin to parse a string and populate the cd_values array itself.
2. Example Usage
C API Comparison
Current (Manual Type Punning):
// User must know internal layout and bit-casting
unsigned int cd_vals[4];
double rate = 3.5;
cd_vals[0] = H5Z_ZFP_MODE_RATE;
memcpy(&cd_vals[2], &rate, sizeof(double)); // Complex & unsafe
H5Pset_filter(plist, H5Z_FILTER_ZFP, flags, 4, cd_vals);Proposed:
// User specifies intent; Plugin handles the bits
H5Pset_filter_config(plist, H5Z_FILTER_ZFP, "mode=rate; rate=3.5");CLI Impact (h5repack)
This would allow h5repack to support a generic string flag, removing the need for external helper scripts like print_h5repack_farg.
Current:
./print_h5repack_farg rate=3.5
# Output: -f UD=32013,0,4,1,0,0,1074528256
h5repack -f UD=32013,0,4,1,0,0,1074528256 in.h5 out.h5Proposed:
h5repack -f ZFP="mode=rate; rate=3.5" in.h5 out.h53. References
- Related Issue: llnl/H5Z-ZFP#159 - Discusses the complexity of generic usage.
- Precedent: HDF5 1.14 VFD configuration strings (
H5Pset_driver_by_nameconfiguration argument).
Metadata
Metadata
Assignees
Labels
Type
Projects
Status