Skip to content

Feature Request: String-Based Configuration Interface for Filters #6153

@brtnfld

Description

@brtnfld

1. Problem Description

Currently, the generic interface for configuring HDF5 filters relies exclusively on cd_values, an array of unsigned int. While this works for simple integer parameters, it creates significant friction for modern filters (like H5Z-ZFP) that require floating-point parameters (rates, precision, thresholds).

Pain Points:

  • Type Punning: Users must manually bit-cast double or float values into integer arrays. This is error-prone, endian-sensitive, and difficult to implement in high-level languages (Python, Java, Matlab) without dedicated wrappers.
  • CLI Usability: Tools like h5repack require users to pass "magic numbers" (e.g., -f UD=32013,0,4,1,0,0,1074921472) instead of human-readable settings.
  • Opaque Configuration: It is difficult to introspect a file's creation properties and understand what compression settings were used (e.g., cd_values[2] is meaningless compared to "rate=4.0").

2. Proposed Solution

Introduce a mechanism to configure filters using a string-based key-value interface, mirroring modern capabilities recently added for Virtual File Drivers (VFDs) and VOL connectors.

The underlying storage format (the cd_values array stored in the object header) should remain unchanged. This feature would simply be a usability layer that allows the filter plugin to parse a string and populate the cd_values array itself.

2. Example Usage

C API Comparison

Current (Manual Type Punning):

// User must know internal layout and bit-casting
unsigned int cd_vals[4];
double rate = 3.5;
cd_vals[0] = H5Z_ZFP_MODE_RATE;
memcpy(&cd_vals[2], &rate, sizeof(double)); // Complex & unsafe
H5Pset_filter(plist, H5Z_FILTER_ZFP, flags, 4, cd_vals);

Proposed:

// User specifies intent; Plugin handles the bits
H5Pset_filter_config(plist, H5Z_FILTER_ZFP, "mode=rate; rate=3.5");

CLI Impact (h5repack)

This would allow h5repack to support a generic string flag, removing the need for external helper scripts like print_h5repack_farg.

Current:

./print_h5repack_farg rate=3.5
# Output: -f UD=32013,0,4,1,0,0,1074528256
h5repack -f UD=32013,0,4,1,0,0,1074528256 in.h5 out.h5

Proposed:

h5repack -f ZFP="mode=rate; rate=3.5" in.h5 out.h5

3. References

  • Related Issue: llnl/H5Z-ZFP#159 - Discusses the complexity of generic usage.
  • Precedent: HDF5 1.14 VFD configuration strings (H5Pset_driver_by_name configuration argument).

Metadata

Metadata

Assignees

Labels

Component - C LibraryCore C library issues (usually in the src directory)HDFG-internalInternally coded for use by the HDF Group

Projects

Status

Scheduled/On-Deck

Relationships

None yet

Development

No branches or pull requests

Issue actions