Skip to content

[Feature]: Documenting h5py & tables incompatibilities for future reference #981

@rly

Description

@rly

What would you like to see added to the NWB GUIDE?

#972 and #917 surfaced incompatibilities between h5py and tables/pytables accessing hdf5 libraries when packaging within electron.

For example, in https://github.com/NeurodataWithoutBorders/nwb-guide/actions/runs/13467758128/job/37636885970?pr=972

Error:   File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  ...
  File "h5py/__init__.py", line 25, in <module>
ImportError: dlopen(/Users/runner/work/nwb-guide/nwb-guide/build/flask/nwb-guide/_internal/h5py/_errors.cpython-312-darwin.so, 0x0002): Symbol not found: _H5T_IEEE_F16BE_g
  Referenced from: <E63CF6C4-706B-3E0B-953D-CC7FF32770E8> /Users/runner/work/nwb-guide/nwb-guide/build/flask/nwb-guide/_internal/h5py/.dylibs/libhdf5_hl.310.dylib
  Expected in:     <483A540C-F351-35F9-826C-8C3E4E72FAF5> /Users/runner/work/nwb-guide/nwb-guide/build/flask/nwb-guide/_internal/tables/.dylibs/libhdf5.310.dylib

And in https://github.com/NeurodataWithoutBorders/nwb-guide/actions/runs/14324511705/job/40148890925?pr=917

ImportError: dlopen(/Users/runner/work/nwb-guide/nwb-guide/build/flask/nwb-guide/_internal/h5py/_errors.cpython-312-darwin.so, 0x0002): Symbol not found: _H5T_IEEE_F16BE_g
  Referenced from: <E63CF6C4-706B-3E0B-953D-CC7FF32770E8> /Users/runner/work/nwb-guide/nwb-guide/build/flask/nwb-guide/_internal/h5py/.dylibs/libhdf5_hl.310.dylib
  Expected in:     <1DD687E0-02AD-31AA-8A12-EF9CE10D0F40> /Users/runner/work/nwb-guide/nwb-guide/build/flask/nwb-guide/_internal/tables/.dylibs/libhdf5.310.dylib

Both involve Symbol not found: _H5T_IEEE_F16BE_g

And in https://github.com/NeurodataWithoutBorders/nwb-guide/actions/runs/14325753917/job/40150847053?pr=917

Error:   File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "manageNeuroconv/__init__.py", line 2, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "manageNeuroconv/manage_neuroconv.py", line [17](https://github.com/NeurodataWithoutBorders/nwb-guide/actions/runs/14324511705/job/40148890925?pr=917#step:15:18), in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "pynwb/__init__.py", line 10, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 4[19](https://github.com/NeurodataWithoutBorders/nwb-guide/actions/runs/14324511705/job/40148890925?pr=917#step:15:20), in exec_module
  File "h5py/__init__.py", line 33, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "h5py/version.py", line 15, in <module>
  File "h5py/h5.pyx", line 1, in init h5py.h5
ImportError: dlopen(/Users/runner/work/nwb-guide/nwb-guide/build/flask/nwb-guide/_internal/h5py/defs.cpython-312-darwin.so, 0x0002): Symbol not found: _H5Pget_fapl_ros3
  Referenced from: <55BF9E97-954A-3CEF-A912-48F59[25](https://github.com/NeurodataWithoutBorders/nwb-guide/actions/runs/14324511705/job/40148890925?pr=917#step:15:26)8FCB5> /Users/runner/work/nwb-guide/nwb-guide/build/flask/nwb-guide/_internal/h5py/defs.cpython-312-darwin.so
  Expected in:     <1DD687E0-02AD-31AA-8A12-EF9CE10D0F40> /Users/runner/work/nwb-guide/nwb-guide/build/flask/nwb-guide/_internal/tables/.dylibs/libhdf5.310.dylib

This involves Symbol not found: _H5Pget_fapl_ros3

Longgggg story short:

  1. Re: _H5T_IEEE_F16BE_g. H5T_IEEE_F16BE is a new data type introduced in HDF5 1.14.4.
    If tables-3.9.1 (comes with older hdf5) and h5py-3.13.0 (comes with hdf5 1.14.6) are installed, then for some reason, h5py will look at the hdf5 libraries in the tables package rather than its own. Installing hdf5 via conda does not help.

  2. Re: _H5Pget_fapl_ros3. If HDF5 is installed from conda (e.g., via pytables, or perhaps also h5py) then it gets packaged with ROS3 support. However, if conda pytables is uninstalled (and perhaps also conda h5py is uninstalled) and pip pytables is installed due to pip-related requirements, then it comes with its own version of HDF5 and electron/h5py gets confused about what HDF5 features are available and where the libraries are.

Do you have any interest in helping implement the feature?

Yes.

Code of Conduct

Yes

Did you confirm this feature was not already reported?

Yes

Metadata

Metadata

Assignees

Labels

documentationImprovements or additions to documentationenhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions