Installation fails due to unmaintained datrie dependency in Snakemake on modern Linux
Summary
When installing reana-client (or any package depending on reana-commons[snakemake]) on modern Linux systems with GCC >=14, the installation fails due to compilation errors in the datrie dependency. This issue originates from the fact that datrie is unmaintained and incompatible with recent compilers.
Details
datrie is a dependency of older Snakemake versions (<8.29.1).- Snakemake is used in
reana-commons for workflow support, and reana-commons is a dependency of reana-client.
- The
datrie repository has not been maintained for over 5 years. There is a pull request that addresses the compilation issue, but it has not been merged.
- Related issue in datrie: pytries/datrie#101
Example error log
src/datrie.c: In function ‘__pyx_f_6datrie_8AlphaMap_copy’:
src/datrie.c:26374:61: error: passing argument 1 of ‘alpha_map_clone’ from incompatible pointer type [-Wincompatible-pointer-types]
26374 | __pyx_v_clone->_c_alpha_map = alpha_map_clone(__pyx_v_self->_c_alpha_map);
| ~~~~~~~~~~~~^~~~~~~~~~~~~~
| |
| struct AlphaMap *
src/../libdatrie/datrie/alpha-map.h:70:46: note: expected ‘const AlphaMap *’ {aka ‘const struct _AlphaMap *’} but argument is of type ‘struct AlphaMap *’
70 | AlphaMap * alpha_map_clone (const AlphaMap *a_map);
| ~~~~~~~~~~~~~~~~^~~~~
src/datrie.c:26374:31: error: assignment to ‘struct AlphaMap *’ from incompatible pointer type ‘AlphaMap *’ {aka ‘struct _AlphaMap *’} [-Wincompatible-pointer-types]
26374 | __pyx_v_clone->_c_alpha_map = alpha_map_clone(__pyx_v_self->_c_alpha_map);
| ^
src/datrie.c: In function ‘__pyx_f_6datrie_8AlphaMap__add_range’:
src/datrie.c:27044:50: error: passing argument 1 of ‘alpha_map_add_range’ from incompatible pointer type [-Wincompatible-pointer-types]
27044 | __pyx_v_code = alpha_map_add_range(__pyx_v_self->_c_alpha_map, __pyx_v_begin, __pyx_v_end);
| ~~~~~~~~~~~~^~~~~~~~~~~~~~
| |
| struct AlphaMap *
src/../libdatrie/datrie/alpha-map.h:74:45: note: expected ‘AlphaMap *’ {aka ‘struct _AlphaMap *’} but argument is of type ‘struct AlphaMap *’
74 | int alpha_map_add_range (AlphaMap *alpha_map,
| ~~~~~~~~~~~^~~~~~~~~
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for datrieUpstream Fix in Snakemake
- The Snakemake team removed the
datrie dependency in PR #3176, addressing #3105 and #2970.
- This fix is available starting from v8.29.1.
Current State in reana-commons
reana-commons currently uses snakemake==8.27.1 and snakemake-storage-plugin-xrootd which depends on snakemake ^8.18.0.- As a result, installation fails on modern systems due to the unmaintained
datrie dependency.
Proposed Solution
- Update
snakemake-storage-plugin-xrootd to latest version.
- Update the Snakemake dependency in
reana-commons to >=8.29.1 (which is used in snakemake-storage-plugin-xrootd:latest or the latest compatible version).
- Ensure all tests pass and there are no regressions.
References
Impact:
This issue blocks the installation of any component depending on reana-commons[snakemake] on modern Linux distributions and affects reproducibility and maintainability for new users and contributors.
Notes
I've tested the installation process on Fedora 41, Fedora 42, Arch Linux, and Ubuntu 24.04 LTS (Desktop), using Python 3.13, 3.12, and 3.11, with gcc 14 and 15 (All failed) .
If there's any consideration I might have missed, please let me know.
Installation fails due to unmaintained
datriedependency in Snakemake on modern LinuxSummary
When installing
reana-client(or any package depending onreana-commons[snakemake]) on modern Linux systems with GCC >=14, the installation fails due to compilation errors in thedatriedependency. This issue originates from the fact thatdatrieis unmaintained and incompatible with recent compilers.Details
datrieis a dependency of older Snakemake versions (<8.29.1).reana-commonsfor workflow support, andreana-commonsis a dependency ofreana-client.datrierepository has not been maintained for over 5 years. There is a pull request that addresses the compilation issue, but it has not been merged.Example error log
Upstream Fix in Snakemake
datriedependency in PR #3176, addressing #3105 and #2970.Current State in reana-commons
reana-commonscurrently usessnakemake==8.27.1andsnakemake-storage-plugin-xrootdwhich depends onsnakemake ^8.18.0.datriedependency.Proposed Solution
snakemake-storage-plugin-xrootdto latest version.reana-commonsto>=8.29.1(which is used insnakemake-storage-plugin-xrootd:latestor the latest compatible version).References
Impact:
This issue blocks the installation of any component depending on
reana-commons[snakemake]on modern Linux distributions and affects reproducibility and maintainability for new users and contributors.Notes
I've tested the installation process on Fedora 41, Fedora 42, Arch Linux, and Ubuntu 24.04 LTS (Desktop), using Python 3.13, 3.12, and 3.11, with gcc 14 and 15 (All failed) .
If there's any consideration I might have missed, please let me know.