enhance `binutils` easyblock to ensure system search path is used for `ld` instead of the default one by setting `$LIB_PATH` in build step by Crivella · Pull Request #4204 · easybuilders/easybuild-easyblocks

Crivella · 2026-08-12T15:10:25Z

This should fix the problem observed in easybuilders/easybuild-easyconfigs#20375

Currently the binutils EBlock determines the system search path through gcc -print-search-dirs and uses it to rpath builds in case of a system toolchain.

The problem is that the SEARCH_DIRS of ld are not properly set to the system ones and the default (on eg Debian/Ubuntu) misses /usr/lib/x86_64-linux-gnu and /lib/x86_64-linux-gnu.
For nvidia-compilers this causes failures when liking eg libc libdl and libpthread through ld directly.

This is not a problem in GCC based toolchains as gcc does its own logic to add this paths with -L statements
nvc++ in nvidia-compilers instead both adds -lc -ldl -lpthread to every invocation and passes it directly to ld causing failures of the type

/home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-14.2.0/bin/ld: cannot find -ldl: No such file or directory
/home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-14.2.0/bin/ld: cannot find -lpthread: No such file or directory
/home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-14.2.0/bin/ld: cannot find -lc: No such file or directory

Details

/tmp/eb-av4z3kjy/tmpf22k8are$ nvc++ minimal.cpp  -v
Export NVCOMPILER=/home/crivella/.local/easybuild/software/nvidia-compilers/25.3-CUDA-12.8.0/Linux_x86_64/25.3
Export PGI=/home/crivella/.local/easybuild/software/nvidia-compilers/25.3-CUDA-12.8.0

/home/crivella/.local/easybuild/software/nvidia-compilers/25.3-CUDA-12.8.0/Linux_x86_64/25.3/compilers/bin/tools/nvcpfe -- --llalign -D__GNUC__=14 -D__GNUC_MINOR__=2 -D__GNUC_PATCHLEVEL__=0 -D__unix -D__unix__ -Dunix -Dlinux -D__linux -D__linux__ -D__ELF__ -D__NO_MATH_INLINES -D__LP64__ -D__x86_64 -D__x86_64__ -D__LONG_MAX__=9223372036854775807L '-D__SIZE_TYPE__=unsigned long int' '-D__PTRDIFF_TYPE__=long int' -D__amd64 -D__amd64__ -D__k8 -D__k8__ -D__MMX__ -D__SSE_MATH__ -D__MMX_WITH_SSE__ -D__SSE__ -D__SSE2__ -D__SSE2_MATH__ -D__SSE3__ -D__SSSE3__ -D__SSE4_1__ -D__SSE4_2__ -D__ABM__ -D__ADX__ -D__AES__ -D__AVX__ -D__AVX2__ -D__AVXVNNI__ -D__BMI__ -D__BMI2__ -D__CLFLUSHOPT__ -D__CLWB__ -D__CX16__ -D__F16C__ -D__FMA__ -D__FSGSBASE__ -D__FXSR__ -D__GFNI__ -D__HRESET__ -D__KL__ -D__LZCNT__ -D__MOVBE__ -D__MOVDIR64B__ -D__MOVDIRI__ -D__PCLMUL__ -D__PCONFIG__ -D__PKU__ -D__POPCNT__ -D__PRFCHW__ -D__PTWRITE__ -D__RDPID__ -D__RDRND__ -D__RDSEED__ -D__LAHF_SAHF__ -D__SERIALIZE__ -D__SHA__ -D__SHSTK__ -D__VAES__ -D__VPCLMULQDQ__ -D__WAITPKG__ -D__WIDEKL__ -D__XSAVE__ -D__XSAVEC__ -D__XSAVEOPT__ -D__XSAVES__ -D__PGI -D__NVCOMPILER -D_GNU_SOURCE -D_PGCG_SOURCE -I- -I/home/crivella/.local/easybuild/software/numactl/2.0.19-GCCcore-14.2.0/include -I/home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-14.2.0/include -I/home/crivella/.local/easybuild/software/zlib/1.3.1-GCCcore-14.2.0/include -I/home/crivella/.local/easybuild/software/CUDA/12.8.0/nvvm/include -I/home/crivella/.local/easybuild/software/CUDA/12.8.0/extras/CUPTI/include -I/home/crivella/.local/easybuild/software/CUDA/12.8.0/targets/x86_64-linux/include --sys_include /home/crivella/.local/easybuild/software/nvidia-compilers/25.3-CUDA-12.8.0/Linux_x86_64/25.3/compilers/include --sys_include /home/crivella/.local/easybuild/software/nvidia-compilers/25.3-CUDA-12.8.0/Linux_x86_64/25.3/compilers/include-stdexec --sys_include /home/crivella/.local/easybuild/software/GCCcore/14.2.0/lib/gcc/x86_64-pc-linux-gnu/14.2.0/../../../../include/c++/14.2.0 --sys_include /home/crivella/.local/easybuild/software/GCCcore/14.2.0/lib/gcc/x86_64-pc-linux-gnu/14.2.0/../../../../include/c++/14.2.0/x86_64-pc-linux-gnu --sys_include /home/crivella/.local/easybuild/software/GCCcore/14.2.0/lib/gcc/x86_64-pc-linux-gnu/14.2.0/../../../../include/c++/14.2.0/backward --sys_include /home/crivella/.local/easybuild/software/GCCcore/14.2.0/lib/gcc/x86_64-pc-linux-gnu/14.2.0/include --sys_include /home/crivella/.local/easybuild/software/GCCcore/14.2.0/include --sys_include /home/crivella/.local/easybuild/software/GCCcore/14.2.0/lib/gcc/x86_64-pc-linux-gnu/14.2.0/include-fixed/x86_64-linux-gnu --sys_include /home/crivella/.local/easybuild/software/GCCcore/14.2.0/lib/gcc/x86_64-pc-linux-gnu/14.2.0/include-fixed --sys_include /usr/include/x86_64-linux-gnu --sys_include /usr/include -D__PGLLVM__ -D__NVCOMPILER_LLVM__ -D__extension__= --preinclude _cplus_preinclude.h --preinclude_macros _cplus_macros.h --gnu_version=140200 -D__pgnu_vsn=140200 --no_fixed_bp -q -o /tmp/nvc++0r1geCKx_rymw.il minimal.cpp -- minimal.cpp -opt 1 -x 119 0xa10000 -x 122 0x40 -x 123 0x1000 -x 127 4 -x 127 17 -x 19 0x400000 -x 28 0x40000 -x 120 0x10000000 -x 70 0x8000 -x 122 1 -x 125 0x20000 -quad -x 59 4 -tp alderlake -x 120 0x1000 -astype 0 -x 121 1 -fn minimal.cpp -il /tmp/nvc++0r1geCKx_rymw.il -x 117 0x200 -x 123 0x80000000 -x 123 4 -x 2 0x400 -x 209 0x10 -x 119 0x20 -def __pgnu_vsn=140200 -x 70 0x40000000 -x 183 4 -x 121 0x800 -x 6 0x20000 -x 249 190 -x 120 0x200000 -x 70 0x40000000 -x 8 0x40000000 -x 164 0x800000 -x 71 0x2000 -x 71 0x4000 -x 34 0x40000000 -x 120 0x1000000 -x 73 0x04 -x 231 0x800 -x 68 0x1 -x 39 4 -x 56 0x10 -x 26 0x10 -x 26 1 -x 56 0x4000 -x 197 0 -x 175 0 -x 203 0 -x 204 0 -x 227 0xFFFF -y 163 0xc0000000 -x 189 0x10 -y 189 0x4000000 -x 187 0x8000000 -x 60 512 -gnuvsn 140200 -x 69 0x200 -mach 0 -nomach 0 -x 123 0x400 -cmdline '+nvc++ minimal.cpp -v' -asm /tmp/nvc++0r1geCh1VHIvs.ll
NVC++/x86-64 Linux 25.3-0: compilation successful

/home/crivella/.local/easybuild/software/nvidia-compilers/25.3-CUDA-12.8.0/Linux_x86_64/25.3/compilers/share/llvm/bin/llc /tmp/nvc++0r1geCh1VHIvs.ll -march=x86-64 -mcpu=native -mattr=+mmx -mattr=+sse -mattr=+sse2 -mattr=+sse3 -mattr=+ssse3 -mattr=+sse4.1 -mattr=+sse4.2 -mattr=+avx -mattr=+avx2 -mattr=+f16c -mattr=+fma -mattr=+xsave -mattr=+xsaveopt -mattr=+xsavec -mattr=+xsaves -mattr=+popcnt -mattr=+sha -mattr=+aes -mattr=+pclmul -mattr=+clflushopt -mattr=+fsgsbase -mattr=+rdrnd -mattr=+bmi -mattr=+bmi2 -mattr=+lzcnt -mattr=+fxsr -mattr=+pku -mattr=+gfni -mattr=+vaes -mattr=+vpclmulqdq -mattr=+movdiri -mattr=+movdir64b -mattr=+adx -mattr=+avxvnni -mattr=+clwb -mattr=+cx16 -mattr=+hreset -mattr=+kl -mattr=+movbe -mattr=+pconfig -mattr=+prfchw -mattr=+ptwrite -mattr=+rdpid -mattr=+rdseed -mattr=+sahf -mattr=+serialize -mattr=+shstk -mattr=+waitpkg -mattr=+widekl -O1 -x86-cmov-converter=0 -dwarf-directory=false -non-global-value-max-name-size=-1 --align-all-functions=6 -nvhpc-memzero -nvhpc-dso-opt -filetype=obj --frame-pointer=none -o /tmp/nvc++ur1ge8cE4jlqm.o

/home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-14.2.0/bin/ld /usr/lib/x86_64-linux-gnu/crt1.o /usr/lib/x86_64-linux-gnu/crti.o /home/crivella/.local/easybuild/software/GCCcore/14.2.0/lib/gcc/x86_64-pc-linux-gnu/14.2.0//crtbegin.o --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -T /home/crivella/.local/easybuild/software/nvidia-compilers/25.3-CUDA-12.8.0/Linux_x86_64/25.3/compilers/lib/nvhpc.ld -L/home/crivella/.local/easybuild/software/nvidia-compilers/25.3-CUDA-12.8.0/Linux_x86_64/25.3/compilers/lib -L/home/crivella/.local/easybuild/software/numactl/2.0.19-GCCcore-14.2.0/lib -L/home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-14.2.0/lib -L/home/crivella/.local/easybuild/software/zlib/1.3.1-GCCcore-14.2.0/lib -L/home/crivella/.local/easybuild/software/GCCcore/14.2.0/lib64 -L/home/crivella/.local/easybuild/software/GCCcore/14.2.0/lib -L/home/crivella/.local/easybuild/software/CUDA/12.8.0/stubs/lib64 -L/home/crivella/.local/easybuild/software/CUDA/12.8.0/nvvm/lib64 -L/home/crivella/.local/easybuild/software/CUDA/12.8.0/extras/CUPTI/lib64 -L/home/crivella/.local/easybuild/software/CUDA/12.8.0/targets/x86_64-linux/lib -L/home/crivella/.local/easybuild/software/nvidia-compilers/25.3-CUDA-12.8.0/Linux_x86_64/25.3/compilers/lib -L/home/crivella/.local/easybuild/software/nvidia-compilers/25.3-CUDA-12.8.0/Linux_x86_64/25.3/compilers/lib -L/usr/lib64 -L/home/crivella/.local/easybuild/software/GCCcore/14.2.0/lib/gcc/x86_64-pc-linux-gnu/14.2.0/ /tmp/nvc++ur1ge8cE4jlqm.o -rpath /home/crivella/.local/easybuild/software/nvidia-compilers/25.3-CUDA-12.8.0/Linux_x86_64/25.3/compilers/lib -rpath /home/crivella/.local/easybuild/software/nvidia-compilers/25.3-CUDA-12.8.0/Linux_x86_64/25.3/compilers/lib -rpath /home/crivella/.local/easybuild/software/GCCcore/14.2.0/lib64 -L/home/crivella/.local/easybuild/software/GCCcore/14.2.0/lib64 -lstdc++ -lnvomp -ldl -lnvhpcatm --as-needed -latomic --no-as-needed -lpthread -lnvcpumath -lnsnvc -lnvc -lgcc -lc -lgcc_s -lm /home/crivella/.local/easybuild/software/GCCcore/14.2.0/lib/gcc/x86_64-pc-linux-gnu/14.2.0//crtend.o /usr/lib/x86_64-linux-gnu/crtn.o
/home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-14.2.0/bin/ld: cannot find -ldl: No such file or directory
/home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-14.2.0/bin/ld: cannot find -lpthread: No such file or directory
/home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-14.2.0/bin/ld: cannot find -lc: No such file or directory
nvc++-Fatal-linker completed with exit code 1

whit this PR the search dirs of ld will include the one given by the system compiler which should solve this issue

SEARCH_DIRS before PR

crivella@crivella-desktop:~$ module load binutils/2.42-GCCcore-14.2.0 
crivella@crivella-desktop:~$ ld -verbose | grep SEARCH | tr ';' '\n'
SEARCH_DIR("=/home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-14.2.0/x86_64-pc-linux-gnu/lib64")
 SEARCH_DIR("=/home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-14.2.0/lib64")
 SEARCH_DIR("=/usr/local/lib64")
 SEARCH_DIR("=/lib64")
 SEARCH_DIR("=/usr/lib64")
 SEARCH_DIR("=/home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-14.2.0/x86_64-pc-linux-gnu/lib")
 SEARCH_DIR("=/home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-14.2.0/lib")
 SEARCH_DIR("=/usr/local/lib")
 SEARCH_DIR("=/lib")
 SEARCH_DIR("=/usr/lib")

crivella@crivella-desktop:~$ which ld
/home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-14.2.0/bin/ld

SEARCH_DIRS after PR

crivella@crivella-desktop:~$ module use /tmp/eb_tmpdir-hCRyszlA/modules/all/
crivella@crivella-desktop:~$ module load binutils/2.42-GCCcore-14.2.0 
crivella@crivella-desktop:~$ ld -verbose | grep SEARCH | tr ';' '\n'
SEARCH_DIR("=/tmp/eb_tmpdir-hCRyszlA/software/binutils/2.42-GCCcore-14.2.0/x86_64-pc-linux-gnu/lib64")
 SEARCH_DIR("/home/crivella/.local/easybuild/software/zlib/1.3.1-GCCcore-14.2.0/lib64")
 SEARCH_DIR("/home/crivella/.local/easybuild/software/binutils/2.42/lib64")
 SEARCH_DIR("/home/crivella/.local/easybuild/software/flex/2.6.4-GCCcore-14.2.0/lib64")
 SEARCH_DIR("/home/crivella/.local/easybuild/software/GCCcore/14.2.0/lib64")
 SEARCH_DIR("/lib/x86_64-linux-gnu")
 SEARCH_DIR("/lib64")
 SEARCH_DIR("/lib")
 SEARCH_DIR("=/tmp/eb_tmpdir-hCRyszlA/software/binutils/2.42-GCCcore-14.2.0/x86_64-pc-linux-gnu/lib")

crivella@crivella-desktop:~$ which ld
/tmp/eb_tmpdir-hCRyszlA/software/binutils/2.42-GCCcore-14.2.0/bin/ld
crivella@crivella-desktop:~$

Potentially we might also want to exclude the non system libraries locations that are now detected (eg zlib and flex which are present due to being loaded at build time)

Crivella · 2026-08-12T15:25:52Z

Test report by @Crivella

Overview of tested easyconfigs (in order)

SUCCESS binutils-2.42-GCCcore-14.2.0.eb
SUCCESS nvidia-compilers-25.3-CUDA-12.8.0.eb

Build succeeded for 2 out of 2 (total: 6 mins 31 secs) (2 easyconfigs in total)
crivella-desktop - Linux Ubuntu 22.04.5 LTS (Jammy Jellyfish), x86_64, 13th Gen Intel(R) Core(TM) i9-13900K (skylake), 1 x NVIDIA NVIDIA GeForce RTX 5060, 580.173.02, Python 3.11.15
See https://gist.github.com/Crivella/e447133dcad48492e5e6f1f29848d30c for a full test report.

…earch dirs

Crivella · 2026-08-12T16:16:13Z

Test report by @Crivella

Overview of tested easyconfigs (in order)

SUCCESS binutils-2.37.eb
SUCCESS binutils-2.38.eb
SUCCESS binutils-2.38-GCCcore-11.3.0.eb
SUCCESS binutils-2.39.eb
SUCCESS binutils-2.39-GCCcore-12.2.0.eb
SUCCESS binutils-2.40.eb
SUCCESS binutils-2.40-GCCcore-12.3.0.eb
SUCCESS binutils-2.40-GCCcore-13.2.0.eb
SUCCESS binutils-2.42.eb
SUCCESS binutils-2.42-GCCcore-13.3.0.eb
SUCCESS binutils-2.44.eb
SUCCESS binutils-2.44-GCCcore-14.3.0.eb
SUCCESS binutils-2.45.eb
SUCCESS binutils-2.45-GCCcore-15.2.0.eb

Build succeeded for 14 out of 14 (total: 37 mins 10 secs) (14 easyconfigs in total)
crivella-desktop - Linux Ubuntu 22.04.5 LTS (Jammy Jellyfish), x86_64, 13th Gen Intel(R) Core(TM) i9-13900K (skylake), 1 x NVIDIA NVIDIA GeForce RTX 5060, 580.173.02, Python 3.11.15
See https://gist.github.com/Crivella/3d0c4c8afa65286522b377be838292cf for a full test report.

craig-grosch · 2026-08-13T13:33:54Z

Test report by @cgross95

Overview of tested easyconfigs (in order)

SUCCESS binutils-2.42-GCCcore-14.2.0.eb
SUCCESS nvidia-compilers-25.3-CUDA-12.8.0.eb

Build succeeded for 2 out of 2 (total: 14 mins 28 secs) (2 easyconfigs in total)
dev-amd20-v100 - Linux Ubuntu 22.04.5 LTS (Jammy Jellyfish), x86_64, Intel(R) Xeon(R) Platinum 8260 CPU @ 2.40GHz, 4 x NVIDIA Tesla V100S-PCIE-32GB, 580.159.04, Python 3.10.12
See https://gist.github.com/cgross95/4488498fa33ba4db4e050b49833a7061 for a full test report.

ocaisa

LGTM

ocaisa · 2026-08-16T09:48:16Z

Going in, thanks @Crivella

boegel · 2026-08-26T14:20:12Z

+        # this should 'harden' the resulting binutils to bootstrap GCC
+        # (no trouble when other libstdc++ is build etc)
+        lib_paths = self.determine_used_library_paths()
+        self.search_paths = lib_paths.copy()


@Crivella Since self.search_paths is used in sanity_check_step, can we already determine the list of library paths in the easyblock constructor instead?

That way, the sanity check commands will also be run when using --sanity-check-only...

In particular to address the second point i am not sure this is possible.
We would need the dependencies to be available to be able to filter them out

boegel · 2026-08-26T14:25:01Z

Potentially we might also want to exclude the non system libraries locations that are now detected (eg zlib and flex which are present due to being loaded at build time)

This bit was left unhandled, correct?
I think we need to make sure that paths to libraries of build-only dependencies of binutils don't get baked into ld...

Ensure system search path is used for ld instead of the default one

13f9bff

Add sanity check to ensure SEARCH_DIRS contain the detected systems s…

bf3f91b

…earch dirs

craig-grosch mentioned this pull request Aug 14, 2026

nvidia-compiler symlinks cause NVHPC installation to fail in make_module_step #4192

Closed

ocaisa approved these changes Aug 16, 2026

View reviewed changes

ocaisa merged commit 39b2f5c into easybuilders:develop Aug 16, 2026
21 checks passed

Crivella deleted the fix-binutils branch August 17, 2026 08:02

Crivella mentioned this pull request Aug 21, 2026

cannot build NVHPC on ubuntu (and workaround) easybuilders/easybuild-easyconfigs#20375

Closed

boegel changed the title ~~binutils Ensure system search path is used for ld instead of the default one~~ enhance binutils easyblock to ensure system search path is used for ld instead of the default one Aug 26, 2026

boegel reviewed Aug 26, 2026

View reviewed changes

boegel changed the title ~~enhance binutils easyblock to ensure system search path is used for ld instead of the default one~~ enhance binutils easyblock to ensure system search path is used for ld instead of the default one by setting $LIB_PATH in build step Aug 26, 2026

boegel added the enhancement label Aug 26, 2026

boegel added this to the next release (5.4.1?) milestone Aug 26, 2026

Crivella mentioned this pull request Aug 26, 2026

Ensure dependencies libpaths are not baked into binutils's ld #4209

Open

Navigation Menu

Uh oh!

enhance `binutils` easyblock to ensure system search path is used for `ld` instead of the default one by setting `$LIB_PATH` in build step - #4204

enhance `binutils` easyblock to ensure system search path is used for `ld` instead of the default one by setting `$LIB_PATH` in build step#4204
ocaisa merged 2 commits into
easybuilders:developfrom
Crivella:fix-binutils

Crivella commented Aug 12, 2026 •

edited

Loading

Uh oh!

Crivella commented Aug 12, 2026

Uh oh!

Crivella commented Aug 12, 2026

Uh oh!

craig-grosch commented Aug 13, 2026

Uh oh!

ocaisa left a comment

Uh oh!

ocaisa commented Aug 16, 2026

Uh oh!

Uh oh!

boegel Aug 26, 2026

Uh oh!

Crivella Aug 26, 2026

Uh oh!

boegel commented Aug 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Crivella commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Crivella commented Aug 12, 2026

Overview of tested easyconfigs (in order)

Uh oh!

Crivella commented Aug 12, 2026

Overview of tested easyconfigs (in order)

Uh oh!

craig-grosch commented Aug 13, 2026

Overview of tested easyconfigs (in order)

Uh oh!

ocaisa left a comment

Choose a reason for hiding this comment

Uh oh!

ocaisa commented Aug 16, 2026

Uh oh!

Uh oh!

boegel Aug 26, 2026

Choose a reason for hiding this comment

Uh oh!

Crivella Aug 26, 2026

Choose a reason for hiding this comment

Uh oh!

boegel commented Aug 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Crivella commented Aug 12, 2026 •

edited

Loading