More flexible linker choice for building Spicy and building Spicy parsers

[awelzel]

Mostly a question/discussion whether different linkers (and possibly other flags) should be separated between the environment where Spicy is built and the environment where Spicy parsers are built. Currently, setting LDFLAGS to contain -fuse-ld=lld when compiling Spicy itself is inherited to building Spicy parsers. This breaks if the "target" system doesn't have lld and instead maybe would prefer to use mold or just ld.

Background: I've added LDFLAGS=-fuse-ld=lld to an internal build environment to speed up linking of Zeek and Spicy. Assume the result is packaged, including zeek, spicyc, spicy-config, spicyz, etc.

Placing the package into an envrionment that doesn't have lld and instead just the classic ld linker results in a build failure of an external analyzer:

$ spicy-config --ldflags
-lspicy-rt -lz -L/home/awelzel/corelight-oss/zeek/build/auxil/spicy/lib -Wl,-rpath,/home/awelzel/corelight-oss/zeek/build/auxil/spicy/lib -lhilti-rt -fuse-ld=lld -fuse-ld=lld -ldl -Wno-unused-command-line-argument


$ cd build && VERBOSE=1 make
...
[ 50%] Compiling Message analyzer
cd /home/awelzel/corelight-oss/projects/spicy-sink-udp/analyzer && /usr/bin/cmake -E env ASAN_OPTIONS=:detect_leaks=0 /home/awelzel/corelight-oss/zeek/build/src/spicy/spicyz/spicyz -o /home/awelzel/corelight-oss/projects/spicy-sink-udp/build/message.hlto analyzer.spicy analyzer.evt

=== Uh oh ... C++ compilation failed ===

This is unexpected and should not happen. Please help us fix the issue
by opening a ticket at https://github.com/zeek/spicy/issues/new.

Please include the content of the following file with your report:

  "/home/awelzel/corelight-oss/projects/spicy-sink-udp/analyzer/hilti-jit-error.log"

That file contains diagnostics from the C++ compiler. Please also
include your Spicy code that triggered this error, ideally reduced to
a small snippet still exhibiting the issue if you can.

===

[error] <Spicy support for Zeek>: JIT compilation failed
make[2]: *** [analyzer/CMakeFiles/Message.dir/build.make:76: message.hlto] Error 1
make[2]: Leaving directory '/home/awelzel/corelight-oss/projects/spicy-sink-udp/build'
make[1]: *** [CMakeFiles/Makefile2:98: analyzer/CMakeFiles/Message.dir/all] Error 2
make[1]: Leaving directory '/home/awelzel/corelight-oss/projects/spicy-sink-udp/build'
make: *** [Makefile:136: all] Error 2

$ cat /home/awelzel/corelight-oss/projects/spicy-sink-udp/analyzer/hilti-jit-error.log
c++ (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

HILTI version 1.15.0-dev.0 (8e768adef)

## [job 7] /usr/bin/c++ -shared -Wl,-undefined -Wl,dynamic_lookup -fuse-ld=lld -fuse-ld=lld -o /tmp/spicy-jit-hlto-XXXXXXHtwpz0 /tmp/__linker___f2677d072f7ded26-2783dff29ed0cbef.cc_f2677d072f7ded26.o /tmp/Test_f2677d072f7ded26-ab95a76f32ce11c5.cc_f2677d072f7ded26.o /tmp/spicy_hooks_Test_f2677d072f7ded26-45ba300c5e766204.cc_f2677d072f7ded26.o /tmp/spicy_init_f2677d072f7ded26-e8f824e0ec6b6e25.cc_f2677d072f7ded26.o /tmp/zeek_rt_f2677d072f7ded26-f4f04683aca6a476.cc_f2677d072f7ded26.o

collect2: fatal error: cannot find ‘ld’
compilation terminated.

Once lld is installed in the other environment, compilation works. I'd think this similarly applies to Zeek's binary packages from OBS. If these are built with LDFLAGS=-fuse-ld=lld, the binary packages then also depend on the lld linker?

For generic flags, they probably should stay consistent, but maybe Spicy should treat the specific linker separately?

[bbannier]

Tools like spicyc also inherit CMAKE_CXX_COMPILER and then us it to compile Spicy-generated code.

The reason we do this is mostly to minimize potential errors from mismatch when linking against the runtime libraries like libhilti-rt. While we do not do any dark linker magic, allowing to in general change linker flags has pretty large surface area and IMO seems hard to control.

If you are certain that in your distribution your intended use case is fine you could still patch the flags to adjust what we injected. The linker flags are compiled into the binary here and set up in the CMake code.

[awelzel]

If you are certain that in your distribution your intended use case is fine you could still patch the flags to adjust what we injected.

I think "patching" would mostly mean stripping -fuse-ld=, so that the selected compiler just uses its default linker in the target environment.

My thought was more that the linker used by spicyc / spicyz might as well be configurable given the availability of -fuse-ld= for both GCC and clang.

Certainly there's nothing to do if that's just me, but found it unfortunate enough to start this discussion and maybe other users can piggy back.