patmos-clang always links libraries

[jeuneS2]

patmos-clang always includes whole libraries given on the command line, even if no function from that library is used. For example, when compiling the program "int main() { return 0; }" with "patmos-clang test.c -o test.elf -lm", the resulting file test.elf includes the whole libm, even though there is obviously no reason to pull in the library code.

[stefanhepp]

Does this happen for other libraries as well? -lm is treated specially,
since we link everything at bitcode level. If -lm is given, then
patmos-clang automatically adds the libmsyms.ll file, which always pulls
in those libm symbols for which calls are generated by llc from llvm
intrinsics. Otherwise they are not recognized as used by llvm-link and
would be removed.
The only way around that is to compile libm (and the other system
libraries) to ELF and link them at binary level.

There is a commandline flag (-nolibsyms) to disable adding those
libsyms.ll files, but then linking will probably fail (you need to make
sure yourself that the required functions are added to the bitcode file
and are marked as used).

On 05/23/2013 02:51 PM, jeuneS2 wrote:

patmos-clang always includes whole libraries given on the command line,
even if no function from that library is used. For example, when
compiling the program "int main() { return 0; }" with "patmos-clang
test.c -o test.elf -lm", the resulting file test.elf includes the whole
libm, even though there is obviously no reason to pull in the library code.

—
Reply to this email directly or view it on GitHub
#1.

[jeuneS2]

It only happens with libm. I understand the reasoning why libm is treated specially, but I am not completely convinced that the benefits outweigh the disadvantages. In particular, I was unable to get clang (for Patmos or x86) to produce bitcode with the intrinsics in question. I completely understand why functions from librt like muldi3 get linked in automatically, I just don't see the point for sqrt, sin, cos etc.

[stefanhepp]

We first started out with only adding functions that where actually
missing after linking, but adding new functions to the list was a
never-ending story, so at some point I searched through the LLVM docs
and the backend code to find all function calls that might possibly be
created somewhere in the backend.
While I agree that this pulls in a lot of unused stuff (but remember,
this only happens if you actually use -lm in your clang options!) and
thus increases the compile time, I see this not so much a question of
benefits and drawbacks but of functionality and stability. Pulling in
all functions is the only safe way to go, devoid of some quite complex
code (see below). I do not want to depend on clang/optimization
passes/.. not doing something that they might/should actually do (e.g.,
using llvm.cos.* intrinsics instead of cos() calls,..).

There are several ways to avoid this problem, if you really do not want
those functions in your linked application:

• Compile newlib and compiler-rt as ELF (use patmos-clang
  -fpatmos-emit-obj). This works (by now), and makes linking and compiling
  individual apps a lot faster (since you do not need to compile the
  whole of newlib every time), but I found that when compiling both newlib
  and the benchmarks from a clean build, it does not make much of a
  difference in total since compiling newlib takes longer, and the
  binaries are actually slightly bigger and the benchmarks slightly slower
  (due to missing whole-program optimizations).
• Add a pass that basically creates lib_syms.ll on the fly from the used
  functions in the bitcode. I tried that, failed miserably (because I
  never got *all_ the required functions since the backend does some
  really magic transformations when it comes to floating-point, so I would
  have to be very conservative again), and gave up. The pass is still
  there somewhere in the patmos-llvm repo, but there is not much point in it.
• Use the LTO linker to pull in bitcode libraries after the calls are
  actually created. Sounds great and clean, but actually does not work
  either because the LTO module first collects all the bitcode inputs,
  links them (using the bitcode symbols to pull in libraries), and then
  compiles the whole linked module, which is basically the same that we do
  now. To fix this, it would be necessary to change the LTO module that it
  compiles the libraries in several steps / iterations / .., but this is
  not trivial, so I stopped looking into it.

It might be possible to do a pass that 'un-uses' pulled in bitcode
library functions right after all functions have been lowered to
MachineInsts (bitcode linking is fast anyway; I adopted the PassManager
infrastructure so that arbitrary combinations of ModulePasses and
MachineFunctionPasses are now possible in patmos-llvm) though. Sadly
there is not enough time (and gain) for me to look into that option as
well..

Cheers,
Stefan

On 05/23/2013 04:51 PM, jeuneS2 wrote:

It only happens with libm. I understand the reasoning why libm is
treated specially, but I am not completely convinced that the benefits
outweigh the disadvantages. In particular, I was unable to get clang
(for Patmos or x86) to produce bitcode with the intrinsics in question.
I completely understand why functions from librt like muldi3 get linked
in automatically, I just don't see the point for sqrt, sin, cos etc.

—
Reply to this email directly or view it on GitHub
#1 (comment).