Install script does not use pg_config's LIBDIR for runtime linking

[gtzampanakis]

The install script does not pass pg_config's LDFLAGS to the linker. In my case pg_config outputs the following LDFLAGS:

LDFLAGS = -L/usr/lib64/llvm5.0/lib -L/usr/lib64 -Wl,--as-needed -Wl,-rpath,'/usr/pgsql-12/lib',--enable-new-dtags

But the psycopg2 install script ignores everything that doesn't start with "-L", therefore it ignores the rpath configuration. Then, when psycopg2 is loaded by a uwsgi executable that has its own rpath the process exits with a "missing name" error because the fact that the executable has its own rpath prevents it from loading the psycopg2 library even though ldconfig has run and has properly configured the system.

To work around this I do

LDFLAGS='-L/usr/lib64/llvm5.0/lib -L/usr/lib64 -Wl,--as-needed -Wl,-rpath,/usr/pgsql-12/lib,--enable-new-dtags' pip install psycopg2

and then psycopg2 seems to work fine.

Please consider loading pg_config's LDFLAGS and passing them to the linker without user intervention.

[dvarrazzo]

Have you got a MR to suggest?

[gtzampanakis]

See #1357. My original suggestion to pass pg_config's LDFLAGS to the linker was not correct because those flags are the ones that were used to build postgres so it doesn't follow that they should be used to build psycopg2 too. My solution in #1357 is to take just the LIBDIR from pg_config (which psycopg2's setup already used for compile-time linking) and use it for runtime linking.

[zyv]

My original suggestion to pass pg_config's LDFLAGS to the linker was not correct because those flags are the ones that were used to build postgres so it doesn't follow that they should be used to build psycopg2 too.

So, the real problem is that some "smarter" linking systems like libtool would use list of library directories to generate LDFLAGS for the linker in such a way, that if the library is already on the default linker path, then this path will be omitted, and if it is not, then it will be passed along with the -rpath flag to the linker, like -Wl,-rpath,'/usr/pgsql-12/lib', so that this path will be remembered in the shared module and used later at the load time.

For better or worse, distutils does not do those libtool dances, so what happens here is that the linking is successful, but loading at runtime fails, because in your case Postgres libdir is not on the loader path for whatever reason (broken installation, broken package, genius plan gone south, ...).

A more "correct" solution as compared to #1357, where you are adding libdir to rpath unconditionally on Linux, would be to actually replicate some of the libtool logic and write platform-specific code to check whether the libdir where libpq resides (or even all library_dirs — how about other dependencies of pscycopg?) is on the default loader path, and if not, then to add those paths that aren't to self.rpath. This would be platform-specific, e.g. on Linux something like the following would work to obtain the loader paths:

$ ldconfig -v 2>/dev/null | grep -v ^$'\t' | cut -d ":" -f 1
/usr/lib64//bind9-export
/usr/lib64/mysql
/lib
/lib64
/lib/sse2
/lib64/sse2
/lib64/tls

Your system is configured in a way that Postgres is not on the loader path, and if you don't like it, then you should fix your system by adding it, or else, if you have reasons why you configured it this way, then you have to live with the consequences of this choice by either supplying LDFLAGS=-Wl,-rpath,'/usr/pgsql-12/lib' at the compile time or adding libdir to LD_LIBRARY_PATH when you run the software depending on libpq.

Anyway, I would not complicate the world to support obscure edge cases like this with a lot of effort, or by adding libdir to rpath for everyone, which is bound to introduce a whole set of different problems and security issues... especially because if such systems are really configured this way for a reason, the best idea maybe would be to not interfere with that.