[OTHER]: Allow modules to be statically linked

[JeWe37]

Please don't use.

As (linux) python modules are shared objects, they typically link to libc. For compatibility, they thus need to be built on a system which uses a libc which is older than the user system, as (g)libc is only forwards compatible. Typically this is addressed by standardizing on one of the manylinux variants as outlined in PEP 513, PEP 600 etc., however this requires that the project can be practically built on a system that old. This can often be challenging.

To address this, libc could be linked statically into the shared library, allowing the resulting shared object to be loaded on any linux system, even if built on a modern distribution, vastly reducing build complexity.

However, statically linking glibc is widely regarded as extremely poorly supported, and in fact distros such as Debian have not even shipped the required PIC variant of libc for years, that would be required for this. Yet, as statically linking libc means that which libc is used is irrelevant, so instead building a project on a modern musl libc based distro(e.g., alpine), will still allow it to be loaded even on glibc based distros.

Long story short, on any musl based system, the following CMake will generate a Python module that can be loaded regardless of distribution:

set(OLD_CMAKE_FIND_LIBRARY_SUFFIXES ${CMAKE_FIND_LIBRARY_SUFFIXES})
set(CMAKE_FIND_LIBRARY_SUFFIXES ".a") # Force finding a static library
find_library(
    LIBC_LIB
    NAMES c
)
set(CMAKE_FIND_LIBRARY_SUFFIXES ${OLD_CMAKE_FIND_LIBRARY_SUFFIXES})

target_link_libraries(my_mod PRIVATE  -nolibc -static-libgcc -static-libstdc++ ${LIBC_LIB})

This will break if any other shared objects are linked into the module, but that can usually be avoided comparatively easily(and is preferable for redistribution anyway), so long as the complexity of libc is taken care of.

I am wondering if there would be interest in integrating this feature into nanobind. The above code can be used to allow the user to configure this, but in so far as I can tell nobody has ever proposed this approach to dealing with libc compatibility for python modules, meaning figuring this out was complex to say the least, but is currently working for my module.

I suspect this would make nanobind the only binder(for any language) that has this feature, at least I have never seen anyone mention it.

[wjakob]

@JeWe37 That's an interesting approach. The restriction to a musl based system seems somewhat strict. Can such a build also be done on a glibc-based OS?

By the way, this seems like more of a discussion than in issue. Will convert.