[Discussion] how to model GLIBC version for conan packages

[SSE4]

To help us debug your issue please explain:

• I've read the CONTRIBUTING guide.
• I've specified the Conan version, operating system version and any tool that can be relevant.
• I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.

/cc @rdeterre @bc-lee @elizagamedev

this is more generic version of conan-io/conan#3963
related issues:
bincrafters/community#296
bincrafters/community#267

historically, conan didn't model glibc version, relying only on compiler version for binary compatibility checks.
this causes some issues in the past, as binaries compiled with the same compiler weren't compatible across various linux distributions.
for instance, if we compiled libraries or executables via GCC 4.8 on Ubuntu 12.04, these executables and libraries fail to run/link on CentOS 7 machines. same situation with other popular Linux distributions, like RHEL, Fedora, etc.

[danimtb]

We should discuss this issue once we face the cross-building model one as this might affect the current settings and have further implications in Linux packages.

[scddev]

I am facing a similar issue regarding libc when building packages for glibc and musl (e.g. Alpine). Both are Linux, x86_64 but linked against a different libc.

We modified settings.yml like this:

compiler:
  ...
  gcc:
    ...
    libc: [None, glibc, musl]
  clang:
    ..
    libc: [None, glibc, musl]

This does not cover the fact of different versions of glibc or musl but covers at least our use case.

[vector-of-bool]

To note: There is more than just the glibc version that defines the system ABI on Linux. Things like dynamic linking SONAMEs to system libraries present another dimension of whether a given ELF can be loaded correctly on a given system. Issues with differing system libraries were my trouble with Qt, even if there were compatible glibc versions.

I have a local fork of Conan at $job and I experimented with the introduction of two settings: os_family and os_distribution. The former is more general, with values like RedHat, Debian, SUSE and Windows, while the latter is more specific, with things like CentOS, RHEL, Ubuntu, and OpenSUSE. Another setting, os_release would cover the major revisions of a particular OS family/distribution, with values like 7 (for RHEL 7 and CentOS 7), 16.04, or 19.04.

A tuple of (os_family, os_release) is almost always enough to distinguish the basic ABI of the system, since the libc version and important system SONAMEs are stable within a given release of a distribution. In my tests, I was able to successfully generate Qt packages using this scheme that reliably rebuilt between a Ubuntu and CentOS system without them accidentally sharing binaries.

Of course, not all packages are so sensitive to these system attributes, and may want to "opt-in" to become more granular.

One case this design does not consider is that a CentOS 6 package is probably compatible with CentOS 7 (as long as the proper SONAMEs match), but I personally don't consider that a compelling enough feature to warrant additional complexity.

[rdeterre]

As a workaround for this issue, is there a conan install option we could use to prevent a machine from using prebuilt binaries from the remote server, and rebuild packages from source if it does not have a locally built package?

Simply using --build solves the issue, except that all the dependencies get rebuilt every time even when perfectly valid prebuilt binaries are already available locally. This is problematic for big dependencies like Boost.

[Minimonium]

I mean, always building the stuff from public servers maybe not that nice, since then why do we store it all then.
Indeed, it would be great to have a distribution + its version combination to define the whole distribution environment, not just glibc.
But also we could consider providing an alternative approach of the glibc + standard library and its version combination for "compatible" binaries if your package doesn't use other system libraries. Maybe not that useful for public binaries, but it's good to have such default settings out of the box for private workshops to not reinvent the wheel.

[kwinsch]

Taking the suggestions from https://docs.conan.io/en/latest/extending/custom_settings.html as a base and incorporating the ideas from @scddev , the following custom settings are working for me:

os:
...
  Linux:
    distro: [None, RHEL6, RHEL7, Ubuntu18.04]
...
compiler:
  ...
  gcc:
    ...
    libc: [None, glibc, musl]
...

Whe also have some custom made distros for embedded device which we all have a distro name for. Unlike the glibc mess, working with musl as a libc, does usually not require the definition of the distro.