Proposal: Package splitting

[atrniv]

This is a proposal to follow a common pattern to split up certain groups on packages that have cyclic runtime/build dependencies.

The core idea is to repackage various package components into separate packages that can have a different set of dependencies. In most cases this should enable us to prevent cycles in our build graph.

Below are some concrete use cases for this pattern:

glibc

Problem

glibc has multiple binaries which are shell scripts with a #!/bin/sh and #!/usr/bin/perl header. If you installed glibc in an environment which does not have /bin/sh (eg: by exporting the package to a docker image), it would fail to run these binaries. This also violates Habitat's principle of not depending on the host system to guarantee runtime portability.

Solution

• Build glibc from source as glibc-lib and exclude the bin folder in the plan so that the binaries are not picked up in the runtime path.
• Build bash using glibc-lib as a build dep via gcc.
• Build perl
• Build glibc-bin which depends on glibc-lib, bash and perl. We will copy the binaries from glibc-lib to glibc-bin and expose the bin folder.
• Build glibc which depends on glibc-lib and glibc-bin

glibc provides the complete glibc package with libs and bins for end users to consume.

zstd

Problem

zstd has two binaries zstdless and zstdgrep that depend on grep and less at runtime. However both these packages are built after zstd

Solution

• Build zstd from source as zstd-base which has libzstd that is needed for using zstd as a build dep
• Build less and grep
• Build zstd which depends zstd-base, grep and less

readline

Problem

readline depends on the C libraries available in the ncurses package to be built. However a complete ncurses package requires a C++ compiler.

Solution

• Build ncurses-base from source with only the C libraries using the C compiler from build-tools-gcc.
• Build readline with ncurses-base
• Build ncurses after gcc is built using the full C++ compiler

[mwrock]

This makes sense to me and seems like a good idea.