fix: portable Linux binaries (glibc 2.28 baseline) — Raspberry Pi & Amazon Linux 2023 out of the box#179
Merged
Merged
Conversation
… build Linux release binaries previously required GLIBC_2.38 / GLIBCXX_3.4.32 (built directly on ubuntu-24.04 runners), failing on Raspberry Pi OS, Amazon Linux 2023, and anything older than ~mid-2024 with 'GLIBC_2.38 not found'. The Linux CI build now runs inside quay.io/pypa/manylinux_2_28 (AlmaLinux 8, glibc 2.28, gcc-toolset-14) via scripts/build_portable_linux.sh, which also builds static OpenSSL 3 + Boost program_options into a cached prefix and hard-fails if any produced binary exceeds the glibc 2.28 / EL8 GLIBCXX baseline. CI smoke-tests the binaries in debian:bookworm (the Raspberry Pi OS userland) and amazonlinux:2023 containers on every run. Verified locally end-to-end on both userlands (server boot, extension INSTALL/LOAD, client query). Resulting symbol maxima: GLIBC_2.28, GLIBCXX_3.4.22, CXXABI_1.3.11. Build-system changes this surfaced: - Forward OPENSSL_ROOT_DIR/OPENSSL_USE_STATIC_LIBS into the Arrow and jwt-cpp ExternalProjects (GIZMOSQL_EP_OPENSSL_*) — they only found OpenSSL implicitly via system paths before. - Pin -DCMAKE_INSTALL_LIBDIR=lib for DuckDB: Red Hat-family hosts default to lib64, breaking the hard-coded third_party/duckdb/lib paths. - No perf impact: same gcc 14 codegen; glibc picks optimized routines at runtime via IFUNC on the target machine. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The integration test binary embeds $<TARGET_FILE:gizmosql_client> as an absolute path at compile time. Building at /work inside the container made that path dangle when tests ran on the host. Mounting the workspace at the host's own path keeps every baked-in absolute path valid. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…/work The first portable run built at /work inside the container; its cached CMake build trees embed that path and CMake refuses to reuse them now that the workspace is mounted at the host path (which is stable across runs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
If the server did not come up within the probe deadline, ASSERT_TRUE
returned from the test with server_thread still joinable — std::thread's
destructor then called std::terminate, aborting the whole test binary
('terminate called without an active exception', exit 134). Join the
thread before asserting, and double the probe deadline to 30s for loaded
4-core CI runners.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…citly Inside the manylinux container (root) the cache-restored, runner-owned git clones tripped git's 'dubious ownership' guard. That silently broke two version detections: - GizmoSQL's own get_latest_git_tag() (empty version) - DuckDB's git describe → fell back to v0.0.1, so every runtime extension INSTALL 404'd against extensions.duckdb.org/v0.0.1/ — the cause of the arm64-lts test failures (env-var server init SQL INSTALL spatial, instrumentation json extension, fixture SetUps). Fix both: git config safe.directory '*' inside the container, and pass -DOVERRIDE_GIT_DESCRIBE=<tag> to DuckDB (officially supported) so its version always equals the tag we check out, independent of git state. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Linux release binaries previously required
GLIBC_2.38/GLIBCXX_3.4.32(a side effect of building directly on theubuntu-24.04runners), so they failed on Raspberry Pi OS, Amazon Linux 2023, and anything older than ~mid-2024 with "GLIBC_2.38 not found" — workable only via loader hacks (Homebrew glibc + manualld-linuxinvocation).The Linux CI build now runs inside
quay.io/pypa/manylinux_2_28(AlmaLinux 8, glibc 2.28 baseline, gcc-toolset-14) viascripts/build_portable_linux.sh. Resulting symbol ceilings:GLIBC_2.28,GLIBCXX_3.4.22,CXXABI_1.3.11— covering Raspberry Pi OS (bullseye and bookworm), Amazon Linux 2023, Ubuntu 20.04+, Debian 10+.No performance impact: identical gcc 14 code generation; glibc selects its optimized SIMD routines at runtime on the target machine (IFUNC), not at build time. This is the same recipe DuckDB uses for its own portable Linux binaries.
Guard rails (every CI run)
scripts/build_portable_linux.shhard-fails if any produced binary exceeds the glibc 2.28 / EL8 GLIBCXX baseline (objdump symbol-version check).debian:bookworm-slim(the Raspberry Pi OS userland) andamazonlinux:2023containers.Build-system fixes this surfaced
GIZMOSQL_EP_OPENSSL_ROOT_ARG/GIZMOSQL_EP_OPENSSL_STATIC_ARG.lib64/on Red Hat-family hosts, breaking the hard-codedthird_party/duckdb/libpaths — pinned-DCMAKE_INSTALL_LIBDIR=lib.$<TARGET_FILE:gizmosql_client>path compiled into the test binary) stay valid when tests run on the host.program_optionsare built into a cached prefix (build/portable-deps, stamp-guarded) since EL8's system versions are too old (1.1.1 / 1.66).Verification
INSTALL spatial; LOAD spatial(validates DuckDB extensiondlopenon the old baseline) and a client query round-trip indebian:bookworm; binary startup onamazonlinux:2023.Dockerfile.cinote updated: the trixie-only base constraint is gone (any Debian 11+ base now works).🤖 Generated with Claude Code