Skip to content

fix(lambda): use PyPI wheels for numpy/pandas on AL2023 layer builds … #3398

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kukushking merged 1 commit into from
Jul 15, 2026
Merged

fix(lambda): use PyPI wheels for numpy/pandas on AL2023 layer builds … #3398

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions building/lambda/Dockerfile.al2023
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,14 @@ RUN pip3 install --upgrade cmake==3.31.6
# libcst (used for build-time stub docstring extraction) are required.
RUN pip3 install --upgrade six "cython>=3.1" hypothesis uv build scikit-build-core "libcst>=1.8.6"

ENV PIP_NO_BINARY="numpy,pandas"
# AL2023 has glibc 2.34, which satisfies the manylinux_2_28 tag that PyPI
# ships numpy/pandas wheels under. Building numpy/pandas from source here
# without locking the build-time numpy to the runtime numpy produces an
# ABI-mismatched pandas binary that SIGSEGVs on datetime64 construction
# (issue #3393). The AL2-based Dockerfile still forces source builds because
# AL2's glibc 2.26 can't load manylinux_2_28 wheels.
ENV UV_PROJECT_ENVIRONMENT="/var/lang"
RUN uv sync --frozen --inexact --no-dev --no-install-project \
--no-binary-package numpy \
--no-binary-package pandas
RUN uv sync --frozen --inexact --no-dev --no-install-project

RUN rm -f pyproject.toml uv.lock

Expand Down
13 changes: 12 additions & 1 deletion building/lambda/build-lambda-layer.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,18 @@ popd

pushd /aws-sdk-pandas

pip3 install . --no-binary numpy,pandas --find-links="${PYARROW_WHEEL_DIR}" -t ./python ".[redshift,mysql,postgres,gremlin,opensearch,openpyxl]" "pyarrow==${ARROW_VERSION}"
# Building numpy/pandas from source on AL2023 without pinning the build-time
# numpy to the runtime numpy produces an ABI-mismatched pandas binary that
# SIGSEGVs on datetime64 construction (issue #3393). AL2023's glibc 2.34
# satisfies the manylinux_2_28 tag that PyPI ships numpy/pandas wheels under,
# so use those wheels. AL2 (glibc 2.26) is too old to load manylinux_2_28,
# so keep forcing source builds there.
NO_BINARY_FLAG=""
if [ -r /etc/os-release ] && grep -q '^VERSION_ID="2"' /etc/os-release; then
NO_BINARY_FLAG="--no-binary numpy,pandas"
fi

pip3 install . ${NO_BINARY_FLAG} --find-links="${PYARROW_WHEEL_DIR}" -t ./python ".[redshift,mysql,postgres,gremlin,opensearch,openpyxl]" "pyarrow==${ARROW_VERSION}"

# CVE-2026-41066: upgrade lxml past redshift-connector's <=6.0.2 cap.
# pyproject.toml's [tool.uv] override-dependencies only applies to uv, not pip,
Expand Down
Loading