Building multiarch image randomly fails in CI with "error: command '/usr/bin/gcc' failed with exit code -11"

[bugblasterX]

Hi,
I'm trying to debug the issue of my CI build failing randomly ( sometimes job succeeds, other times it does not ).
Error seems to happen during a script that runs poetry install of packages. For example the following CI failed on the "psutils" package build.
The CI looks like:

`variables:
  DOCKER_DRIVER: overlay2
  DOCKER_REGISTRY: "docker.io"
  DOCKER_HUB_REGISTRY: "docker.io"

.docker_build_script: &docker_build_script
  - docker context create builder
  - docker buildx version
  - docker buildx create builder --use
  - docker buildx build --platform linux/amd64,linux/arm64 --build-arg DOCKER_REGISTRY=${DOCKER_REGISTRY} ${EXTRA_BUILD_ARGS:-} -t ${IMAGE_TAG} --push .

.before_script_common: &before_script_common
  - apk update
  - apk add --no-cache gcc python3-dev musl-dev linux-headers openldap-dev openssl libffi-dev libjpeg-turbo-dev libxml2-dev libxslt-dev
  - echo "$DOCKER_HUB_PASS" | docker login $DOCKER_HUB_URL --username "$DOCKER_HUB_USER" --password-stdin

.Publish multiarch latest docker image template:
  stage: release
  interruptible: true
  image: docker:latest
  services:
    - name: "docker:dind"
      command: ["--mtu=1400"]
  rules:
    - if: '$CI_PIPELINE_SOURCE == "schedule"'
      when: never
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
  variables:
    IMAGE_TAG: "${TARGET}/${CI_PROJECT_NAME}:latest"
    EXTRA_BUILD_ARGS: ""  # We have extra build args in homeport
  cache: []
  before_script: *before_script_common
  script: *docker_build_script
`

And the error is:

`#24 87.02   - Installing psutil (6.1.1)
#24 87.04   - Installing pyasn1 (0.6.1)
#24 87.07   - Installing pyparsing (3.2.1)
#24 88.33   - Installing pyperclip (1.9.0)
#24 88.37   - Installing pyyaml (6.0.2)
#24 88.49   - Installing stevedore (5.4.0)
#24 88.89   - Installing typing-extensions (4.12.2)
#24 89.36   - Installing tzdata (2024.2)
#24 89.58   - Installing wcwidth (0.2.13)
#24 111.0 
#24 111.0   ChefBuildError
#24 111.0 
#24 111.0   Backend subprocess exited when trying to invoke build_wheel
#24 111.0   
#24 111.0   running bdist_wheel
#24 111.0   running build
#24 111.0   running build_py
#24 111.0   creating build/lib.linux-aarch64-cpython-311/psutil
#24 111.0   copying psutil/_psbsd.py -> build/lib.linux-aarch64-cpython-311/psutil
#24 111.0   copying psutil/_pssunos.py -> build/lib.linux-aarch64-cpython-311/psutil
#24 111.0   copying psutil/_pswindows.py -> build/lib.linux-aarch64-cpython-311/psutil
#24 111.0   copying psutil/_psosx.py -> build/lib.linux-aarch64-cpython-311/psutil
#24 111.0   copying psutil/__init__.py -> build/lib.linux-aarch64-cpython-311/psutil
#24 111.0   copying psutil/_psposix.py -> build/lib.linux-aarch64-cpython-311/psutil
#24 111.0   copying psutil/_pslinux.py -> build/lib.linux-aarch64-cpython-311/psutil
#24 111.0   copying psutil/_compat.py -> build/lib.linux-aarch64-cpython-311/psutil
#24 111.0   copying psutil/_common.py -> build/lib.linux-aarch64-cpython-311/psutil
#24 111.0   copying psutil/_psaix.py -> build/lib.linux-aarch64-cpython-311/psutil
#24 111.0   creating build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/test_system.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/test_linux.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/test_bsd.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/test_process_all.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/test_posix.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/test_unicode.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/test_aix.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/__init__.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/test_connections.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/test_osx.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/test_sunos.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/test_testutils.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/test_process.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/test_contracts.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/__main__.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/test_misc.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/test_windows.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   copying psutil/tests/test_memleaks.py -> build/lib.linux-aarch64-cpython-311/psutil/tests
#24 111.0   running build_ext
#24 111.0   building 'psutil._psutil_linux' extension
#24 111.0   creating build/temp.linux-aarch64-cpython-311/psutil
#24 111.0   creating build/temp.linux-aarch64-cpython-311/psutil/arch/linux
#24 111.0   gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=611 -DPy_LIMITED_API=0x03060000 -DPSUTIL_LINUX=1 -I/tmp/tmpie1mq016/.venv/include -I/usr/local/include/python3.11 -c psutil/_psutil_common.c -o build/temp.linux-aarch64-cpython-311/psutil/_psutil_common.o
#24 111.0   gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=611 -DPy_LIMITED_API=0x03060000 -DPSUTIL_LINUX=1 -I/tmp/tmpie1mq016/.venv/include -I/usr/local/include/python3.11 -c psutil/_psutil_linux.c -o build/temp.linux-aarch64-cpython-311/psutil/_psutil_linux.o
#24 111.0   gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=611 -DPy_LIMITED_API=0x03060000 -DPSUTIL_LINUX=1 -I/tmp/tmpie1mq016/.venv/include -I/usr/local/include/python3.11 -c psutil/_psutil_posix.c -o build/temp.linux-aarch64-cpython-311/psutil/_psutil_posix.o
#24 111.0   gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=611 -DPy_LIMITED_API=0x03060000 -DPSUTIL_LINUX=1 -I/tmp/tmpie1mq016/.venv/include -I/usr/local/include/python3.11 -c psutil/arch/linux/disk.c -o build/temp.linux-aarch64-cpython-311/psutil/arch/linux/disk.o
#24 111.0   psutil could not be installed from sources. Perhaps Python header files are not installed. Try running:
#24 111.0     sudo apk add gcc python3-dev musl-dev linux-headers
#24 111.0   error: command '/usr/bin/gcc' failed with exit code -11
#24 111.0   
#24 111.0 
#24 111.0   at /usr/local/lib/python3.11/site-packages/poetry/installation/chef.py:164 in _prepare
#24 111.0       160│ 
#24 111.0       161│                 error = ChefBuildError("\n\n".join(message_parts))
#24 111.0       162│ 
#24 111.0       163│             if error is not None:
#24 111.0     → 164│                 raise error from None
#24 111.0       165│ 
#24 111.0       166│             return path
#24 111.0       167│ 
#24 111.0       168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:
#24 111.0 
#24 111.0 Note: This error originates from the build backend, and is likely not a problem with poetry but with psutil (6.1.1) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "psutil (==6.1.1)"'.
#24 111.0 
#24 ERROR: process "/dev/.buildkit_qemu_emulator /bin/sh -c bash docker_build.sh" did not complete successfully: exit code: 1
------
 > [linux/arm64  9/12] RUN bash docker_build.sh:
111.0       162│ 
111.0       163│             if error is not None:
111.0     → 164│                 raise error from None
111.0       165│ 
111.0       166│             return path
111.0       167│ 
111.0       168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:
111.0 
111.0 Note: This error originates from the build backend, and is likely not a problem with poetry but with psutil (6.1.1) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "psutil (==6.1.1)"'.
111.0 
------`

The Dockerfile:

`ARG DOCKER_REGISTRY=docker.io/

FROM ${DOCKER_REGISTRY}python:3.11-alpine

ENV LANG C.UTF-8

RUN apk update && \
    apk add --no-cache \
    git\>=2.45 \
    bash\>=5.2 \
    shadow\>=4.15 \
    file\>=5.45 \
    openldap-dev\>=2.6 \
    openssl\>=3.3 \
    libffi-dev\>=3.4 \
    libjpeg-turbo-dev\>=3.0 \
    libxml2-dev\>=2.12 \
    libxslt-dev\>=1.1 \
    nginx\>=1.26 \
    xmlsec\>=1.3 \
    build-base\>=0.5 \
    jpeg-dev\>=9 \
    zlib-dev\>=1.3 \
    # Needed for psutil
    gcc\>=14.2 \
    python3-dev\>=3.12 \
    musl-dev\>=1.2 \
    linux-headers\>=6.6

# Set up locales
RUN echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen


COPY . /usr/src/project/

COPY docker/rootfs /

# Delete all test directories
RUN bash docker_build.sh

# Delete build-base package
RUN apk del build-base


ENTRYPOINT ["/app-entrypoint.sh"]
CMD ["/bin/bash"]`

I added the gcc dependencies with apk add in the base CI image and even in the Dockerfile. I'm not sure where to dig.

[bugblasterX]

Bump