{Continuous Integration} Wheels - Add Linux aarch64 build matrix leg (facebookresearch#387)

Summary:
X-link: https://github.com/facebookexternal/projectaria_private_libs/pull/35

Pull Request resolved: facebookresearch#387

Explanation:
Adds a native Linux aarch64 wheel to the `projectaria_tools` build matrix using GitHub's `ubuntu-24.04-arm` runner (free for public repos, so no QEMU emulation is needed). Three changes make the aarch64 build work:

1. `build_third_party_libs/install_manylinux_deps.sh` — two fixes, both of which also apply to x86_64:
   - The aarch64 `manylinux2014` image does not enable EPEL by default, so `gtest-devel` and `xxhash-devel` fail to resolve and `yum install` aborts the script. Add `yum install -y epel-release` first; this is a no-op on the x86_64 image, which already has EPEL enabled.
   - `libjpeg-turbo` 3.0.1 declares a `cmake_minimum_required` below 3.5, which CMake 4.x rejects outright. The `manylinux2014` image bumped CMake to 4.4 on both arches, so pass the CMake-suggested `-DCMAKE_POLICY_VERSION_MINIMUM=3.5`.

2. `setup.py` — Ocean's NEON RGB-conversion code passes signed vector types to unsigned intrinsics (for example `int8x16_t` into `vget_low_u8`). Clang accepts this by default, which is why Ocean's Android and iOS builds are unaffected, but GCC on Linux aarch64 rejects it and asks for `-flax-vector-conversions`. Add that flag to `CMAKE_CXX_FLAGS`, gated on Linux aarch64 so x86_64 and macOS builds are untouched.

3. `.github/workflows/build-wheels-and-deploy.yml` and `.github/workflows/test-for-build-wheels.yml` — add `ubuntu-24.04-arm` to the OS matrix, extend the ffmpeg-install step's condition to cover it, and add a build step with `CIBW_ARCHS: aarch64`. The step relies on `cibuildwheel`'s default `auditwheel repair`, which vendors the third-party shared libraries into the wheel and retags it `manylinux2014_aarch64`. NVDEC is not wired up on this leg because `libnvcuvid` has no aarch64 build.

Reproducibility:
Built and tested end-to-end in native `linux/arm64` containers. The resulting `manylinux2014_aarch64` wheel installs into a clean `python:3.11-slim` image — one with none of the build-time libraries present — and passes all 115 Python tests in the repository, including the H.265 random-access test that reads `data/gen2/aria_gen2_unit_test_sequence_with_pframe.vrs`. See the test plan for the full breakdown.

Reviewed By: SeaOtocinclus, nrraina

Differential Revision: D114764549

fbshipit-source-id: efb3c83e7fe5f55edced50a66dac3456f47a0c22

Author: kongchen1992

.github/workflows/build-wheels-and-deploy.yml

@@ -89,7 +89,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        os: [macos-14, ubuntu-latest] # macos-14 is macSilicon
+        os: [macos-14, ubuntu-latest, ubuntu-24.04-arm]  # macos-14 = macSilicon; ubuntu-24.04-arm = native Linux aarch64
         python-version: ["3.10", "3.11", "3.12"]
         include: # Customize cibuildwheel python version to build
           - python-version: "3.10"
@@ -119,7 +119,7 @@ jobs:
           python -m pip install --upgrade pip
           python -m pip install cibuildwheel==2.21.0
       - name: Install ffmpeg dependency lib
-        if: ${{ (matrix.os == 'macos-14') || (matrix.os == 'ubuntu-latest') }}
+        if: ${{ (matrix.os == 'macos-14') || (matrix.os == 'ubuntu-latest') || (matrix.os == 'ubuntu-24.04-arm') }}
         shell: bash
         run: |
           ./build_third_party_libs/build_ffmpeg_linuxunix.sh
@@ -147,6 +147,20 @@ jobs:
           CIBW_REPAIR_WHEEL_COMMAND_LINUX: "auditwheel repair --exclude libcuda.so.1 --exclude libnvcuvid.so.1 -w {dest_dir} {wheel}"
           CIBW_SKIP: "*-manylinux_i686 *-musllinux_*"
           CIBW_ARCHS: "x86_64"
+      - name: Build wheels for CPython
+        if: ${{ matrix.os == 'ubuntu-24.04-arm' }}
+        shell: bash
+        run: |
+          CMAKE_BUILD_PARALLEL_LEVEL=4 python -m cibuildwheel --output-dir dist
+        env:
+          CIBW_BUILD_VERBOSITY: 3
+          CIBW_BUILD: ${{ matrix.cibw_build }}
+          CIBW_BEFORE_BUILD_LINUX: bash build_third_party_libs/install_manylinux_deps.sh
+          # Unlike the x86_64 leg above, NVDEC is not enabled here: NVIDIA ships
+          # libnvcuvid for x86_64 only, so there is nothing for the CUDA env vars
+          # or the auditwheel exclusions to act on.
+          CIBW_SKIP: "*-manylinux_i686 *-musllinux_*"
+          CIBW_ARCHS: "aarch64"
       - name: Build wheels for CPython
         if: matrix.os == 'windows-2022'
         shell: bash

.github/workflows/test-for-build-wheels.yml

@@ -91,7 +91,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        os: [macos-14, ubuntu-latest] # macos-14 is macSilicon
+        os: [macos-14, ubuntu-latest, ubuntu-24.04-arm]  # macos-14 = macSilicon; ubuntu-24.04-arm = native Linux aarch64
         python-version: ["3.10", "3.11", "3.12"]
         include:
           - python-version: "3.10"
@@ -120,7 +120,7 @@ jobs:
           python -m pip install --upgrade pip
           python -m pip install cibuildwheel==2.21.0
       - name: Install ffmpeg dependency lib
-        if: ${{ (matrix.os == 'macos-14') || (matrix.os == 'ubuntu-latest') }}
+        if: ${{ (matrix.os == 'macos-14') || (matrix.os == 'ubuntu-latest') || (matrix.os == 'ubuntu-24.04-arm') }}
         shell: bash
         run: |
           ./build_third_party_libs/build_ffmpeg_linuxunix.sh
@@ -148,6 +148,20 @@ jobs:
           CIBW_REPAIR_WHEEL_COMMAND_LINUX: "auditwheel repair --exclude libcuda.so.1 --exclude libnvcuvid.so.1 -w {dest_dir} {wheel}"
           CIBW_SKIP: "*-manylinux_i686 *-musllinux_*"
           CIBW_ARCHS: "x86_64"
+      - name: Build wheels for CPython
+        if: ${{ (matrix.os == 'ubuntu-24.04-arm') }}
+        shell: bash
+        run: |
+          CMAKE_BUILD_PARALLEL_LEVEL=4 python -m cibuildwheel --output-dir dist
+        env:
+          CIBW_BUILD_VERBOSITY: 3
+          CIBW_BUILD: ${{ matrix.cibw_build }}
+          CIBW_BEFORE_BUILD_LINUX: bash build_third_party_libs/install_manylinux_deps.sh
+          # Unlike the x86_64 leg above, NVDEC is not enabled here: NVIDIA ships
+          # libnvcuvid for x86_64 only, so there is nothing for the CUDA env vars
+          # or the auditwheel exclusions to act on.
+          CIBW_SKIP: "*-manylinux_i686 *-musllinux_*"
+          CIBW_ARCHS: "aarch64"
       - name: Build wheels for CPython
         if: matrix.os == 'windows-2022'
         shell: bash

build_third_party_libs/install_manylinux_deps.sh

@@ -22,6 +22,9 @@ sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /
 
 # Install system deps
 # Toolchains
+# EPEL provides gtest-devel and xxhash-devel on manylinux2014_aarch64;
+# already enabled on the x86_64 image, so this is idempotent there.
+yum install -y epel-release
 yum install -y cmake git gtest-devel \
     lz4-devel libzstd-devel xxhash-devel libpng-devel libjpeg-turbo-devel wget
 
@@ -34,9 +37,12 @@ cd /tmp; git clone https://github.com/fmtlib/fmt.git -b 10.2.1 \
     && cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_POSITION_INDEPENDENT_CODE=TRUE -DFMT_TEST=OFF .; make -j$thread install; rm -rf /tmp/fmt;
 
 # Build lib jpeg turbo
+# libjpeg-turbo 3.0.1's cmake_minimum_required predates CMake 4.x's removal
+# of pre-3.5 policy compatibility; the manylinux2014 image (both arches) now
+# ships CMake 4.4+, so we opt into the CMake-suggested policy floor.
 cd /tmp && git clone https://github.com/libjpeg-turbo/libjpeg-turbo.git -b 3.0.1 \
     && cd libjpeg-turbo \
-    && cmake -DCMAKE_BUILD_TYPE=Release -DWITH_JPEG8=1 -DCMAKE_INSTALL_DEFAULT_PREFIX=/usr .;make -j$thread install; rm -rf /tmp/libjpeg-turbo;
+    && cmake -DCMAKE_BUILD_TYPE=Release -DWITH_JPEG8=1 -DCMAKE_INSTALL_DEFAULT_PREFIX=/usr -DCMAKE_POLICY_VERSION_MINIMUM=3.5 .;make -j$thread install; rm -rf /tmp/libjpeg-turbo;
 
 # Build Boost
 cd /tmp && git clone --recursive https://github.com/boostorg/boost.git -b boost-1.84.0 \

setup.py

@@ -13,6 +13,7 @@
 # limitations under the License.
 
 import os
+import platform
 import re
 import subprocess
 import sys
@@ -64,6 +65,14 @@ def build_extension(self, ext):
             "-DCMAKE_POLICY_VERSION_MINIMUM=3.5",
         ]
 
+        # Ocean's NEON RGB-conversion code mixes signed/unsigned vector types
+        # in intrinsic calls (e.g. `int8x16_t` passed to `vget_low_u8`), which
+        # Ocean's Android/iOS builds accept because Clang is permissive by
+        # default. GCC on Linux aarch64 refuses the mismatch and requires
+        # `-flax-vector-conversions` to allow it, matching Clang's behavior.
+        if sys.platform.startswith("linux") and platform.machine() == "aarch64":
+            cmake_args.append("-DCMAKE_CXX_FLAGS=-flax-vector-conversions")
+
         # Note: env var uses "CUDA" (recognizable to users), CMake flag uses
         # "NVCODEC" (precise for build-system maintainers). Intentional.
         #