fix: reduce Lambda layer size by building Arrow without ICU (#3336)

* fix: reduce Lambda layer size by building Arrow without ICU (#3331)

Arrow 22 links against libicu when it is present in the build environment.
Dockerfile.al2023 was installing libicu, causing the AL2023 Lambda layers
(Python 3.12/3.13/3.14) to bundle libicudata.so.67 (~30 MB),
libicui18n.so.67, and libicuuc.so.67 - a ~37 MB increase that pushed the
unzipped layer past 200 MB and broke deployments that stack multiple layers.

The AL2 Dockerfile (Python 3.9-3.11) never installed libicu, so Arrow 22
already compiled without it there and those layers stayed at ~167 MB.

Fix:
- Remove libicu from Dockerfile.al2023 dnf install.
- Add -DARROW_WITH_ICU=OFF to the Arrow cmake invocation.
- Drop libicudata.so.67, libicui18n.so.67, libicuuc.so.67 from the bundle loop.
- Strip the bundled lib/*.so files to recover a few extra MB.

Expected layer size after this fix: ~167 MB (back to pre-3.16.1 levels).

Co-Authored-By: Bujji Babu Katta & Srihari Ponakala

* Strip debug info from shared libraries

Added a step to strip symbol tables and debug info from shared libraries to reduce binary size before zipping.

---------

Co-authored-by: S4srihari <srihariponakala@gmail.com>
Co-authored-by: Anton Kukushkin <kukushkin.anton@gmail.com>

Author: 3 people

building/lambda/Dockerfile.al2023

@@ -8,7 +8,6 @@ RUN dnf install -y \
     jemalloc-devel \
     libxml2-devel \
     libxslt-devel \
-    libicu \
     libatomic \
     bison \
     make \

building/lambda/build-lambda-layer.sh

@@ -47,6 +47,7 @@ cmake \
     -DARROW_WITH_LZ4=OFF \
     -DARROW_WITH_BROTLI=OFF \
     -DARROW_BUILD_TESTS=OFF \
+    -DARROW_WITH_ICU=OFF \
     -GNinja \
     ..
 
@@ -110,8 +111,10 @@ find python -regex '^.*\(__pycache__\|\.py[co]\)$' -delete
 # libatomic for pyarrow 22+). Lambda extracts layers to /opt/ and /opt/lib
 # is on LD_LIBRARY_PATH. Search ldconfig cache first, then fall back to a
 # filesystem search (libatomic under gcc10 lives in /usr/lib/gcc/*).
+# ICU libraries are intentionally excluded: Arrow is built with -DARROW_WITH_ICU=OFF
+# so pyarrow has no ICU dependency, keeping the layer size minimal.
 mkdir -p lib
-for libfile in libxslt.so.1 libexslt.so.0 libatomic.so.1 libicudata.so.67 libicui18n.so.67 libicuuc.so.67; do
+for libfile in libxslt.so.1 libexslt.so.0 libatomic.so.1; do
   src=$(ldconfig -p 2>/dev/null | awk -v lib="${libfile}" '$1 == lib { print $NF; exit }')
   if [ -z "${src}" ] || [ ! -e "${src}" ]; then
     src=$(find /usr/lib /usr/lib64 -name "${libfile}" -print -quit 2>/dev/null)
@@ -124,6 +127,9 @@ for libfile in libxslt.so.1 libexslt.so.0 libatomic.so.1 libicudata.so.67 libicu
   fi
 done
 
+# Strip symbol tables and debug info to reduce binary size
+find lib -name '*.so*' -type f -exec strip "{}" \;
+
 zip -r9 "${FILENAME}" ./python ./lib
 mv "${FILENAME}" dist/