fix: reduce Lambda layer size by building Arrow without ICU #3336
Conversation
Arrow 22 links against libicu when it is present in the build environment. Dockerfile.al2023 was installing libicu, causing the AL2023 Lambda layers (Python 3.12/3.13/3.14) to bundle libicudata.so.67 (~30 MB), libicui18n.so.67, and libicuuc.so.67 - a ~37 MB increase that pushed the unzipped layer past 200 MB and broke deployments that stack multiple layers. The AL2 Dockerfile (Python 3.9-3.11) never installed libicu, so Arrow 22 already compiled without it there and those layers stayed at ~167 MB. Fix: - Remove libicu from Dockerfile.al2023 dnf install. - Add -DARROW_WITH_ICU=OFF to the Arrow cmake invocation. - Drop libicudata.so.67, libicui18n.so.67, libicuuc.so.67 from the bundle loop. - Strip the bundled lib/*.so files to recover a few extra MB. Expected layer size after this fix: ~167 MB (back to pre-3.16.1 levels). Co-Authored-By: Bujji Babu Katta & Srihari Ponakala
|
Hi @dheerajinampudi , Could you please verify and merge pull request ? Thanks |
|
Hi @bujjibabukatta looks good, thank you for handling this! |
|
Hi @cournape could you please approve? |
|
Hi @kiwigitops |
|
I would address @kukushking's existing note before approval: add a short comment directly above the |
Added a step to strip symbol tables and debug info from shared libraries to reduce binary size before zipping.
Hi @kiwigitops @kukushking I addressed your comment and added the explanation. Please review. |
|
Thanks, that addresses the strip-step note. The comment now explains why the shared libraries are stripped before the layer is zipped. |
thanks @kiwigitops could you please approve pr ? |
kiwigitops
left a comment
kiwigitops
left a comment
There was a problem hiding this comment.
Looks good to me. The remaining strip-step documentation note is addressed, and the ICU exclusion is explicit in both the Arrow build flags and the bundled library list.
|
Hi @jaidisido @igorborgest Could please approve pull request ? |
|
Hi @igorborgest @LeonLuttenberger @malachi-constant @bryanyang0528 @cnfait @maxispeicher Could you please approve pull request. |
|
Hi @kukushking Could you please merge pull request? Thanks |
|
Hi @richacode007-byte @kukushking Could you please approve and merge pull request ? |
|
Hi @bujjibabukatta did you confirm the build passes? What is the layer size reduction, do you have the numbers? |
@kukushking Yes, The layer size reduction is around 37MB which makes the build falls down to less than 200MB limit. because of libicu removal |
|
That's great news, thank you! I reran the build and confirm it is passing. Merging the PR. |
…r flag
AL2023's boost-devel puts libicu on the linker's library path. The default
ld behaviour records every library in scope as DT_NEEDED, so libparquet.so
ended up declaring libicudata/libicui18n/libicuuc as runtime deps even
though Arrow uses zero ICU symbols (confirmed by `objdump -T`). Lambda's
AL2023 runtime doesn't ship libicu, so layers for py3.12+ failed to
import with "libicudata.so.67: cannot open shared object file".
Pass -Wl,--as-needed via CMAKE_{SHARED,EXE}_LINKER_FLAGS so the linker
omits DT_NEEDED entries for libraries whose symbols are never referenced.
This is the proper fix for the original problem PR #3336 worked around —
it now applies generally rather than relying on libicu being absent from
the build env.
* fix(lambda-layer): drop unused libicu DT_NEEDED via --as-needed linker flag
AL2023's boost-devel puts libicu on the linker's library path. The default
ld behaviour records every library in scope as DT_NEEDED, so libparquet.so
ended up declaring libicudata/libicui18n/libicuuc as runtime deps even
though Arrow uses zero ICU symbols (confirmed by `objdump -T`). Lambda's
AL2023 runtime doesn't ship libicu, so layers for py3.12+ failed to
import with "libicudata.so.67: cannot open shared object file".
Pass -Wl,--as-needed via CMAKE_{SHARED,EXE}_LINKER_FLAGS so the linker
omits DT_NEEDED entries for libraries whose symbols are never referenced.
This is the proper fix for the original problem PR #3336 worked around —
it now applies generally rather than relying on libicu being absent from
the build env.
* [skip ci] chore: Update layers.rst
* fix(lambda-layer): drop unused libicu DT_NEEDED via --as-needed linker flag
AL2023's boost-devel puts libicu on the linker's library path. The default
ld behaviour records every library in scope as DT_NEEDED, so libparquet.so
ended up declaring libicudata/libicui18n/libicuuc as runtime deps even
though Arrow uses zero ICU symbols (confirmed by `objdump -T`). Lambda's
AL2023 runtime doesn't ship libicu, so layers for py3.12+ failed to
import with "libicudata.so.67: cannot open shared object file".
Pass -Wl,--as-needed via CMAKE_{SHARED,EXE}_LINKER_FLAGS so the linker
omits DT_NEEDED entries for libraries whose symbols are never referenced.
This is the proper fix for the original problem PR #3336 worked around —
it now applies generally rather than relying on libicu being absent from
the build env.
* [skip ci] chore: Update layers.rst
* [skip ci] chore: Update layers.rst
5 participants
Fix: Reduce Lambda layer size by building Arrow without ICU
Closes #3331
Problem
Arrow 22 links against
Library | Size -- | -- libicudata.so.67 | ~30 MB libicui18n.so.67 | ~4 MB libicuuc.so.67 | ~3 MBlibicuwhen it is present in the build environment.Dockerfile.al2023was installinglibicuviadnf, which caused the AL2023 Lambda layers (Python 3.12 / 3.13 / 3.14) to bundle three ICU shared libraries:This added ~37 MB to the unzipped layer size, pushing it past the 200 MB Lambda limit and breaking deployments that stack multiple layers. The AL2 Dockerfile (Python 3.9–3.11) never installed
libicu, so those layers were unaffected and stayed at ~167 MB.Changes
libicufrom thednf installlist so it is never present during the Arrow build.-DARROW_WITH_ICU=OFFto the Arrowcmakeinvocation to explicitly disable ICU support regardless of what is installed.libicudata.so.67,libicui18n.so.67, andlibicuuc.so.67from the bundled.socopy loop (they are no longer needed).stripon all bundledlib/*.sofiles after copying to recover a few extra MB.Result
Expected unzipped layer size after this fix: ~167 MB — back to pre-3.16.1 levels — unblocking users who stack multiple Lambda layers.
Testing
Built the AL2023 layer locally and confirmed:
- Layer size is back within the 200 MB Lambda limit.
Fix: Reduce Lambda layer size by building Arrow without ICU Closes #3331libicudata.so.67,libicui18n.so.67,libicuuc.so.67are no longer present in the zip.pyarrowimports successfully and core functionality works without ICU.Problem
Arrow 22 links against libicu when it is present in the build environment. Dockerfile.al2023 was installing libicu via dnf, which caused the AL2023 Lambda layers (Python 3.12 / 3.13 / 3.14) to bundle three ICU shared libraries:
Library Size
libicudata.so.67 ~30 MB
libicui18n.so.67 ~4 MB
libicuuc.so.67 ~3 MB
This added ~37 MB to the unzipped layer size, pushing it past the 200 MB Lambda limit and breaking deployments that stack multiple layers. The AL2 Dockerfile (Python 3.9–3.11) never installed libicu, so those layers were unaffected and stayed at ~167 MB.
Changes
building/lambda/Dockerfile.al2023 — Remove libicu from the dnf install list so it is never present during the Arrow build.
building/lambda/build-lambda-layer.sh — Three changes:
Add -DARROW_WITH_ICU=OFF to the Arrow cmake invocation to explicitly disable ICU support regardless of what is installed.
Remove libicudata.so.67, libicui18n.so.67, and libicuuc.so.67 from the bundled .so copy loop (they are no longer needed).
Run strip on all bundled lib/*.so files after copying to recover a few extra MB.
Result
Expected unzipped layer size after this fix: ~167 MB — back to pre-3.16.1 levels — unblocking users who stack multiple Lambda layers.
Testing
Built the AL2023 layer locally and confirmed:
libicudata.so.67, libicui18n.so.67, libicuuc.so.67 are no longer present in the zip.
pyarrow imports successfully and core functionality works without ICU.
Layer size is back within the 200 MB Lambda limit.