build(before-code-freeze): Export the stub's dynamic symbols when libpython is static - #451
Merged
mc-nv merged 2 commits intoAug 12, 2026
Conversation
The manylinux base container builds CPython with --disable-shared, so on that platform triton_python_backend_stub links libpython3.x.a and the Py_* symbols live in the executable rather than a shared library. Python C extension modules are dlopen'ed and deliberately leave Py_* undefined -- every one of the 77 lib-dynload modules in the image does, math.so alone with 63 undefined symbols and no libpython in DT_NEEDED -- so they can only bind against the stub's dynamic symbol table. Without the export the stub links and starts, then fails on the first `import math`. Set ENABLE_EXPORTS, which makes CMake add the platform's flag (-Wl,--export-dynamic on Linux, per Platform/Linux.cmake), matching how CPython links its own interpreter. Gate it on the resolved library being an archive rather than on the platform. A shared libpython needs no export, so the Debian build is unaffected, and the condition stays correct if a target moves between the two linkage modes. The condition reads PYTHON_LIBRARY, not PYTHON_LIBRARIES: pybind11 arrives through FetchContent_MakeAvailable, which uses add_subdirectory, so only the cache entry find_library creates crosses back into this scope. PYTHON_LIBRARIES is a plain variable in pybind11's directory and is empty here, which would have made this a silent no-op.
The comment said RHEL_BUILD existed because pybind would otherwise pick up Python 3.6 in the RHEL base container, so PYBIND11_PYTHON_VERSION was set to force 3.12. That pin has been dropped from build.py -- pybind11 prefers the newest entry of its own Python_ADDITIONAL_VERSIONS list, and the manylinux base container puts a single interpreter first on PATH. RHEL_BUILD itself stays: it still selects the stub's link settings further down. Only the justification was wrong.
This was referenced Aug 5, 2026
Merged
mc-nv
added
build
Contributor
Author
|
@greptileai review |
Greptile SummaryUpdates the stub link configuration so executables embedding a static libpython export their dynamic symbols, allowing Python extension modules to resolve
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains. Important Files Changed
Reviews (2): Last reviewed commit: "docs: Correct the stale RHEL_BUILD ratio..." | Re-trigger Greptile |
mc-nv
changed the title
mc-nv
marked this pull request as ready for review
August 5, 2026 23:31
yinggeh
approved these changes
Aug 12, 2026
mc-nv
added a commit
that referenced
this pull request
Sep 2, 2026
…python is static (#451) (#453) * build: Export the stub's dynamic symbols when libpython is static The manylinux base container builds CPython with --disable-shared, so on that platform triton_python_backend_stub links libpython3.x.a and the Py_* symbols live in the executable rather than a shared library. Python C extension modules are dlopen'ed and deliberately leave Py_* undefined -- every one of the 77 lib-dynload modules in the image does, math.so alone with 63 undefined symbols and no libpython in DT_NEEDED -- so they can only bind against the stub's dynamic symbol table. Without the export the stub links and starts, then fails on the first `import math`. Set ENABLE_EXPORTS, which makes CMake add the platform's flag (-Wl,--export-dynamic on Linux, per Platform/Linux.cmake), matching how CPython links its own interpreter. Gate it on the resolved library being an archive rather than on the platform. A shared libpython needs no export, so the Debian build is unaffected, and the condition stays correct if a target moves between the two linkage modes. The condition reads PYTHON_LIBRARY, not PYTHON_LIBRARIES: pybind11 arrives through FetchContent_MakeAvailable, which uses add_subdirectory, so only the cache entry find_library creates crosses back into this scope. PYTHON_LIBRARIES is a plain variable in pybind11's directory and is empty here, which would have made this a silent no-op. * docs: Correct the stale RHEL_BUILD rationale The comment said RHEL_BUILD existed because pybind would otherwise pick up Python 3.6 in the RHEL base container, so PYBIND11_PYTHON_VERSION was set to force 3.12. That pin has been dropped from build.py -- pybind11 prefers the newest entry of its own Python_ADDITIONAL_VERSIONS list, and the manylinux base container puts a single interpreter first on PATH. RHEL_BUILD itself stays: it still selects the stub's link settings further down. Only the justification was wrong. (cherry picked from commit 96d6021)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does the PR do?
Set
ENABLE_EXPORTSontriton-python-backend-stubwhen libpython resolves to astatic archive. The manylinux base builds CPython
--disable-shared, so thePy_*symbols live in the stub executable; the 77lib-dynload/*.somodules linkno libpython and carry undefined
Py_*(mathalone: 63), so without the exportthe stub links and starts, then fails on the first
import math.Checklist
<commit_type>: <Title>Commit Type:
Related PRs:
Where should the reviewer start?
The
if(PYTHON_LIBRARY MATCHES "\\.a$")block inCMakeLists.txt.Test plan:
Covered by the internal RHEL/manylinux pipeline for the 26.08 upstream bump.
Caveats:
Gated on the resolved library being an archive, not on the platform, so the Debian
build (shared libpython) is unaffected. Uses
PYTHON_LIBRARY, notPYTHON_LIBRARIES: pybind11 arrives viaFetchContent_MakeAvailable, which usesadd_subdirectory, so only thefind_librarycache entry crosses into this scope.Background
Part of the 26.08 "build against latest upstream container" work: the base image
moved to
cuda:13.4-devel-manylinux--26.08, which changed the default GCCtoolset, the Python layout and the bundled OpenSSL.
Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
- Resolves: TRI-1650
CI (internal): [#61271932](http://tritonserver.local/ci/pipelines/61271932)