Skip to content

Commit 103a168

Browse files
committed
build(before-code-freeze): Export the stub's dynamic symbols when libpython is static (#451)
* build: Export the stub's dynamic symbols when libpython is static The manylinux base container builds CPython with --disable-shared, so on that platform triton_python_backend_stub links libpython3.x.a and the Py_* symbols live in the executable rather than a shared library. Python C extension modules are dlopen'ed and deliberately leave Py_* undefined -- every one of the 77 lib-dynload modules in the image does, math.so alone with 63 undefined symbols and no libpython in DT_NEEDED -- so they can only bind against the stub's dynamic symbol table. Without the export the stub links and starts, then fails on the first `import math`. Set ENABLE_EXPORTS, which makes CMake add the platform's flag (-Wl,--export-dynamic on Linux, per Platform/Linux.cmake), matching how CPython links its own interpreter. Gate it on the resolved library being an archive rather than on the platform. A shared libpython needs no export, so the Debian build is unaffected, and the condition stays correct if a target moves between the two linkage modes. The condition reads PYTHON_LIBRARY, not PYTHON_LIBRARIES: pybind11 arrives through FetchContent_MakeAvailable, which uses add_subdirectory, so only the cache entry find_library creates crosses back into this scope. PYTHON_LIBRARIES is a plain variable in pybind11's directory and is empty here, which would have made this a silent no-op. * docs: Correct the stale RHEL_BUILD rationale The comment said RHEL_BUILD existed because pybind would otherwise pick up Python 3.6 in the RHEL base container, so PYBIND11_PYTHON_VERSION was set to force 3.12. That pin has been dropped from build.py -- pybind11 prefers the newest entry of its own Python_ADDITIONAL_VERSIONS list, and the manylinux base container puts a single interpreter first on PATH. RHEL_BUILD itself stays: it still selects the stub's link settings further down. Only the justification was wrong. (cherry picked from commit 96d6021)
1 parent 35a3c13 commit 103a168

1 file changed

Lines changed: 34 additions & 4 deletions

File tree

CMakeLists.txt

Lines changed: 34 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Copyright 2020-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
1+
# Copyright 2020-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
22
#
33
# Redistribution and use in source and binary forms, with or without
44
# modification, are permitted provided that the following conditions
@@ -93,9 +93,15 @@ FetchContent_Declare(
9393
GIT_SHALLOW ON
9494
)
9595

96-
# RHEL base container has multiple version of Python installed. By default
97-
# it seems like pybind will pickup v3.6, so we specifically assign it to
98-
# search for 3.12 here.
96+
# Detect a RHEL-family build. Used further down to adjust how the stub
97+
# executable is linked.
98+
#
99+
# This used to also justify passing PYBIND11_PYTHON_VERSION, because pybind
100+
# would otherwise pick up an older interpreter present in the RHEL base
101+
# container. That is no longer needed: pybind11 prefers the newest entry of
102+
# its own Python_ADDITIONAL_VERSIONS list, and the manylinux base container
103+
# puts a single interpreter first on PATH. Set PYBIND11_PYTHON_VERSION
104+
# explicitly if a build ever has to target a specific version again.
99105
set(RHEL_BUILD OFF)
100106
if(LINUX)
101107
file(STRINGS "/etc/os-release" DISTRO_ID_LIKE REGEX "ID_LIKE")
@@ -285,6 +291,30 @@ target_compile_options(
285291
)
286292
target_compile_definitions(triton-python-backend-stub PRIVATE TRITON_PB_STUB)
287293

294+
# When libpython is a static archive -- the manylinux base container builds
295+
# CPython with --disable-shared -- the Py_* symbols land inside the stub
296+
# executable instead of a shared library. Python C extension modules are
297+
# dlopen'ed and deliberately leave Py_* undefined, so they can only resolve
298+
# against the stub's dynamic symbol table. ENABLE_EXPORTS makes CMake add the
299+
# platform's flag for that (-Wl,--export-dynamic on Linux), the same way
300+
# CPython links its own interpreter binary.
301+
#
302+
# A shared libpython needs none of this, since modules bind to it directly.
303+
# Key off the linkage rather than the platform so this stays correct if a
304+
# target ever moves between the two.
305+
#
306+
# PYTHON_LIBRARY rather than PYTHON_LIBRARIES: pybind11 is pulled in with
307+
# FetchContent_MakeAvailable, which uses add_subdirectory, so only the cache
308+
# entry that find_library creates survives into this scope. PYTHON_LIBRARIES
309+
# is a plain variable set inside pybind11's directory and reads empty here.
310+
if(PYTHON_LIBRARY MATCHES "\\.a$")
311+
set_target_properties(
312+
triton-python-backend-stub
313+
PROPERTIES
314+
ENABLE_EXPORTS TRUE
315+
)
316+
endif()
317+
288318
# RHEL assets are not released in a container environment nor do the current
289319
# Python lib versions in the manylinux base container match those currently
290320
# available for RHEL8 package managers. Therefore, we package the correct

0 commit comments

Comments
 (0)