Skip to content

Override bloom dh_shlibdeps behavior? Issues pip dependencies that bundle shared libraries #110

@villapx-path

Description

@villapx-path

I am using catkin_virtualenv 0.6.1 (this is the only version that's currently published for ROS Melodic and Noetic, see this issue), and some of my pip dependencies bundle shared libraries in them.

After successfully building my package with catkin_virtualenv as a dependency, I am attempting to use Bloom to create a rosdebian package:

bloom-generate rosdebian \
  --ros-distro melodic \
  --os-name ubuntu \
  --os-version bionic

debian/rules binary

The resulting debian/rules Makefile is throwing errors from dh_shlibdeps for all of my pip dependencies that bundle shared libraries, for example:

#24 42.65 dpkg-shlibdeps: error: cannot find library libgfortran-ed201abd.so.3.0.0 needed by debian/ros-melodic-redacted-pkg-name/opt/ros/melodic/share/redacted_pkg_name/venv/lib/python3.6/site-packages/numpy/.libs/libopenblasp-r0-34a18dc3.3.7.so (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')

But libgfortran-ed201abd.so.3.0.0 is correctly bundled in the numpy pip package:

# ls -l install/share/redacted_pkg_name/venv/lib/python3.6/site-packages/numpy/.libs/
total 30028
-rwxr-xr-x 1 root root  1023960 Dec  7 21:58 libgfortran-ed201abd.so.3.0.0
-rwxr-xr-x 1 root root 29724672 Dec  7 21:58 libopenblasp-r0-34a18dc3.3.7.so

I think what's tripping up dh_shlibdeps is the fact that pip doesn't publicly set an LD_LIBRARY_PATH in the venv nor an RPATH in the "consuming" shared lib (libopenblasp in this case), so if you ldd libopenblasp-r0-34a18dc3.3.7.so it will say that it can't find libgfortran.

But this is expected, as pip does its own internal shared library resolution at runtime.


What is the proper way to get around this in my CI pipeline? Basically in my bloom-generate call (or after it), I need to either turn off dh_shlibdeps, or at least tell it not to worry about shared libs bundled within the packages in the venv.

I feel like this has to be something folks have run into before when using catkin_virtualenv with Bloom, especially since many ROS applications work with sensor data and the like, and thus depend on pip packages that bundle shared libraries.

I appreciate the help!!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions