Bug: package discovery nested cmake project

[balthasarschuess]

In our buildsystem we encountered the bug, that the foxglove_msgs ament.cmake package was not discoverable by colon anymore after installing colcon-cargo and colcon-ros-cargo.
I suspect that this happens, because:

• the foxglove_msgs package is a nested package inside https://github.com/foxglove/foxglove-sdk
• but foxglove-sdk is also a rust project with a Cargo.toml on top level

[esteve]

I can confirm this bug in our own repo if we add a toplevel Cargo.toml file for a Cargo workspace (see ros2-rust/ros2_rust#502 (comment)). I believe the issue might be in https://github.com/colcon/colcon-cargo/blob/main/colcon_cargo/package_discovery/cargo_workspace.py#L28

[cottsay]

This is a bit tricky. Until #59, colcon had never supported packages residing in subdirectories of other packages. It isn't generally a behavior you'd want with CMake and it is explicitly disallowed for ROS package.xml. Cargo does allow other cargo packages to reside in subdirectories.

Continuing to recursively traverse the subdirectories of all packages is not really an option. There are several packages I've encountered which have CMakeLists.txt, setup.cfg, and package.xml files as templates in their sources and as resources used in testing, and we must not detect them as tests.

I don't have any great ideas about how to move forward here. What are you expecting to happen for your repositories? Would you like to see all of the cargo packages as well as the ROS package appear in colcon's package list? Or ignore the cargo packages and only detect the ROS package?

[esteve]

@cottsay I've implemented #64 , while not optimal, it gets the job done and we would be able to set up rclrs as a Cargo workspace, which in turn would allow ros2-rust/ros2_rust#502 to work. I very much prefer having a suboptimal solution that works for now and that supports a use case like Foxglove's and rclrs' in ros2-rust/ros2_rust#502

[balthasarschuess]

What are you expecting to happen for your repositories?

In our use case we build a ros workspace and require foxglove_msgs both in a ros2_rust based rust crate and rclpy /rclcpp based packages.
In our build pipeline we use vcstool to pull in all dependencies.
We don't actually depend on anything apart from the foxglove_msgs so we worked around it by manually cloning and copying only the relevant subdir to our ros workspace. Additionally, we avoided creating our own top level Cargo.toml.
However, having a top level has some drawback in the first place, because building with colcon generates Cargo.lock in the sub directory and we don't want it to compete with the rust workspace instance.

In the case of foxglove_msgs I would not have wanted to add the additional build steps, because the Cargo.toml on top level is:

• clearly not a ros package, but a rust workspace package
• the msgs_package is clearly a ros message package and only c++ based