Code coverage support for both ROS1 and ROS2

[Briancbn]

Feature Request

Environment variables to easily enable code coverage measurement in cpp, python or hybrid packages on both ROS1 and ROS2 distros.

Reason

There currently isn't a code coverage solution for ici, that can support both ROS1 and ROS2 on both python and cpp code base. There have been past discussion (#178, #112) on this issue that seems inactive for years now. There is also features included in PR #452 that enable the support for cpp lcov

Implementation Consideration

• ROS 1 side, I think this is a very good example , but we can integrate this into the CI shell script instead of having external dependencies.

• ROS 2 side, colcon mixin can be utilized to generate coverage for both cpp and python

• Code coverage reporting tools, I think both codecov.io and coveralls.io can be supported.

I currently have bandwidth to work on this, but please let me know if this feature is needed or you have any suggestions.

[rarrais]

Hi, following a discussion on this topic at the ROS-Industrial Community Meeting today, I wanted to share an effort that we are pursuing at our lab to address code coverage and code quality analysis in some of our ROS repositories.

https://github.com/rarrais/ros_coverage/

It builds on top of the industrial_ci methodology and makes use of the DOCKER_RUN_OPTS environment variable to invoke a script which outputs the coverage content to Codecov and to provide code analytics to Codacy.

We are currently actively using this methodology to monitor some of our internal ROS repos and also an ongoing open-source project, ROBIN (funded by the ROSIN project). Please check the banners at both repos for further details.

I had the objective of attempting to integrate this in industrial_ci at the time of development, but had to change focus for a while and this effort was postponed. It seems that @Briancbn is doing a great job at PR #504 , but perhaps some of the logic at the ros_coverage repo could be useful for integration. In particular, the integration with Codacy, which might be something that I might find some time to work on. Let me know what you think and congrats on the effort!

[Briancbn]

Thanks @rarrais, wonderful work there in ros_coverage. Thanks for the suggestions.
Corrected me if I am wrong, I listed down some of the useful features to incorporate here.

• Codacy integration
• Simultaneously upload reports to multiple reporting services
• Fail under threshold
• Python2 coverage support for ROS1 melodic

[agutenkunst]

I would love to have the "Fail under threshold" feature!