The `--pytest-args` option in `colcon test` does not affect the commands executed

[sea-bass]

I'm running some tests with colcon test --pytest-with-coverage on rolling.

This automatically enables the options --cov-report html:<some path> and --cov-report xml:<some path>.

However, there is also a --cov-report term option that should display test results to console, which I want to add manually using the --pytest-args option.

If I do the following, the results are still not displayed:

colcon test --event-handlers console_direct+ --pytest-with-coverage --pytest-args " --cov-report term"

The equivalent command that is run, per the console logs, is:

1: -- run_test.py: invoking following command in '/home/sebastian/pyrobosim_ws/src/pyrobosim/pyrobosim_ros':
1:  - /home/sebastian/python-virtualenvs/pyrobosim/bin/python3.10 -u -m pytest
 /home/sebastian/pyrobosim_ws/src/pyrobosim/pyrobosim_ros/test/test_ros_conversions.py 
-o cache_dir=/home/sebastian/pyrobosim_ws/build/pyrobosim_ros/test/ament_cmake_pytest/test_ros_conversions/.cache 
--junit-xml=/home/sebastian/pyrobosim_ws/build/pyrobosim_ros/test_results/pyrobosim_ros/test_ros_conversions.xunit.xml 
--junit-prefix=pyrobosim_ros --cov=/home/sebastian/pyrobosim_ws/src/pyrobosim/pyrobosim_ros/test 
--cov-report=html:/home/sebastian/pyrobosim_ws/build/pyrobosim_ros/test/pytest_cov/test_ros_conversions/coverage.html 
--cov-report=xml:/home/sebastian/pyrobosim_ws/build/pyrobosim_ros/test/pytest_cov/test_ros_conversions/coverage.xml 
--cov-branch

As you see, the --cov-report term text is nowhere to be found in the above snippet.

I then realized if I add any arguments, like --pytest-args " --hello", not even that gets reflected.

Am I missing something, or does the --pytest-args option not currently work?

[sea-bass]

This may also be an issue with the ament_add_pytest_test function here: https://github.com/ament/ament_cmake/blob/rolling/ament_cmake_pytest/cmake/ament_add_pytest_test.cmake

[slim71]

Any news on this? I've noticed the same when trying to select a single test to run with colcon test --pytest args -k <test_name>

[cottsay]

@sea-bass is absolutely correct here. When ament_cmake wraps pytest, it doesn't give colcon (or anything else) a way to pass arbitrary arguments to the wrapped pytest invocation. The --pytest-args argument to colcon is intended for use in a python or ament_python package, not a cmake or ament_cmake package.

All that said, if ament_cmake_pytest were to expose a mechanism for passing these arbitrary arguments through to the invoked process, we could do some wiring between colcon-cmake and colcon-ros to pass through --pytest-args if specified. One major challenge here is that ament_cmake_pytest embeds the pytest command to a ctest file at build time, and --pytest-args is an argument to colcon's test verb, so we can't just introduce an extra CMake argument to ament_add_pytest_test.cmake, it would likely have to be an environment variable.

In any case, I'm going to move this ticket to colcon-ros. There isn't anything that can be done in colcon until ament_cmake provides a mechanism to achieve the desired behavior.

[zacharyyamaoka]

Is there any news on this? Thanks! I am also running into the same issue

[anton-matosov]

A workaround is to use environment variable PYTEST_ADDOPTS="extra args go here"

For your specific example:

colcon test --event-handlers console_direct+ --pytest-with-coverage --pytest-args " --cov-report term"

becomes

env PYTEST_ADDOPTS="--cov-report term" colcon test --event-handlers console_direct+ --pytest-with-coverage

Please note that -pytest-with-coverage wouldn't work as is either. See this stack overflow answer for the way to enable it in your cmake builds.