Skip to content

Commit 846e285

Browse files
saxbophonesaxbophone
authored
Merge pull request #14 from saxbophone/josh/use-catch2-cmake-integration
Use Catch2's CMake integration to enable auto-discovery of test cases
2 parents f037159 + 17d3dcd commit 846e285

File tree

5 files changed

+491
-2
lines changed

5 files changed

+491
-2
lines changed

‎.travis.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,4 +42,4 @@ before_script:
4242
- cmake -G "$CMAKE_GENERATOR" ..
4343
script:
4444
- cmake --build . -j
45-
- ctest -V
45+
- ctest

‎CMakeLists.txt‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,9 @@ if (MSVC)
7171
add_compile_options(/WX /W3) # treat all warnings as errors, set warning level 3
7272
endif()
7373

74+
# add custom dependencies directory
75+
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/cmake/Modules/")
76+
7477
# begin dependencies
7578

7679
# ...Add additional dependencies here...
@@ -109,7 +112,12 @@ set_target_properties(dengr-cli PROPERTIES OUTPUT_NAME dengr)
109112
add_executable(unit_tests ${TEST_SOURCES})
110113
target_link_libraries(unit_tests dengr)
111114
enable_testing()
112-
add_test(unit_tests unit_tests)
115+
116+
# auto-discover and add Catch2 tests from unit tests program
117+
include(CTest)
118+
include(Catch)
119+
120+
catch_discover_tests(unit_tests)
113121

114122
install(
115123
TARGETS dengr

‎cmake/Modules/Catch.cmake‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
# Distributed under the OSI-approved BSD 3-Clause License. See accompanying
2+
# file Copyright.txt or https://cmake.org/licensing for details.
3+
4+
#[=======================================================================[.rst:
5+
Catch
6+
-----
7+
8+
This module defines a function to help use the Catch test framework.
9+
10+
The :command:`catch_discover_tests` discovers tests by asking the compiled test
11+
executable to enumerate its tests. This does not require CMake to be re-run
12+
when tests change. However, it may not work in a cross-compiling environment,
13+
and setting test properties is less convenient.
14+
15+
This command is intended to replace use of :command:`add_test` to register
16+
tests, and will create a separate CTest test for each Catch test case. Note
17+
that this is in some cases less efficient, as common set-up and tear-down logic
18+
cannot be shared by multiple test cases executing in the same instance.
19+
However, it provides more fine-grained pass/fail information to CTest, which is
20+
usually considered as more beneficial. By default, the CTest test name is the
21+
same as the Catch name; see also ``TEST_PREFIX`` and ``TEST_SUFFIX``.
22+
23+
.. command:: catch_discover_tests
24+
25+
Automatically add tests with CTest by querying the compiled test executable
26+
for available tests::
27+
28+
catch_discover_tests(target
29+
[TEST_SPEC arg1...]
30+
[EXTRA_ARGS arg1...]
31+
[WORKING_DIRECTORY dir]
32+
[TEST_PREFIX prefix]
33+
[TEST_SUFFIX suffix]
34+
[PROPERTIES name1 value1...]
35+
[TEST_LIST var]
36+
)
37+
38+
``catch_discover_tests`` sets up a post-build command on the test executable
39+
that generates the list of tests by parsing the output from running the test
40+
with the ``--list-test-names-only`` argument. This ensures that the full
41+
list of tests is obtained. Since test discovery occurs at build time, it is
42+
not necessary to re-run CMake when the list of tests changes.
43+
However, it requires that :prop_tgt:`CROSSCOMPILING_EMULATOR` is properly set
44+
in order to function in a cross-compiling environment.
45+
46+
Additionally, setting properties on tests is somewhat less convenient, since
47+
the tests are not available at CMake time. Additional test properties may be
48+
assigned to the set of tests as a whole using the ``PROPERTIES`` option. If
49+
more fine-grained test control is needed, custom content may be provided
50+
through an external CTest script using the :prop_dir:`TEST_INCLUDE_FILES`
51+
directory property. The set of discovered tests is made accessible to such a
52+
script via the ``<target>_TESTS`` variable.
53+
54+
The options are:
55+
56+
``target``
57+
Specifies the Catch executable, which must be a known CMake executable
58+
target. CMake will substitute the location of the built executable when
59+
running the test.
60+
61+
``TEST_SPEC arg1...``
62+
Specifies test cases, wildcarded test cases, tags and tag expressions to
63+
pass to the Catch executable with the ``--list-test-names-only`` argument.
64+
65+
``EXTRA_ARGS arg1...``
66+
Any extra arguments to pass on the command line to each test case.
67+
68+
``WORKING_DIRECTORY dir``
69+
Specifies the directory in which to run the discovered test cases. If this
70+
option is not provided, the current binary directory is used.
71+
72+
``TEST_PREFIX prefix``
73+
Specifies a ``prefix`` to be prepended to the name of each discovered test
74+
case. This can be useful when the same test executable is being used in
75+
multiple calls to ``catch_discover_tests()`` but with different
76+
``TEST_SPEC`` or ``EXTRA_ARGS``.
77+
78+
``TEST_SUFFIX suffix``
79+
Similar to ``TEST_PREFIX`` except the ``suffix`` is appended to the name of
80+
every discovered test case. Both ``TEST_PREFIX`` and ``TEST_SUFFIX`` may
81+
be specified.
82+
83+
``PROPERTIES name1 value1...``
84+
Specifies additional properties to be set on all tests discovered by this
85+
invocation of ``catch_discover_tests``.
86+
87+
``TEST_LIST var``
88+
Make the list of tests available in the variable ``var``, rather than the
89+
default ``<target>_TESTS``. This can be useful when the same test
90+
executable is being used in multiple calls to ``catch_discover_tests()``.
91+
Note that this variable is only available in CTest.
92+
93+
#]=======================================================================]
94+
95+
#------------------------------------------------------------------------------
96+
function(catch_discover_tests TARGET)
97+
cmake_parse_arguments(
98+
""
99+
""
100+
"TEST_PREFIX;TEST_SUFFIX;WORKING_DIRECTORY;TEST_LIST"
101+
"TEST_SPEC;EXTRA_ARGS;PROPERTIES"
102+
${ARGN}
103+
)
104+
105+
if(NOT _WORKING_DIRECTORY)
106+
set(_WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}")
107+
endif()
108+
if(NOT _TEST_LIST)
109+
set(_TEST_LIST ${TARGET}_TESTS)
110+
endif()
111+
112+
## Generate a unique name based on the extra arguments
113+
string(SHA1 args_hash "${_TEST_SPEC} ${_EXTRA_ARGS}")
114+
string(SUBSTRING ${args_hash} 0 7 args_hash)
115+
116+
# Define rule to generate test list for aforementioned test executable
117+
set(ctest_include_file "${CMAKE_CURRENT_BINARY_DIR}/${TARGET}_include-${args_hash}.cmake")
118+
set(ctest_tests_file "${CMAKE_CURRENT_BINARY_DIR}/${TARGET}_tests-${args_hash}.cmake")
119+
get_property(crosscompiling_emulator
120+
TARGET ${TARGET}
121+
PROPERTY CROSSCOMPILING_EMULATOR
122+
)
123+
add_custom_command(
124+
TARGET ${TARGET} POST_BUILD
125+
BYPRODUCTS "${ctest_tests_file}"
126+
COMMAND "${CMAKE_COMMAND}"
127+
-D "TEST_TARGET=${TARGET}"
128+
-D "TEST_EXECUTABLE=$<TARGET_FILE:${TARGET}>"
129+
-D "TEST_EXECUTOR=${crosscompiling_emulator}"
130+
-D "TEST_WORKING_DIR=${_WORKING_DIRECTORY}"
131+
-D "TEST_SPEC=${_TEST_SPEC}"
132+
-D "TEST_EXTRA_ARGS=${_EXTRA_ARGS}"
133+
-D "TEST_PROPERTIES=${_PROPERTIES}"
134+
-D "TEST_PREFIX=${_TEST_PREFIX}"
135+
-D "TEST_SUFFIX=${_TEST_SUFFIX}"
136+
-D "TEST_LIST=${_TEST_LIST}"
137+
-D "CTEST_FILE=${ctest_tests_file}"
138+
-P "${_CATCH_DISCOVER_TESTS_SCRIPT}"
139+
VERBATIM
140+
)
141+
142+
file(WRITE "${ctest_include_file}"
143+
"if(EXISTS \"${ctest_tests_file}\")\n"
144+
" include(\"${ctest_tests_file}\")\n"
145+
"else()\n"
146+
" add_test(${TARGET}_NOT_BUILT-${args_hash} ${TARGET}_NOT_BUILT-${args_hash})\n"
147+
"endif()\n"
148+
)
149+
150+
if(NOT ${CMAKE_VERSION} VERSION_LESS "3.10.0")
151+
# Add discovered tests to directory TEST_INCLUDE_FILES
152+
set_property(DIRECTORY
153+
APPEND PROPERTY TEST_INCLUDE_FILES "${ctest_include_file}"
154+
)
155+
else()
156+
# Add discovered tests as directory TEST_INCLUDE_FILE if possible
157+
get_property(test_include_file_set DIRECTORY PROPERTY TEST_INCLUDE_FILE SET)
158+
if (NOT ${test_include_file_set})
159+
set_property(DIRECTORY
160+
PROPERTY TEST_INCLUDE_FILE "${ctest_include_file}"
161+
)
162+
else()
163+
message(FATAL_ERROR
164+
"Cannot set more than one TEST_INCLUDE_FILE"
165+
)
166+
endif()
167+
endif()
168+
169+
endfunction()
170+
171+
###############################################################################
172+
173+
set(_CATCH_DISCOVER_TESTS_SCRIPT
174+
${CMAKE_CURRENT_LIST_DIR}/CatchAddTests.cmake
175+
)

‎cmake/Modules/CatchAddTests.cmake‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Distributed under the OSI-approved BSD 3-Clause License. See accompanying
2+
# file Copyright.txt or https://cmake.org/licensing for details.
3+
4+
set(prefix "${TEST_PREFIX}")
5+
set(suffix "${TEST_SUFFIX}")
6+
set(spec ${TEST_SPEC})
7+
set(extra_args ${TEST_EXTRA_ARGS})
8+
set(properties ${TEST_PROPERTIES})
9+
set(script)
10+
set(suite)
11+
set(tests)
12+
13+
function(add_command NAME)
14+
set(_args "")
15+
foreach(_arg ${ARGN})
16+
if(_arg MATCHES "[^-./:a-zA-Z0-9_]")
17+
set(_args "${_args} [==[${_arg}]==]") # form a bracket_argument
18+
else()
19+
set(_args "${_args} ${_arg}")
20+
endif()
21+
endforeach()
22+
set(script "${script}${NAME}(${_args})\n" PARENT_SCOPE)
23+
endfunction()
24+
25+
# Run test executable to get list of available tests
26+
if(NOT EXISTS "${TEST_EXECUTABLE}")
27+
message(FATAL_ERROR
28+
"Specified test executable '${TEST_EXECUTABLE}' does not exist"
29+
)
30+
endif()
31+
execute_process(
32+
COMMAND ${TEST_EXECUTOR} "${TEST_EXECUTABLE}" ${spec} --list-test-names-only
33+
OUTPUT_VARIABLE output
34+
RESULT_VARIABLE result
35+
)
36+
# Catch --list-test-names-only reports the number of tests, so 0 is... surprising
37+
if(${result} EQUAL 0)
38+
message(WARNING
39+
"Test executable '${TEST_EXECUTABLE}' contains no tests!\n"
40+
)
41+
elseif(${result} LESS 0)
42+
message(FATAL_ERROR
43+
"Error running test executable '${TEST_EXECUTABLE}':\n"
44+
" Result: ${result}\n"
45+
" Output: ${output}\n"
46+
)
47+
endif()
48+
49+
string(REPLACE "\n" ";" output "${output}")
50+
51+
# Parse output
52+
foreach(line ${output})
53+
set(test ${line})
54+
# Escape characters in test case names that would be parsed by Catch2
55+
set(test_name ${test})
56+
foreach(char , [ ])
57+
string(REPLACE ${char} "\\${char}" test_name ${test_name})
58+
endforeach(char)
59+
# ...and add to script
60+
add_command(add_test
61+
"${prefix}${test}${suffix}"
62+
${TEST_EXECUTOR}
63+
"${TEST_EXECUTABLE}"
64+
"${test_name}"
65+
${extra_args}
66+
)
67+
add_command(set_tests_properties
68+
"${prefix}${test}${suffix}"
69+
PROPERTIES
70+
WORKING_DIRECTORY "${TEST_WORKING_DIR}"
71+
${properties}
72+
)
73+
list(APPEND tests "${prefix}${test}${suffix}")
74+
endforeach()
75+
76+
# Create a list of all discovered tests, which users may use to e.g. set
77+
# properties on the tests
78+
add_command(set ${TEST_LIST} ${tests})
79+
80+
# Write CTest script
81+
file(WRITE "${CTEST_FILE}" "${script}")

0 commit comments

Comments
 (0)