-
Notifications
You must be signed in to change notification settings - Fork 295
Description
Question
I built cFS with make SIMULATION=native ENABLE_UNIT_TESTS=true prep, and doing make test && make lcov tests and reports the coverage information as stated in the README.md. However, I expected the .gcda files to be generated after running and closing ./core-cpu1 (without running make test && make lcov) because I expected it to be compiled with gcc's --coverage option, but that didn't happen. I tested closing the application by sending SIGTERM, SIGINT and SIGABRT instead of pressing Ctrl+C, but there was nothing different.
What did make test && make lcov do, which I didn't? make test is passed 100%, so I'm presuming it's not a build issue. Is --coverage turned on for unit tests only, during compilation? If so, how do I measure the line coverage for an app or a module that I want to specify? I'm looking forward to your guidance.
Here's how I checked for .gcda files:
(base) user@host:/path/to/cFS$ find . -name "*.gcda"
<no output>
I checked for .gcno files and it seems like the --coverage worked:
user@host:/path/to/cFS$ find . -name "*.gcno"
./build/native/default_cpu1/config/ut-coverage/CMakeFiles/coverage-config-ALL-object.dir/__/fsw/src/cfe_config_get.c.gcno
./build/native/default_cpu1/config/ut-coverage/CMakeFiles/coverage-config-ALL-object.dir/__/fsw/src/cfe_config_lookup.c.gcno
./build/native/default_cpu1/config/ut-coverage/CMakeFiles/coverage-config-ALL-object.dir/__/fsw/src/cfe_config_set.c.gcno
./build/native/default_cpu1/config/ut-coverage/CMakeFiles/coverage-config-ALL-object.dir/__/fsw/src/cfe_config_init.c.gcno
./build/native/default_cpu1/osal/unit-test-coverage/vxworks/CMakeFiles/utobj_coverage-vxworks-tasks.dir/home/spacelaser/Desktop/llm-fuzz/cFS/osal/src/os/vxworks/src/os-impl-tasks.c.gcno
<output truncated>
References
From Gcov documentation:
The .gcno notes file is generated when the source file is compiled with the GCC -ftest-coverage option. It contains information to reconstruct the basic block graphs and assign source line numbers to blocks.
The .gcda count data file is generated when a program containing object files built with the GCC -fprofile-arcs option is executed. A separate .gcda file is created for each object file compiled with this option. It contains arc transition counts, value profile counts, and some summary information.
From the official cFS README.md:
... Unit tests can be added with `ENABLE_UNIT_TESTS=true` during the prep step, run with `make test`, and coverage reported with `make lcov`. Functional tests can be run by including `ENABLE_UNIT_TESTS=true` during prep and including the cfe_testcase app in the runtime app configuration (.scr file).
From build_options.cmake inside /path/to/cFS/osal/src/bsp/generic-linux/:
##########################################################################
#
# Build options for "generic-linux" BSP
#
##########################################################################
# C flags that should be used when (re-) compiling code for unit testing.
# Note: --coverage is just a shortcut for "-ftest-coverage" and "-fprofile-arcs"
# This also does not work well when cross compiling since paths to the _compile_ dir
# are baked into the executables, so they will not be there when copied to the target
# Note - although GCC understands the same flags for compile and link here, this may
# not be true on all platforms so the compile and link flags are specified separately.
if (NOT CMAKE_CROSSCOMPILING AND NOT OSAL_OMIT_DEPRECATED)
# The variables here (UT_COVERAGE_COMPILE_FLAGS/LINK_FLAGS) should be phased out, prefer
# to use the interface libraries (ut_coverage_compile/link) instead, which are more flexible.
set(UT_COVERAGE_COMPILE_FLAGS -pg --coverage)
set(UT_COVERAGE_LINK_FLAGS -pg --coverage)
endif()
From the Makefile inside /path/to/cFS/cfe/sample_defs/:
# Grab lcov baseline before running tests
test:
lcov --capture --initial --directory $(O)/$(ARCH) --output-file $(O)/$(ARCH)/coverage_base.info
(cd $(O)/$(ARCH) && ctest -O ctest.log)
lcov:
lcov --capture --rc lcov_branch_coverage=1 --directory $(O)/$(ARCH) --output-file $(O)/$(ARCH)/coverage_test.info
lcov --rc lcov_branch_coverage=1 --add-tracefile $(O)/$(ARCH)/coverage_base.info --add-tracefile $(O)/$(ARCH)/coverage_test.info --output-file $(O)/$(ARCH)/coverage_total.info
genhtml $(O)/$(ARCH)/coverage_total.info --branch-coverage --output-directory $(O)/$(ARCH)/lcov
@/bin/echo -e "\n\nCoverage Report Link: file:$(CURDIR)/$(O)/$(ARCH)/lcov/index.html\n"