Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .clang-tidy
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"Checks": "-*,bugprone-*,cert-*,clang-analyzer-*,cppcoreguidelines-*,hicpp-*,misc-*,modernize-*,performance-*,portability-*,readability-*,-modernize-use-nullptr,-hicpp-use-nullptr",
"Checks": "-*,bugprone-*,cert-*,clang-analyzer-*,clang-diagnostic-*,cppcoreguidelines-*,hicpp-*,misc-*,modernize-*,performance-*,portability-*,readability-*,-modernize-use-nullptr,-hicpp-use-nullptr",
"HeaderFilterRegex": ".*",
"ExcludeHeaderFilterRegex": "boost/.*",
"WarningsAsErrors": "*",
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,4 @@ CMakeUserPresets.json
/.vs
out/
**/cppcheck-checkers.report
/infer-out
2 changes: 1 addition & 1 deletion CMakePresets.json
Original file line number Diff line number Diff line change
Expand Up @@ -363,7 +363,7 @@
"vendor": {
"microsoft.com/VisualStudioSettings/CMake/1.0": {
"intelliSenseMode": "windows-clangcl-x86",
"clangTidyChecks": "-*,bugprone-*,cert-*,clang-analyzer-*,cppcoreguidelines-*,hicpp-*,misc-*,modernize-*,performance-*,portability-*,readability-*,-modernize-use-nullptr,-hicpp-use-nullptr",
"clangTidyChecks": "-*,bugprone-*,cert-*,clang-analyzer-*,clang-diagnostic-*,cppcoreguidelines-*,hicpp-*,misc-*,modernize-*,performance-*,portability-*,readability-*,-modernize-use-nullptr,-hicpp-use-nullptr",
"enableClangTidyCodeAnalysis": true
}
}
Expand Down
130 changes: 114 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,20 @@ The library is most useful for educational projects or small
For testing more complex projects I would recommend using
[GoogleTest](https://github.com/google/googletest).

The code is ~2k lines.
The code is ~1.2k lines.
The Release builds are optimized for speed over size, so the
build could probably be tweaked to get a smaller binary if
desired.

# Features

See the various Milestones in GitHub to get a feel for where the
project is headed and what features are planned for the future.

Here are the lists of current and planned features.

## Current Features

Features of the test framework are very minimal and include:
- Test case definition
- Test suite definition
Expand All @@ -49,6 +56,72 @@ Features of the test framework are very minimal and include:
or failing in certain cases in a managed way that reduces
boilerplate.

## Planned Features

There is no plan for advanced features like mocking or death tests.
The goal is to keep the library simple and easy to use, and to delegate
more complex testing to more advanced libraries like GoogleTest, and
more advanced features like mocking/faking to external libraries;
maybe this will change in the future, since this is a project of
exploration with no real constraints other than to maintain
simplicity (complex things can be simple).

The following features are currently planned:
- Test result output to files or stdout/stderr in a variety of formats
- JUnit XML
- xUnit XML
- CRTF JSON
- TAP (Become a TAP Producer)
- Maven Surefire XML
- Additional assertions, mostly convenient extensions of the current
assertions
- Numerical Analysis Assertions
- assertLessThan
- assertGreaterThan
- assertLessThanOrEqual
- assertGreaterThanOrEqual
- assertIsEven
- assertIsOdd
- assertIsPrime
- assertIsNotPrime
- assertIsComposite
- assertIsNotComposite
- assertIsPerfect
- assertIsNotPerfect
- assertIsPowerOf
- assertIsNotPowerOf
- assertIsDivisorOf
- assertIsNotDivisorOf
- assertIsMultipleOf
- assertIsNotMultipleOf
- assertIsFactorOf
- assertIsNotFactorOf
- assertIsDivisibleBy
- assertIsNotDivisibleBy
- Range Assertions - Dates, Times, Durations, Numbers, strings,
custom-defined ranges of varying types
- assertWithinRange
- assertNotWithinRange
- Container/string Assertions
- assertEmpty
- assertNotEmpty
- assertContains
- assertNotContains
- assertStartsWith
- assertEndsWith
- assertSizeIs
- String-only Assertions
- assertMatches
- assertNotMatches
- Miscellaneous Assertions
- assertIsOneOf
- assertIsNotOneOf
- Will apply lexicographically or numerically depending on the
type of the arguments, with parameters to control what "close"
means
- assertIsCloseTo
- assertIsNotCloseTo

# Getting Started

## Acquiring the Library
Expand All @@ -74,28 +147,26 @@ Then add it to your test executable target through

### vcpkg

*In process, not working yet*
*Will be available for 1.5 release*

As of 0.1.0-beta.1, you might soon be able to also include it
in your `vcpkg` project by running the following in the root of
your project (pending vcpkg PR approval that I have in right
now https://github.com/microsoft/vcpkg/pull/37471):
You be able to include TestCPP in your `vcpkg` project by running the
following in the root of your project (pending vcpkg PR approval that
I have in right now https://github.com/microsoft/vcpkg/pull/37471):
```
vcpkg add port testcpp
vcpkg add port eljonny-testcpp
```

### Conan

*In process, not working yet*
*Will be available for 2.0 release*

As of 0.1.1-beta.2, you might soon be able to also include it
in your `conan` project (pending approval from the Conan
community).
You will be able to include it in your `conan` project (pending
approval from the Conan community).
First add it as a dependency to your project in conanfile.txt:
```
[requires]
...
testcpp/0.1.1-beta.2
testcpp/2.0
```

Then installing TestCPP into your project by running the
Expand Down Expand Up @@ -224,7 +295,7 @@ There are a number of CMake Presets defined in CMakePresets.json that
align with different build configurations and analysis profiles,
including all build feature variations (with/without the Demo project,
with/without the tests, and subsequently with/without stacktrace
support via Boost.StackTrace).
support via Boost.StackTrace, and finally Debug and Release).

To get started, open the root project folder in Visual Studio 2022 and
select the desired CMake Preset from the Build Configurations dropdown
Expand Down Expand Up @@ -403,9 +474,11 @@ Both Release and Debug configurations support building with stack trace

# Static Analysis of TestCPP

You can run cppcheck and clang-tidy on the generated
You can run cppcheck, infer, and clang-tidy on the generated
compile_commands.json.

Another tool that is helpful to run is flawfinder.

For clang-tidy, you can also use the CMake flag to have CMake run
clang-tidy during the build.

Expand Down Expand Up @@ -452,6 +525,31 @@ My suggested options for cppcheck are:
I would recommend against using the -j parameter with cppcheck because
it disables certain checks that are used in the project.

For infer, you can run the following command after configuring the
build:
```
infer run --compilation-database path/to/compile_commands.json
```

For flawfinder, you can run the following command to analyze src and
then include and generate HTML reports about the results in the:
```
flawfinder --minlevel=0 --html --context --columns src > out/report/flawfinder/src-flaws.html
flawfinder --minlevel=0 --html --context --columns include > out/report/flawfinder/i-flaws.html
```

For a concise report on the command line, you can run the following
commands:
```
flawfinder --minlevel=0 --dataonly --quiet --columns src
flawfinder --minlevel=0 --dataonly --quiet --columns include
```

I would also recommend using the following flawfinder arguments:
- --error-level=0
- Sets the error level to 0, which is the lowest level of error
reporting; this is essentially all-warnings-as-errors.

# Testing and Code Coverage

The library is tested using itself.
Expand Down Expand Up @@ -582,8 +680,8 @@ The workflows are as follows:
- Builds and tests the library on Linux with stack traces enabled and
generates code coverage reports that are then pushed to CodeCov for
helpful visualizations and reporting.
- codeql.yml
- Runs the CodeQL static analysis tool on the library code.
- cmake-static-analysis.yml
- Runs clang-tidy and cppcheck on the library code using
JacobDomagala/StaticAnalysis@master
- codeql.yml
- Runs the CodeQL static analysis tool on the library code.
6 changes: 6 additions & 0 deletions cmake/Includes.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,10 @@ if (BUILD_TESTING)
test/include
include
)

target_include_directories (
${PROJECT_NAME}_Exceptions_test PRIVATE
test/include
include
)
endif ()
8 changes: 7 additions & 1 deletion cmake/Targets.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,13 @@ if (BUILD_TESTING)

add_executable (
${PROJECT_NAME}_Assertions_test
test/src/Assertions/AssertionsTests.cpp
test/src/Assertions/BasicAssertionsTests.cpp
test/src/TestCPPAssertionsMain.cpp
)

add_executable (
${PROJECT_NAME}_Exceptions_test
test/src/Exceptions/ExceptionsTests.cpp
test/src/TestCPPExceptionsMain.cpp
)
endif ()
5 changes: 5 additions & 0 deletions cmake/Testing.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,9 @@ if (BUILD_TESTING)
NAME ${PROJECT_NAME}AssertionsTests
COMMAND ${PROJECT_NAME}_Assertions_test
)

add_test (
NAME ${PROJECT_NAME}ExceptionsTests
COMMAND ${PROJECT_NAME}_Exceptions_test
)
endif ()
5 changes: 5 additions & 0 deletions cmake/build/DebugCompileDefs.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,9 @@ if (BUILD_TESTING)
PUBLIC
DEBUG_LOG
)
target_compile_definitions (
${PROJECT_NAME}_Exceptions_test
PUBLIC
DEBUG_LOG
)
endif ()
6 changes: 6 additions & 0 deletions cmake/build/GCCClangDebug.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -103,5 +103,11 @@ else ()
PUBLIC
${GCC_CLANG_DEBUG_BUILD_OPTS}
)

target_compile_options (
${PROJECT_NAME}_Exceptions_test
PUBLIC
${GCC_CLANG_DEBUG_BUILD_OPTS}
)
endif ()
endif ()
6 changes: 6 additions & 0 deletions cmake/build/GCCClangRelease.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -96,4 +96,10 @@ if (BUILD_TESTING)
PUBLIC
${GCC_CLANG_RELEASE_BUILD_OPTS}
)

target_compile_options (
${PROJECT_NAME}_Exceptions_test
PUBLIC
${GCC_CLANG_RELEASE_BUILD_OPTS}
)
endif ()
6 changes: 6 additions & 0 deletions cmake/build/GCCCoverage.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -49,3 +49,9 @@ target_compile_options (
PUBLIC
${COVERAGE_BUILD_OPTS}
)

target_compile_options (
${PROJECT_NAME}_Exceptions_test
PUBLIC
${COVERAGE_BUILD_OPTS}
)
15 changes: 6 additions & 9 deletions cmake/build/MSVCDebug.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -51,15 +51,6 @@ if ("${CMAKE_CXX_FLAGS}" MATCHES "/analyze:ruleset")
)
endif ()

if (BUILD_TESTING)
list (
APPEND
MSVC_DEBUG_BUILD_OPTS
/wd4514 # It's ok if the compiler removes unreferenced inline
# functions.
)
endif ()

target_compile_options (
${PROJECT_NAME}
PUBLIC
Expand Down Expand Up @@ -92,4 +83,10 @@ if (BUILD_TESTING)
PUBLIC
${MSVC_DEBUG_BUILD_OPTS}
)

target_compile_options (
${PROJECT_NAME}_Exceptions_test
PUBLIC
${MSVC_DEBUG_BUILD_OPTS}
)
endif ()
15 changes: 6 additions & 9 deletions cmake/build/MSVCRelease.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -41,15 +41,6 @@ if (${TESTCPP_STACKTRACE_ENABLED})
)
endif ()

if (BUILD_TESTING)
list (
APPEND
MSVC_RELEASE_BUILD_OPTS
/wd4514 # It's ok if the compiler removes unreferenced inline
# functions.
)
endif ()

target_compile_options (
${PROJECT_NAME}
PUBLIC
Expand Down Expand Up @@ -82,6 +73,12 @@ if (BUILD_TESTING)
PUBLIC
${MSVC_RELEASE_BUILD_OPTS}
)

target_compile_options (
${PROJECT_NAME}_Exceptions_test
PUBLIC
${MSVC_RELEASE_BUILD_OPTS}
)
endif ()

# This section removes the /RTC flags from the CMAKE_CXX_FLAGS_*
Expand Down
15 changes: 15 additions & 0 deletions cmake/link/Tests.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,12 @@ if (${TESTCPP_STACKTRACE_ENABLED} AND MSVC)
ole32
dbgeng
)
target_link_libraries (
${PROJECT_NAME}_Exceptions_test
${PROJECT_NAME}
ole32
dbgeng
)

elseif (${TESTCPP_STACKTRACE_ENABLED})
target_link_libraries (
Expand All @@ -34,6 +40,11 @@ elseif (${TESTCPP_STACKTRACE_ENABLED})
${PROJECT_NAME}
dl
)
target_link_libraries (
${PROJECT_NAME}_Exceptions_test
${PROJECT_NAME}
dl
)

else ()
target_link_libraries (
Expand All @@ -48,4 +59,8 @@ else ()
${PROJECT_NAME}_Assertions_test
${PROJECT_NAME}
)
target_link_libraries (
${PROJECT_NAME}_Exceptions_test
${PROJECT_NAME}
)
endif ()
Loading