Use CTest to help with CI testing - #413
Conversation
The executable targets to run in acpp CI workflow are currently derived from shell scripting, which is a relatively fragile way to things. This PR adds a ctest target for each of the executable created by the build, such that a user can easily run all the binaries by invoking `$ ctest` from the build directory. ctest was chosen as it has straightforward CMake integration, and I've tested it locally with both DPC++ and AdaptiveCPP builds of SYCL Academy. I've updated the GitHub workflow for AdaptiveCpp to use the new functionality, passing a regex exclude filter for the tests known to be failing.
b058dd7 to
8170718
Compare
There was a problem hiding this comment.
Great idea! My guess is that the failure in Matrix_transpose_source is because that file is a skeleton source code which in its current form submits an empty command group, which AdaptiveCpp would (rightfully) reject.
If we also want to CI-test the source skeletons then we might have to be a bit more careful with them since then even in their incomplete form they would need to be valid SYCL.
EDIT: Maybe it would be good if CI also printed the mentioned log file in /home/runner/work/syclacademy/syclacademy/build/Testing/Temporary/LastTest.log so that we can see what's going on, in case there's an issue?
Sources may not be valid SYCL
EwanC
changed the title
I thought it might be nice to test the lesson source executables too, but it's more scope creep than i'm prepared for in this PR to go change lesson sources to become valid SYCL, so I've just omitted them from being added as ctest targets. Good point about being able to see the |
illuhad
left a comment
illuhad
left a comment
There was a problem hiding this comment.
LGTM!
I thought it might be nice to test the lesson source executables too, but it's more scope creep than i'm prepared for in this PR to go change lesson sources to become valid SYCL, so I've just omitted them from being added as ctest targets.
Yeah, I kind of expected that it might be more work than it looks at first glance. I'm fine with this approach.
Good point about being able to see the LastTest.log, i've made the job print that file if ctest fails. You can see an example output of that in this action
Looks great :)
2 participants
The executable targets to run in acpp CI workflow are currently derived from shell scripting, which is a relatively fragile way to things.
This PR adds a ctest target for each of the executable created by the build, such that a user can easily run all the binaries by invoking
$ ctestfrom the build directory. ctest was chosen as it has straightforward CMake integration, and I've tested it locally with both DPC++ and AdaptiveCPP builds of SYCL Academy.I've updated the GitHub workflow for AdaptiveCpp to use the new functionality, passing a regex exclude filter for the tests known to be failing.