Fix ctest reporting success when requested tests match nothing (#4915)

CHANGELOG.md

@@ -23,6 +23,7 @@ Improvements:
 - Pass mandatory compiler arguments from `CMAKE_<LANG>_COMPILER` to cpptools so it can properly determine system include paths and built-in preprocessor macro definitions. Requires CMake 4.3 or newer. [#4627](https://github.com/microsoft/vscode-cmake-tools/pull/4627) [@cwalther](https://github.com/cwalther)
 
 Bug Fixes:
+- Fix a test run that matched no tests being reported as success. When specific tests are requested (e.g. programmatically via the API/Copilot) but none match a discovered test — for example an agent passing a partial or non-existent test name — CTest exits 0 ("No tests were found!!!"), which callers could misread as "all tests passed". Such runs now return a failure with a clear message when tests exist but the requested ones weren't found. [#4915](https://github.com/microsoft/vscode-cmake-tools/issues/4915)
 - Fix running a single test from the inline Test CodeLens or the project outline building the default (or all) target instead of the test's own executable; single-test runs now build only that test's target.
 - Fix test results showing a bare "The test case did not report any output." when a test did not actually run (for example when its executable was not built, so CTest reported it as "Not Run" or matched nothing). The test is now marked with an actionable message explaining that the executable may not have been built. CTest result parsing was also hardened so that compressed test output is decoded and a single test with missing or empty measurements no longer discards the results of the whole run. [#4451](https://github.com/microsoft/vscode-cmake-tools/issues/4451)
 - Fix test runs launched from the Test Explorer or the inline Test CodeLens appearing to run forever (the progress spinner never cleared) when a step in the run failed. The test run is now always finalized, even when building or running the tests throws, so the spinner stops and the failure is reported. Also, a single-test run no longer accidentally executes the entire suite when the full list of tests is not yet known. [#4451](https://github.com/microsoft/vscode-cmake-tools/issues/4451)

src/ctest.ts

@@ -591,8 +591,18 @@ export class CTestDriver implements vscode.Disposable {
         }
         const ctestArgs = await this.getCTestArgs(driver, customizedTask, testPreset, singleTestName) || [];
         if (testsToRun && testsToRun.length > 0) {
+            // If specific tests were requested but none of them match a discovered test, ctest
+            // would run with a filter that matches nothing and still exit 0 ("No tests were
+            // found!!!"), which callers (e.g. the API/Copilot) would misread as success. Only
+            // treat this as a failure when tests exist but the requested ones weren't found.
+            const availableTests = this.getTestNamesForSourceDir(driver.sourceDir) || [];
+            if (availableTests.length > 0 && !testsToRun.some(t => availableTests.includes(t))) {
+                const message = localize('ctest.no.matching.tests', 'None of the requested test(s) matched a discovered test: {0}. Nothing was run. Try refreshing the tests.', testsToRun.join(', '));
+                log.warning(message);
+                return { exitCode: -1, stdout: consumer?.stdout, stderr: message };
+            }
             ctestArgs.push("-R");
-            const superset = this.getTestNamesForSourceDir(driver.sourceDir) || [];
+            const superset = availableTests;
             const testsNamesRegex = getMinimalRegexFragments(superset, testsToRun).join('|');
             ctestArgs.push(testsNamesRegex);
         }
@@ -770,9 +780,17 @@ export class CTestDriver implements vscode.Disposable {
         await this.fillDriverMap(tests, run, cancellation, driverMap, driver, ctestPath, ctestArgs, testsToRun, customizedTask);
 
         if (testsToRun && testsToRun.length > 0 && driverMap.size === 0) {
-            // A specific set of tests was requested (e.g. from the inline CodeLens) but none of
-            // them matched a discovered test id. Surface this so a "nothing ran" outcome is not silent.
-            log.warning(localize('ctest.no.matching.tests', 'None of the requested test(s) matched a discovered test: {0}. Nothing was run. Try refreshing the tests.', testsToRun.join(', ')));
+            // A specific set of tests was requested (e.g. from the inline CodeLens or
+            // programmatically via the API/Copilot) but none of them matched a discovered test id.
+            // Surface this so a "nothing ran" outcome is not silently reported as success.
+            const message = localize('ctest.no.matching.tests', 'None of the requested test(s) matched a discovered test: {0}. Nothing was run. Try refreshing the tests.', testsToRun.join(', '));
+            log.warning(message);
+            // Only fail when tests exist but the requested ones weren't found. If no tests are
+            // discovered at all, don't force a failure here (a zero-test project is not this error).
+            const availableTests = this.getTestNames() ?? [];
+            if (availableTests.length > 0) {
+                return { exitCode: -1, stdout: consumer?.stdout, stderr: message };
+            }
         }
 
         if (!this.ws.config.ctestAllowParallelJobs) {