Skip to content

Commit 252c795

Browse files
authored
Migrate the Test Explorer to use FSAutoComplete / VSTest (#2106)
* Demonstrate basic test discvoery using language server * Use new test discovery in all places the old cli-based discovery was used Also refactor toward simpler swappability of the two approaches * Primative merge of code locations on test discovery * Implement incremental test discovery updates * Run all tests using the language server Does not currently work for partial test runs. Also doesn't work if there are MSTest theory tests due to discrepancy between test discovery and test results * Improve discovery progress reporting * Fix bug when merging theory test results to the explorer * Match Xunit theory case nesting to Visual Studio's * Demonstrate basic streaming of test results from langauge server * Add incremental test running indicators to language server test runs * Add testCaseFilter support to test Runs Still has a bug where NUnit doesn't respect the filter, but works for all other frameworks * Draft: Attempt to attach debugger on test run. Currently hanging on the server side for unclear reasons * Use vstest environment variable for skipping entry breakpoints instead of waiting and simulating a user continue command * Debug test runs using the language server * Forward test logs to the vscode test run output * Allow client to limit which test projects are run This was mainly motivated by test debugging. Before this feature, the debugger would launch for every test project even if you only wanted to debug one test. With this change, it'll only launch the one project. This also partially mitigates the bug where NUnit doesn't respect filters, now those test won't show unless some tests from an nunit project are selected * Show a warning if test explorer refresh doesn't find any tests * Forward test discovery logs to the test adapter logs for improved error diagnostics * Colorize log levels in the TestRun output * Log test outcomes to the TestRun output * Raise a warning if no test ran for a test run * Factor the language server discovery and run flows to clarify the top-level handlers Also enables experimentation with running discovery before a test run to make the explorer more robust for projects without live update support * Refactor mergeTestResultsToExplorer for smoother use with LSP test results The dotnet cli approach fundamentally expected tests to be run and merged by project, but VSTest doesn't. MergeTestResultsToExplorer really only needed the project and target framework for creating new items. By adding project and target framework to the each test result, we can merge test items without expecting them to be batched by project * Fix error when running parameterized test cases for xUnit or MSTest * Rediscover tests after a test run Test projects without code analysis updates (i.e C# projects or projects using Microsoft.Testing.Platform.VSTestBridge) will show new tests if you run a group of tests that changed, but those new tests will not have code locations. The lack of code locations in this limited case is very confusing and degrades the experience significantly. I looked at several options to fill in these locations. - VSTest does not return test locations on test runs, so mapping a location from results doesn't work - Running discovery before a test run caused test runs to feel sluggish. Especially when trying to run individual tests, which is a core workflow. I tried project-specific discovery and parallel discovery/test run, but still had an average ~1s delay. However, test discovery, even with thousands of tests, only takes a few seconds. That feels sluggish before the test run, but is inconsequential after the test run. The discovery is generally finished before the user has evaluated the test results. This pattern also has some precedent. Visual Studio will generally discover new tests in a group only after you run the group. Thanks for coming to my ted talk. * Improve error reporting when the test run throws an exception * Fix error when test names contain -- We currently store the test's FullyQualifiedName in the id and separate it from other id components with -- This led to incorrectly splitting the test name if it contained a -- It may be a good idea to dynamically add a field to TestItem for FullyQualifiedName similar to what was donen for TestFramework * Fix spelling error * Replace custom Option extension methods with builtin methods tryFallback -> Option.orElseWith tryFallbackValue -> Option.orElse * Error if unexpected TestOutcomeDTO value found. This should not happen. If it does, it means ionide and the language server are out of sync * Discover tests without building on initial workspace load If they've never built their solution, then we won't be able to load tests until they restore anyway. If they have built their solution, we probably don't need to build it again. It's much faster to discover tests without the build and, unlike clicking the refresh button, there is a much smaller chance they're looking for missing test or otherwise solving a weird explorer state where we'd want fresh build artifacts. * Fix bug where tests with the same name could conflict between projects IcedTasks experienced this issue when all tests were run. Merging tests results used to be per-project, but now is not. So the test result venn requires project scope data
1 parent 83e3961 commit 252c795

File tree

4 files changed

+818
-242
lines changed

4 files changed

+818
-242
lines changed

release/package.json

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -901,6 +901,11 @@
901901
"description": "Decides if the test explorer will automatically try discover tests when the workspace loads. You can still manually refresh the explorer to discover tests at any time",
902902
"type": "boolean"
903903
},
904+
"FSharp.TestExplorer.UseLegacyDotnetCliIntegration": {
905+
"default": false,
906+
"description": "Use the dotnet cli to discover and run tests instead of the language server. Will lose features like streamed test results and Microsoft Testing Platform support.",
907+
"type": "boolean"
908+
},
904909
"FSharp.trace.server": {
905910
"default": "off",
906911
"description": "Trace server messages at the LSP protocol level for diagnostics.",

0 commit comments

Comments
 (0)