VS Test Explorer integration; remove VS Code's Run mechanism (#504) - #505
Merged
Merged
Conversation
…504) Investigating issue #504 (VS Code test explorer vs. our Run CodeLens) confirmed VS's Run/Debug delegation to Test Explorer was already correct, but the pass/fail outcome API (ICodeLensTestInformationService) is internal/unsupported - decompiled and documented that finding instead of reflecting into it. In its place, added a "Show in Test Explorer" action (the public SyncTestFromCodeLens command) so users can reach the native pass/fail state without leaving the .feature file. Also fixed the Run/Hook/StepHooks CodeLens providers being tied at the same Priority, which left their left-to-right order on a Scenario line unspecified - now Run leads, matching VS's own TestStatusProvider convention. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…adation (VS, #504) Reflects into VS's internal ICodeLensTestInformationService (the same mechanism its own TestStatusProvider CodeLens uses) to show a pass/fail glyph on the Run CodeLens, since that outcome data has no public API. Reads the outcome by name rather than casting, so a reshaped/renamed enum degrades to "unrecognized" instead of throwing. Distinguishes a permanent shape change (TypeLoadException/ MissingMemberException from a failed reflection lookup - disable for the process's lifetime) from a transient failure (a dropped connection, or the outcome service not yet registered right after a fresh VS launch - log and retry on the next call). The first cut conflated the two and disabled itself permanently on any failure at all, which made the glyph go dark after every VS relaunch. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replaces the CodeLens-only "own execution, no TestController" design (Option 2) with a real vscode.TestController: .feature scenarios now get native Testing-sidebar presence, run/rerun, pass/fail history, and a failed-step marker via TestMessage.location - all previously hand-rolled through TestResultStore/ResultDecorationService (both deleted). Reconsidered because issue #504 confirmed live that C# Dev Kit's own test discovery already places a decoration on a Reqnroll-generated method's .feature-mapped location - the exact "no way to address a .feature line" premise the original Option 2 analysis rested on. Given that, owning a reliable TestController was judged worth the accepted risk of an occasional duplicate Testing-sidebar entry when C# Dev Kit's separately flaky/async discovery also fires for the same scenario. The item tree is still built lazily (document symbols only, no reqnroll/resolveTestTargets calls) to preserve issue #495's fix; the existing TRX-based dotnet test invocation and stdout step-trace parsing are reused as-is, just reported through vscode.TestRun instead of a custom decoration service. Debug remains deliberately unimplemented, matching the CodeLens this replaces - VSTEST_HOST_DEBUG + a debug-adapter attach needs its own spike. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Live testing of the new TestController against C# Dev Kit's own test items found three gaps. This fixes the first: - Output pane showed "The test run did not record any output" because nothing ever called TestRun.appendOutput() - VS Code doesn't populate that tab on its own. Now streams every result's captured stdout (Reqnroll's step trace) for both passed and failed runs, CRLF-normalized as the terminal-style Output view requires. The other two are recorded in the design doc rather than fixed here: Scenario Outline rows don't show as a per-row tree (the LSP server's Examples document-symbol node has no row-level children to build from yet - deferred, not a regression, the Outline still runs correctly) and an intermittent DLL-lock build failure needs no code change (MSBuild's own retry already covers it). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
) On reflection, neither the original CodeLens-based Run action nor the TestController it was migrated to add value that C# Dev Kit's own gutter/Test Explorer integration doesn't already provide for Reqnroll-generated methods (mapped back to .feature files via Reqnroll's #line pragmas). Removes the entire testRunner/ pipeline (dotnet test invocation, TRX parsing, test-filter building, the dotnet-CLI locator) along with the run/test command registrations and manifest entries - VS Code's Reqnroll extension now has no run mechanism of its own. VS and Rider are unaffected; this is VS-Code-specific. Design doc updated to record the full arc (Option 2 shipped -> TestController migration -> this reversal) so it isn't re-litigated from scratch if revisited later. Two CHANGELOG entries pre-dating this session also narrowed to drop their now-inaccurate "VS Code" attribution, since the code they described (the dotnet-CLI locator) no longer exists there. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
) Decompiled the actual pinned Rider 2024.3.5 distribution (via the devcontainer's Gradle cache, no live UI session needed) to find the exact plugin-API accessor the design doc had flagged as an open follow-up under the SMTestProxy hypothesis. Finding: that hypothesis was wrong. Rider's .NET unit testing isn't built on IntelliJ Platform's generic SMTRunnerEventsListener/ SMTestProxy at all - it's an entirely separate, RD-protocol-driven system (com.jetbrains.rider.unitTesting.* frontend UI synced from com.jetbrains.rider.model.RdUnitTest* backend-owned state). The JVM frontend can only fire Run/Debug signals on a session; every test element the tool window shows originates from the .NET ReSharperHost backend. There is no JVM-side extension point for a plugin to inject a synthetic test item. Decision: not pursuing native tool-window integration for Reqnroll scenarios. It would require a separate C#/ReSharperHost-SDK backend plugin project - a different architecture entirely from this repo's existing pure-Kotlin frontend plugin, and a materially larger scope than anything else done for #504. Rider keeps its existing CodeVisionProvider-based own-execution Run lens unchanged. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This was referenced Aug 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤔 What's changed?
Visual Studio — the Run CodeLens now fully integrates with VS's real Test Explorer:
RunTestOutcomeBridge) into VS's internal (unsupported) outcome-cache API — every failure mode is caught and permanently vs. transiently classified, so a future VS update can only ever degrade the glyph back to "none," never crash the CodeLens host.Priority, which left their left-to-right order on aScenario:line unspecified.VS Code — removed the Run CodeLens and its entire supporting pipeline (
dotnet testinvocation, TRX parsing, test-filter building, the dotnet-CLI locator) entirely. An ownedvscode.TestControllerwas tried and reverted along the way (see commit history / design doc for the full arc) — on reflection, neither it nor the original CodeLens added value beyond what C# Dev Kit's own gutter/Test Explorer integration already provides for Reqnroll-generated methods (mapped back to.featurefiles via Reqnroll's#linepragmas). VS Code's Reqnroll extension now has no run mechanism of its own.Rider — research spike only, no code change. Decompiled the actual pinned Rider 2024.3.5 distribution to find the plugin-API accessor the design doc had flagged as an open follow-up. Finding: the design doc's
SMTestProxyhypothesis was wrong — Rider's .NET unit testing has no JVM-side extension point at all; native tool-window integration would need a separate C#/ReSharperHost-SDK backend plugin, a materially larger project than anything else done for this issue. Not pursued; Rider's existingCodeVisionProvider-based Run lens is unchanged.⚡️ What's your motivation?
Issue #504 started as "does VS Code's own gutter/Test Explorer integration make our custom Run CodeLens redundant?" Investigating that question across all three IDEs surfaced follow-on work in each: VS's Run/Debug delegation had an easy win (Test Explorer sync + pass/fail glyph) worth shipping; VS Code's answer ended up being "yes, defer to C# Dev Kit, remove our own mechanism"; Rider's answer was "no native integration path exists without a much bigger separate project."
Fixes #504.
🏷️ What kind of change is this?
🧩 Area(s) touched
src/VisualStudio)src/VSCode)♻️ Anything particular you want feedback on?
RunTestOutcomeBridge) intentionally reaches into an unsupported internal VS API — reviewed and accepted the risk, but flagging for visibility since it's an unusual pattern for this codebase.📋 Checklist:
docs/docs/AsBuilt-Reconciliation-Reminder.md).