Resolve Run CodeLens test targets per-target instead of whole-file (#495)

CHANGELOG.md

@@ -2,10 +2,14 @@
 
 ## Improvements:
 
+* Run CodeLens now resolves each scenario's test target on demand instead of walking the whole `.feature` file on every refresh, fixing it getting stuck on very large feature files (VS, VS Code, Rider) - see #495
+
 ## Bug fixes:
 
 *Contributors of this release (in alphabetical order):*
 
+* [@clrudolphi](https://github.com/clrudolphi)
+
 ---
 
 Development prior to this changelog is not recorded here — see the

docs/LSP-IDE-Support-Feature-Designs.md

@@ -1787,7 +1787,7 @@ stopped at, with a hover tooltip carrying the captured error.
 
 | VS Code | Visual Studio | Rider |
 |---------|---------------|-------|
-| 🔧 Plugin — `CodeLens` + custom gutter decorations, own `dotnet test` execution, no native Testing-panel presence (Option 2, see design doc §5) | 🔧 Plugin — classic CodeLens (F24 pattern), reusing VS's own `.TestExplorer.Run/DebugTestsFromCodeLens` commands | 🔧 Plugin — `RunLineMarkerContributor` |
+| 🔧 Plugin — `CodeLens` + custom gutter decorations, own `dotnet test` execution, no native Testing-panel presence (Option 2, see design doc §5) | 🔧 Plugin — classic CodeLens (F24 pattern), reusing VS's own `.TestExplorer.Run/DebugTestsFromCodeLens` commands | 🔧 Plugin — `CodeVisionProvider` (as-built; `RunLineMarkerContributor` wasn't viable — see design doc §5) |
 
 #### LSP messages
 
@@ -1829,6 +1829,17 @@ stopped at, with a hover tooltip carrying the captured error.
   `#line` pragmas mean PDB-level `.feature` debugging may be a narrower path-mapping problem than a
   from-scratch DAP implementation — recorded as a lead for the deferred "Debug Support for Feature
   Files" item below, not a deliverable of #262. See design doc §7 item 4.
+- **Per-target resolution, not whole-file (issue #495, 2026-08-26):** the initial implementation's
+  IDE-side glue resolved `reqnroll/resolveTestTargets` for every scenario in a `.feature` file on
+  every recompute/refresh, not just the scenario(s) actually needed — cheap per #492's syntax-tree
+  cache in isolation, but still O(scenario count) per recompute, which a 2,000+-scenario stress corpus
+  turned into a 30-45s walk exceeding VS's own CodeLens timeout. Fixed per platform, since each one's
+  extensibility contract determines what's possible: VS's async per-line CodeLens data points and VS
+  Code's `resolveCodeLens` both already support resolving only visible lenses lazily (the fix was
+  using that, not adding new plumbing); Rider's `CodeVisionProvider` has no equivalent hook, so it
+  instead gained `RunTestTargetCache`, an identity-keyed per-scenario cache invalidated by the same
+  `reqnroll/refreshCodeLenses` signal the Hook/StepUsages CodeVision providers already act on. See
+  design doc §3's correction for the full per-client breakdown.
 
 ---
 

docs/Test-Runner-Integration-Design.md

@@ -210,6 +210,44 @@ reuse the same parse. The "post-build only" trade-off itself is unaffected by th
 disk-mode freshness check (last-write-time) is what makes it safe to cache across an actual
 rebuild without a dedicated file watcher.
 
+**Correction (issue #495, 2026-08-26):** #492's shared syntax-tree cache made one
+`reqnroll/resolveTestTargets` call cheap (~150ms → ~3.6ms), but every IDE-side caller still issued
+one such call *per scenario in the whole file* on every recompute, not just for the scenario(s)
+actually needed — a leftover "resolve everything, then filter" shape from before per-line/per-lens
+callers existed. On the `VeryLargeFeature` stress corpus (~2,000+ scenarios) that still meant a
+30-45s wall-clock walk per recompute, independently slow enough to exceed VS's own classic-CodeLens
+per-data-point timeout (~26s) regardless of how cheap any one resolution had become. Root-caused and
+fixed per client, since each platform's own extensibility contract determines what's actually
+possible:
+
+- **VS (classic CodeLens)** — `RunTestCodeLensService` split into `GetTargetsForLineAsync(fileUri,
+  line)` (one resolution, called by each line's own `RunTestCodeLensDataPoint.GetDataAsync`) and
+  `GetTagLocationsAsync(fileUri)` (symbol tree only, zero `resolveTestTargets` calls, used only by
+  `RunTestCodeLensTaggerProvider` to know which lines get a tag placement). The async data-point API
+  was already per-line; the fix was ending the whole-file walk each data point triggered to serve
+  itself, not adding new async plumbing.
+- **VS Code** — `runCodeLens.ts`'s `CodeLensProvider` now implements the standard two-phase
+  contract: `provideCodeLenses` places one unresolved lens per scenario symbol (symbol tree only),
+  and `resolveCodeLens` — which VS Code calls lazily, only for lenses that actually scroll into
+  view — is where the single `reqnroll/resolveTestTargets` call for that one lens happens. VS Code's
+  own API already supported this; the previous implementation simply never used the `resolveCodeLens`
+  half of it.
+- **Rider** — IntelliJ's `CodeVisionProvider.computeCodeVision(editor, uiData)` has no visible-range
+  parameter and no per-entry resolve phase; it is always asked for the whole document, on the
+  platform's own schedule. There is no lever to make Rider ask for only the visible lines. Instead,
+  `RunTestTargetCache` (`Rider/testrunner/`) memoizes each `(uri, line)`'s resolved targets, keyed by
+  a cheap identity string (scenario kind + name) built from the symbol tree — `RunLensSupport.computeEntries`
+  still walks every scenario symbol on every call (unavoidable on this platform), but only re-sends
+  `reqnroll/resolveTestTargets` for a scenario whose identity actually changed since the last walk.
+  The cache is invalidated wholesale by the same `reqnroll/refreshCodeLenses` notification the sibling
+  Hook/StepUsages CodeVision providers already act on (`ReqnrollCodeLensRefreshInterceptor`) — the real
+  signal that an underlying resolution changed independent of the `.feature` file's own text (e.g. a
+  `[Binding]` method renamed in `.cs`).
+
+Net effect: VS and VS Code now issue `reqnroll/resolveTestTargets` proportional to the number of
+*currently visible* Run lenses, not the scenario count of the whole file. Rider still walks the whole
+symbol tree per recompute (platform limitation) but skips the RPC for everything that hasn't changed.
+
 **"Parse the code-behind" is not one uniform operation, though — it splits into two tiers with very
 different stability, because Reqnroll ships five test-framework providers (xUnit, xUnit.v3, NUnit,
 MSTest, TUnit) with different attribute vocabularies and argument shapes that can also change across
@@ -316,7 +354,7 @@ a plain `workspace/executeCommand`-style custom request is enough, matching F17/
 
 | VS Code | Visual Studio | Rider |
 |---------|---------------|-------|
-| 🔧 Plugin — `CodeLens` + custom gutter decorations (own execution, no `TestController`) | 🔧 Plugin — Test Explorer editor-margin `KnownMonikers` | 🔧 Plugin — `RunLineMarkerContributor` |
+| 🔧 Plugin — `CodeLens` + custom gutter decorations (own execution, no `TestController`) | 🔧 Plugin — Test Explorer editor-margin `KnownMonikers` | 🔧 Plugin — `CodeVisionProvider` (as-built; see note below — `RunLineMarkerContributor` wasn't viable) |
 
 All three reuse each platform's own run/debug/pass/fail glyph set rather than inventing
 Reqnroll-branded icons (see the issue's own survey — no distinct Gherkin/BDD icon convention exists
@@ -345,20 +383,34 @@ through the native Testing UI.
 
   **Decision (2026-08-05): Option 2 — own execution, no `TestController`, no native Test Explorer tree
   presence.** `▶ Run` / `🐛 Debug` `CodeLens` per scenario/row (same shape as F18's step-usage
-  `CodeLensProvider` — new provider, no new plumbing pattern), calling `reqnroll/resolveTestTargets` on
-  render, then shelling to `dotnet test --filter "FullyQualifiedName=..."` directly against the
-  resolved `DeclaringTypeFullName`/`MethodName` (confirmed to precisely target one method — §6). Pass/
-  fail and failed-step state are tracked entirely in our own extension state and rendered via custom
-  `TextEditorDecorationType` gutter icons plus the CodeLens label, not through `vscode.TestRun`/
-  `TestMessage`. Trades away native Test Explorer tree presence for avoiding duplicate entries against
-  C# Dev Kit's own listing of the same generated methods, and keeps VS Code's design fully within our
-  own extension's control — no dependency on another extension's behavior or its future changes.
-- **Rider**: `RunLineMarkerContributor` gutter icon on scenario/example lines, using
-  `AllIcons.Actions.Execute`/`StartDebugger` pre-run and `TestState.Green2`/`Red2`/`Yellow2` post-run
-  (standard Rider run-line-marker convention). Invokes Rider's native JVM-side test runner against the
-  resolved method. Note the current `reqnroll/Reqnroll.Rider` plugin has **no** scenario-level run
-  marker today ([reqnroll/Reqnroll.Rider#8](https://github.com/reqnroll/Reqnroll.Rider/issues/8)), so
-  there's no existing behavior to match or avoid conflicting with.
+  `CodeLensProvider` — new provider, no new plumbing pattern), calling `reqnroll/resolveTestTargets`
+  to resolve each lens, then shelling to `dotnet test --filter "FullyQualifiedName=..."` directly
+  against the resolved `DeclaringTypeFullName`/`MethodName` (confirmed to precisely target one
+  method — §6). Pass/fail and failed-step state are tracked entirely in our own extension state and
+  rendered via custom `TextEditorDecorationType` gutter icons plus the CodeLens label, not through
+  `vscode.TestRun`/`TestMessage`. Trades away native Test Explorer tree presence for avoiding
+  duplicate entries against C# Dev Kit's own listing of the same generated methods, and keeps VS
+  Code's design fully within our own extension's control — no dependency on another extension's
+  behavior or its future changes.
+  **Correction (issue #495):** the resolution call happens in `resolveCodeLens`, not
+  `provideCodeLenses` — VS Code calls the former lazily, only for lenses that scroll into view, so
+  `provideCodeLenses` itself only ever places unresolved lens ranges from the symbol tree. See the
+  §3 correction above.
+- **Rider**: as-built, a `CodeVisionProvider` (`RunTestCodeVisionProvider`/`RunLensSupport`), not the
+  `RunLineMarkerContributor` originally proposed here — this plugin registers `.feature` with no
+  `ParserDefinition`/PSI tree at all (see `ReqnrollFeatureLanguage`'s own doc comment), and
+  `RunLineMarkerContributor` is inherently PSI-based, so `CodeVisionProvider` (operates on
+  `Editor`/`Document` offsets, same as every other `.feature` editor feature in this plugin) is used
+  instead — the same substitution already made for the closely analogous hook-match-count lens
+  (`HookCodeVisionProvider`). Rendered inline rather than as a gutter icon as a result (Rider's
+  `CodeVisionProvider` API doesn't offer a gutter-icon presentation), using ▶/✓/✗ glyphs in place of
+  `AllIcons.Actions.Execute`/`TestState.Green2`/`Red2`. Invokes Rider's native JVM-side test runner
+  against the resolved method (`RunTestRunner`). Note the current `reqnroll/Reqnroll.Rider` plugin
+  has **no** scenario-level run marker today
+  ([reqnroll/Reqnroll.Rider#8](https://github.com/reqnroll/Reqnroll.Rider/issues/8)), so there's no
+  existing behavior to match or avoid conflicting with. See the §3 correction above for how this
+  provider avoids re-resolving every scenario on every recompute, given `CodeVisionProvider` has no
+  visible-range or per-entry-resolve hook to lean on the way VS Code's `resolveCodeLens` does.
 - **Visual Studio — resolved, §7 item 3.** There is no separate "Test Explorer editor margin"
   extension point to investigate — decompiling VS 18's own `Microsoft.VisualStudio.TestWindow.CodeLens.dll`
   shows that VS's built-in run/debug/pass-fail affordance for ordinary `[Fact]`/`[TestMethod]`/`[Test]`

src/Rider/src/main/kotlin/com/reqnroll/ide/rider/lsp/ReqnrollCodeLensRefreshInterceptor.kt

@@ -6,14 +6,24 @@ import com.intellij.openapi.project.Project
 import com.intellij.platform.lsp.api.LspServerNotificationsHandler
 import com.reqnroll.ide.rider.codevision.HookCodeVisionProvider
 import com.reqnroll.ide.rider.codevision.StepUsagesCodeVisionProvider
+import com.reqnroll.ide.rider.testrunner.RunTestCodeVisionProvider
+import com.reqnroll.ide.rider.testrunner.RunTestTargetCache
 import java.util.concurrent.CompletableFuture
 
 /**
  * Delegates every [LspServerNotificationsHandler] callback straight through to Rider's own
  * platform-provided [handler], except [refreshCodeLenses] — there it also refreshes this
- * project's "N step usages" CodeVision lens ([StepUsagesCodeVisionProvider]) and the hook-match
+ * project's "N step usages" CodeVision lens ([StepUsagesCodeVisionProvider]), the hook-match
  * lenses ([HookCodeVisionProvider]/`StepHooksCodeVisionProvider`, invalidated together by
- * [HookCodeVisionProvider.refreshOpenFeatureEditors]) before delegating.
+ * [HookCodeVisionProvider.refreshOpenFeatureEditors]), and the Run lens
+ * ([RunTestCodeVisionProvider]) before delegating.
+ *
+ * The Run lens wiring (issue #495) also clears [RunTestTargetCache] first — that cache is what
+ * lets `RunLensSupport.computeEntries` skip re-sending `reqnroll/resolveTestTargets` for a scenario
+ * whose identity hasn't changed since the last recompute, and this notification is the real
+ * staleness signal for when a *resolution* actually changed underneath an unchanged scenario name
+ * (e.g. a `[Binding]` method renamed on the `.cs` side). Without this, a stale cached resolution
+ * could outlive the very change that invalidated it.
  *
  * Rider's CodeVision engine has no signal of its own for "the data behind this lens changed" —
  * unlike inlay hints/semantic tokens, which at least have *a* refresh mechanism once wired (see
@@ -39,6 +49,8 @@ class ReqnrollCodeLensRefreshInterceptor(
                 if (!project.isDisposed) {
                     StepUsagesCodeVisionProvider.refreshOpenCsEditors(project)
                     HookCodeVisionProvider.refreshOpenFeatureEditors(project)
+                    RunTestTargetCache.invalidateAll()
+                    RunTestCodeVisionProvider.refreshOpenFeatureEditors(project)
                 }
             },
             ModalityState.any(),

src/Rider/src/main/kotlin/com/reqnroll/ide/rider/testrunner/RunLensSupport.kt

@@ -41,11 +41,18 @@ internal object RunLensSupport {
     }
 
     /**
-     * Fetches the scenario/Outline symbols for [filePath]'s `.feature` document, resolves each
-     * one's test target(s) via `reqnroll/resolveTestTargets`, and builds one CodeVision entry per
-     * scenario that has at least one resolved target — scenarios with none (not built yet, or a
-     * naming-rule mismatch) get no entry at all, matching the "not built yet" reasoning already
-     * used in the VS Code/VS implementations of this same feature.
+     * Fetches the scenario/Outline symbols for [filePath]'s `.feature` document and builds one
+     * CodeVision entry per scenario that has at least one resolved test target — scenarios with
+     * none (not built yet, or a naming-rule mismatch) get no entry at all, matching the "not built
+     * yet" reasoning already used in the VS Code/VS implementations of this same feature.
+     *
+     * The symbol-tree walk itself runs on every call — `computeCodeVision` is invoked by IntelliJ's
+     * platform on its own schedule (edits, file open, etc.) with no way for this plugin to ask for
+     * only the visible range (issue #495's platform survey). What's skipped per call is the
+     * `reqnroll/resolveTestTargets` RPC: [RunTestTargetCache] reuses the previous resolution for any
+     * scenario whose identity (kind + name) hasn't changed since the last walk, so a large feature
+     * file only pays the RPC cost for scenarios that actually changed, not the whole document every
+     * time.
      */
     fun computeEntries(
         project: Project,
@@ -64,12 +71,17 @@ internal object RunLensSupport {
             val startLine = selectionRange.start.line
             if (startLine < 0 || startLine >= document.lineCount) continue
 
-            val response = ReqnrollRequestSender.resolveTestTargets(
-                project, uri,
-                selectionRange.start.line, selectionRange.start.character,
-                selectionRange.end.line, selectionRange.end.character,
-            )
-            val targets = response?.targets.orEmpty()
+            val identity = "${symbol.detail}|${symbol.name}"
+            val targets = RunTestTargetCache.get(uri, startLine, identity) ?: run {
+                val response = ReqnrollRequestSender.resolveTestTargets(
+                    project, uri,
+                    selectionRange.start.line, selectionRange.start.character,
+                    selectionRange.end.line, selectionRange.end.character,
+                )
+                val resolved = response?.targets.orEmpty()
+                RunTestTargetCache.put(uri, startLine, identity, resolved)
+                resolved
+            }
             if (targets.isEmpty()) continue
 
             val offset = document.getLineStartOffset(startLine)

src/Rider/src/main/kotlin/com/reqnroll/ide/rider/testrunner/RunTestTargetCache.kt

@@ -0,0 +1,46 @@
+package com.reqnroll.ide.rider.testrunner
+
+import com.reqnroll.ide.rider.lsp.protocol.ScenarioTestTargetItem
+import java.util.concurrent.ConcurrentHashMap
+
+/**
+ * Per-(uri, line) cache of resolved `reqnroll/resolveTestTargets` results (issue #495).
+ *
+ * Unlike VS's classic CodeLens (async per-line data points) and VS Code's `CodeLensProvider`
+ * (`provideCodeLenses` + lazy `resolveCodeLens`), IntelliJ's `CodeVisionProvider.computeCodeVision`
+ * has no visible-range parameter and no per-entry resolve phase — it's always asked for the *whole*
+ * document, every time CodeVision recomputes (issue #495's platform survey). Rider therefore can't
+ * skip resolving off-screen scenarios the way the other two clients now do; the only lever available
+ * is skipping the RPC itself for a scenario whose *identity* hasn't changed since the last recompute.
+ *
+ * [identity] is the caller's cheap proxy for "would this scenario resolve to something different?" —
+ * [RunLensSupport] builds it from the symbol's Scenario/Scenario-Outline kind plus its name. A
+ * `computeCodeVision` call still walks every symbol in the document (that part of the cost is
+ * unavoidable on this platform), but only re-sends `reqnroll/resolveTestTargets` for scenarios whose
+ * identity actually changed, instead of for all of them unconditionally.
+ */
+internal object RunTestTargetCache {
+    private data class Key(val uri: String, val line: Int)
+    private data class Entry(val identity: String, val targets: List<ScenarioTestTargetItem>)
+
+    private val entries = ConcurrentHashMap<Key, Entry>()
+
+    /** Returns the cached targets for (uri, line) only if they were cached under the same [identity]; null otherwise (never resolved, or the scenario there changed). */
+    fun get(uri: String, line: Int, identity: String): List<ScenarioTestTargetItem>? {
+        val entry = entries[Key(uri, line)] ?: return null
+        return if (entry.identity == identity) entry.targets else null
+    }
+
+    /** Records the resolved [targets] for (uri, line) under [identity], for [get] to reuse until that identity changes or the cache is invalidated. */
+    fun put(uri: String, line: Int, identity: String, targets: List<ScenarioTestTargetItem>) {
+        entries[Key(uri, line)] = Entry(identity, targets)
+    }
+
+    /** Drops every cached line for [uri] — real staleness signal, e.g. the file's bindings changed underneath an unchanged scenario name. */
+    fun invalidateFile(uri: String) {
+        entries.keys.removeIf { it.uri == uri }
+    }
+
+    /** Drops every cached entry for every file — called from `reqnroll/refreshCodeLenses` (see `ReqnrollCodeLensRefreshInterceptor`), the same workspace-wide signal `HookCodeVisionProvider`/`StepUsagesCodeVisionProvider` already act on. */
+    fun invalidateAll() = entries.clear()
+}