Skip to content

Resolve Run CodeLens test targets per-target instead of whole-file (#495) - #496

Merged
clrudolphi merged 3 commits into
masterfrom
perf/per-target-run-codelens-resolution
Aug 26, 2026
Merged

Resolve Run CodeLens test targets per-target instead of whole-file (#495)#496
clrudolphi merged 3 commits into
masterfrom
perf/per-target-run-codelens-resolution

Conversation

@clrudolphi

Copy link
Copy Markdown
Collaborator

🤔 What's changed?

RunTestCodeLensService/VS Code's runCodeLens.ts/Rider's RunLensSupport all resolved reqnroll/resolveTestTargets for every scenario in a .feature file on every refresh, instead of just the scenario(s) actually needed. Split per client, since each platform's extensibility contract determines what's actually possible:

  • VS (classic CodeLens)RunTestCodeLensService.GetTargetsAsync (whole file) split into GetTargetsForLineAsync(fileUri, line) (one resolution, called by each line's own RunTestCodeLensDataPoint) and GetTagLocationsAsync(fileUri) (symbol tree only, zero resolveTestTargets calls, feeds the tagger). Result cache re-keyed by (fileUri, line).
  • VS CoderunCodeLens.ts now implements the standard two-phase provideCodeLenses/resolveCodeLens contract: provideCodeLenses places unresolved lenses from the symbol tree only, resolveCodeLens resolves lazily, only for lenses that scroll into view.
  • Rider — added RunTestTargetCache, an identity-keyed per-scenario cache, since CodeVisionProvider has no visible-range/resolve-lazily hook the way the other two platforms do. Invalidated via the same reqnroll/refreshCodeLenses signal Hook/StepUsages CodeVision providers already act on — which also fixed a pre-existing gap where the Run lens was never wired to that refresh at all.
  • Benchmarks — added reqnroll/resolveTestTargets to the performance benchmark: an isolated latency scenario, plus a concurrent-callers contention scenario against a synthetic ~2,000-scenario file, following WorkspaceReloadContentionScenario's shape.
  • Docs — updated Test-Runner-Integration-Design.md and LSP-IDE-Support-Feature-Designs.md with the per-client breakdown and Rider's as-built CodeVisionProvider implementation (the doc still described the originally-proposed RunLineMarkerContributor).

⚡️ What's your motivation?

Fixes #495. On the VeryLargeFeature stress corpus (~2,000-2,400 scenarios in one file), the whole-file walk took 30-45s per refresh — independently slow enough to exceed VS's own classic-CodeLens per-data-point timeout (~26s), even after #492's syntax-tree caching made each individual resolution cheap. #492-#494 made the walk's own per-call cost cheap and de-duplicated concurrent callers, but nothing before this reduced the number of calls a walk actually issues.

🏷️ What kind of change is this?

  • ⚡ New feature (non-breaking change which adds new behaviour) — restructures the resolution path without changing the Run CodeLens's user-visible behavior beyond fixing the stuck-lens bug.

🧩 Area(s) touched

  • LSP server (src/LSP) — benchmark tooling only, no handler changes
  • Visual Studio extension (src/VisualStudio)
  • VS Code extension (src/VSCode)
  • Rider plugin (src/Rider)

♻️ Anything particular you want feedback on?

Rider's RunTestTargetCache ceiling ratio (200x) for the new contention benchmark is a placeholder based on one local run (~140x observed) — no reference-machine calibration data exists yet for this specific operation shape (concurrent requests-with-response, unlike the sibling didChange-storm scenario's fire-and-forget notifications).

📋 Checklist:

  • I've changed the behaviour of the code
    • I have added/updated tests to cover my changes.
  • My change alters as-built behaviour described in a design doc under docs/
    • I have updated the relevant doc accordingly.
  • Users should know about my change
    • I have added an entry to the "[vNext]" section of the CHANGELOG, linking to this pull request & included my GitHub handle to the release contributors list.

🤖 Generated with Claude Code

clrudolphi and others added 3 commits August 26, 2026 16:14
…hole file (#495)

RunTestCodeLensService/VS Code's runCodeLens.ts/Rider's RunLensSupport all resolved
reqnroll/resolveTestTargets for every scenario in a .feature file on every call, so a
2,000+-scenario stress corpus took 30-45s per walk, exceeding VS's own CodeLens timeout.

- VS: split GetTargetsForLineAsync (one line, per data point) from GetTagLocationsAsync
  (symbol tree only, no resolution, feeds the tagger); result cache re-keyed by (file, line).
- VS Code: switched runCodeLens.ts to the standard provideCodeLenses/resolveCodeLens
  two-phase contract, so resolution only happens for visible lenses.
- Rider: added RunTestTargetCache, an identity-keyed per-scenario cache, since
  CodeVisionProvider has no per-line/visible-range hook the way the other two do;
  invalidated via the same reqnroll/refreshCodeLenses signal Hook/StepUsages already use
  (which also fixes Run never being wired into that refresh at all).
- Added reqnroll/resolveTestTargets to the performance benchmark: an isolated latency
  scenario plus a concurrent-callers contention scenario against a synthetic
  ~2,000-scenario file, mirroring WorkspaceReloadContentionScenario's shape.
- Updated Test-Runner-Integration-Design.md / LSP-IDE-Support-Feature-Designs.md to
  reflect the per-client split and Rider's as-built CodeVisionProvider implementation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CI's format:check flagged both files — line-wrap only, no logic change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@clrudolphi
clrudolphi merged commit f044020 into master Aug 26, 2026
17 checks passed
@clrudolphi
clrudolphi deleted the perf/per-target-run-codelens-resolution branch August 26, 2026 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Run Scenario CodeLens: full-document resolveTestTargets walk is too slow on large feature files

1 participant