You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
RunTestCodeLensService.GetTargetsAsync resolves every scenario in a .feature file on every call, not just the one line a particular CodeLens data point actually needs. On the VeryLargeFeature stress corpus (~2,000-2,400 scenarios in one file), a single walk takes 30-45 seconds — independently too slow relative to VS's own classic-CodeLens per-data-point timeout (observed at ~26 seconds).
This was masked, not fixed, by the caching work in #492/#493/#494:
Fix #491: cache C# syntax trees to eliminate repeated full-document parses #492 made each individual reqnroll/resolveTestTargets server call fast (~150ms → ~3.6ms) by caching the parsed C# syntax tree, but RunTestCodeLensService.GetTargetsAsync still issues (or processes) one such resolution per scenario in the whole file, so total wall time still scales with document size.
No caching strategy that keeps re-running the full walk from scratch will fully paper over this on a corpus this large. As shipped today, this only manifests as a "stuck CodeLens" symptom the first time a fresh walk is required (cold open, or after a real invalidation from reqnroll/refreshCodeLens) — but on a sufficiently large file, that first walk can still exceed VS's timeout even with everything from #492-#494 in place.
Proposed direction (not yet designed)
Some options worth evaluating, not mutually exclusive:
Incremental/per-scenario caching inside RunTestCodeLensService/ScenarioTestTargetResolver, keyed by scenario identity + a cheap staleness check (e.g. tag/method-signature hash), so a real edit only needs to re-resolve the scenarios it actually touched instead of the whole file.
Resolving lazily/on-demand per visible line instead of eagerly walking the whole document up front, if the classic CodeLens API allows a data point to ask for just its own line's answer without the service needing every other scenario's answer too.
Revisiting whether RunTestCodeLensService.GetTargetsAsync's "resolve everything" contract is the right shape at all now that callers are individual line-scoped RunTestCodeLensDataPoints, not just the whole-file tagger.
Scope: add this traffic to the performance benchmark utility
tests/Performance/Reqnroll.IdeSupport.LSP.Server.Benchmarks* has no coverage at all today for reqnroll/resolveTestTargets / the Run Scenario CodeLens path — it's the one reqnroll/* operation with neither a PerfTargets entry (even an unpublished "measured but no threshold" one, which every sibling operation has) nor an InteractiveScenarios method, despite already being field-instrumented via IOperationDurationRecorder in ResolveTestTargetsHandler. Any fix for this issue should come with:
A PerfTargets.ResolveTestTargets entry + InteractiveScenarios.ResolveTestTargetsAsync() for single-request latency, following the existing pattern used by every other reqnroll/* operation.
Without this, a regression in either the per-scenario cost or the whole-document walk shape would ship silently, the same way this issue's underlying slowness did.
Issue Description
RunTestCodeLensService.GetTargetsAsyncresolves every scenario in a.featurefile on every call, not just the one line a particular CodeLens data point actually needs. On theVeryLargeFeaturestress corpus (~2,000-2,400 scenarios in one file), a single walk takes 30-45 seconds — independently too slow relative to VS's own classic-CodeLens per-data-point timeout (observed at ~26 seconds).This was masked, not fixed, by the caching work in #492/#493/#494:
reqnroll/resolveTestTargetsserver call fast (~150ms → ~3.6ms) by caching the parsed C# syntax tree, butRunTestCodeLensService.GetTargetsAsyncstill issues (or processes) one such resolution per scenario in the whole file, so total wall time still scales with document size.No caching strategy that keeps re-running the full walk from scratch will fully paper over this on a corpus this large. As shipped today, this only manifests as a "stuck CodeLens" symptom the first time a fresh walk is required (cold open, or after a real invalidation from
reqnroll/refreshCodeLens) — but on a sufficiently large file, that first walk can still exceed VS's timeout even with everything from #492-#494 in place.Proposed direction (not yet designed)
Some options worth evaluating, not mutually exclusive:
RunTestCodeLensService/ScenarioTestTargetResolver, keyed by scenario identity + a cheap staleness check (e.g. tag/method-signature hash), so a real edit only needs to re-resolve the scenarios it actually touched instead of the whole file.RunTestCodeLensService.GetTargetsAsync's "resolve everything" contract is the right shape at all now that callers are individual line-scopedRunTestCodeLensDataPoints, not just the whole-file tagger.Scope: add this traffic to the performance benchmark utility
tests/Performance/Reqnroll.IdeSupport.LSP.Server.Benchmarks*has no coverage at all today forreqnroll/resolveTestTargets/ the Run Scenario CodeLens path — it's the one reqnroll/* operation with neither aPerfTargetsentry (even an unpublished "measured but no threshold" one, which every sibling operation has) nor anInteractiveScenariosmethod, despite already being field-instrumented viaIOperationDurationRecorderinResolveTestTargetsHandler. Any fix for this issue should come with:PerfTargets.ResolveTestTargetsentry +InteractiveScenarios.ResolveTestTargetsAsync()for single-request latency, following the existing pattern used by every other reqnroll/* operation.WorkspaceReloadContentionScenario(Performance benchmark suite never exercises concurrent request contention, so it missed the #471/#477 dispatch-stall regression #488) already establishes the right shape for this ("many requests fire together, race a cheap read") and should be followed rather than reinvented.Without this, a regression in either the per-scenario cost or the whole-document walk shape would ship silently, the same way this issue's underlying slowness did.
Related