Skip to content

Flaky: VS Code extension zero-to-running E2E fails in "after each" teardown (AggregateError) on Windows #18098

@radical

Description

@radical

Summary

The VS Code extension zero-to-running E2E shard intermittently fails in its after each teardown hook. The test body itself passes ("1 passing"), but teardown throws an AggregateError, failing the job. This is a flaky CI/teardown failure, not specific to any one PR — it was first observed on an unrelated PR (MAUI OTLP change) and the same shard passes on other runs (including main).

Observed failure

Job: Tests / Run VS Code extension E2E tests / VS Code extension E2E (Windows, zero-to-running)
First seen: https://github.com/microsoft/aspire/actions/runs/27222383303/job/80383131267 (PR #18053, jfversluis/fix-maui-otlp)

  1 passing (2m)
  1 failing

  1) Aspire zero-to-running E2E
       "after each" hook for "creates a new AppHost, adds a package, and debugs to the dashboard":
     AggregateError: Zero-to-running E2E teardown failed.
      at runE2eTeardown (out\test-e2e\test-e2e\helpers\fixtures.js:132:15)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async Context.<anonymous> (out\test-e2e\test-e2e\zeroToRunning.e2e.test.js:46:9)

The exported extension state at failure time shows the workspace had no resources, with:

"hasError": true,
"errorMessage": "No workspace resources were returned. Workspace resources require the AppHost to reference Aspire.Hosting 13.2.0 or newer.",

and the extension log ends with the teardown sequence:

[info] Stopping the Aspire debug session
[info] Requested Aspire CLI exit with args: run --start-debug-session --nologo --apphost ...\ExtensionZeroToRunningApp\apphost.cs
[info] Stopping aspire describe --follow for workspace resources
[warning] aspire describe --follow exited (code 0) without producing data; not auto-restarting.
[warning] No workspace resources were returned. Workspace resources require the AppHost to reference Aspire.Hosting 13.2.0 or newer.
[info] Disposing RPC server

runE2eTeardown (extension/src/test-e2e/helpers/fixtures.ts) runs all cleanups and aggregates any thrown errors into an AggregateError. The mocha output only surfaces the top-level failureMessage; the individual cleanup error(s) are not printed, so the specific failing cleanup step (out of stopDebugging, stopAppHostIfRunning, restoreWorkspaceAppHostConfig, removeGeneratedProject, etc. — see zeroToRunning.e2e.test.ts) is not directly visible from the log.

Repro

  • Workflow: CI → Tests / Run VS Code extension E2E tests
  • Shard: VS Code extension E2E (Windows, zero-to-running)
  • Runner: Windows, Node 20.20.2, VS Code code_version 1.122.1, vscode-extension-tester

Intermittent — not every run reproduces.

Expected vs. actual

  • Expected: When the test body passes, teardown completes cleanly and the shard reports success.
  • Actual: Teardown throws AggregateError: Zero-to-running E2E teardown failed. and the job fails with exit code 1, even though the test assertion passed.

Not PR-specific

  • The originating PR (Fix MAUI OTLP dev tunnel endpoint resolution #18053) changes MAUI OTLP dev tunnel endpoint resolution and does not touch the VS Code extension.
  • The Windows, zero-to-running shard passed on main (run 27287964212) and on the unrelated danegsta/port-allocator PR (run 27302507360).
  • The broader VS Code extension E2E suite shows other intermittent shard failures in the same window (e.g. Linux, debug-dashboard failed on main in run 27287964212), consistent with E2E flakiness rather than a product regression.

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions