Improve telemetry instrumentation and error diagnostics for typespec-vscode (#10847)

# Improve telemetry instrumentation and error diagnostics

## Summary

Based on analysis of 2 weeks of telemetry data (71K+ events, 2,327
unique users), this PR improves telemetry instrumentation gaps and error
diagnostics across the typespec-vscode extension and the compiler LSP
server.

## Changes

### Extension telemetry improvements (`packages/typespec-vscode`)

**`install-tsp-compiler.ts`** — Added `lastStep` tracking and
`logOperationDetailTelemetry` for failure/timeout cases. Previously, all
3 install failures in the dataset had `lastStep=undefined` and zero
error detail.

**`openapi3-preview.ts`** — Refactored `getOpenApi3OutputFilePath` to
return `Result<string>` instead of `string | undefined` to properly
distinguish compile failures from user cancellations. Added `lastStep`
for the success path (`"Preview panel opened"`). Added compile error
details to telemetry.

**`extension.ts` (start-server)** — Added `lastStep` for the "compiler
not found" state before the install prompt, and for the
cancelled-install path. These covered 66 events (34 fail + 32 cancelled)
that previously had `lastStep=undefined`.

**`extension.ts` (server-path-changed)** — Added `lastStep` for the
config change handler. All 30 events previously had
`lastStep=undefined`.

### Better error message for node/tsp not on PATH
(`packages/typespec-vscode`)

**`tsp-executable-resolver.ts`** — When the compiler is found locally
but neither `node` nor `tsp` is available on PATH, the extension now
shows an actionable error message explaining the likely cause
(nvm/fnm/volta PATH not inherited) and three concrete fixes. Previously,
this silently fell through to `spawn tsp ENOENT`. This was the #1 root
cause of blocked users — 107 failure events from users who never
recovered.

### Server-side error detail preservation (`packages/compiler`)

**`serverlib.ts`** — Added `wrapUnhandledError` wrapper around all LSP
server handlers. When a handler crashes, the wrapper catches the error
and re-throws with the full server-side stack trace in the error message
(via `inspect(e)`). Previously, the JSON-RPC layer only forwarded
`error.message` to the client, so the server-side crash location was
completely lost in telemetry. This addresses the 297 `Cannot read
properties of undefined (reading 'kind')` errors that were previously
opaque.

## Telemetry data highlights

- **2,327 total users** in the 2-week window
- **101 users (4.3%) completely blocked** — never had a successful
`start-server` in 2 weeks
- **#1 root cause**: `spawn tsp ENOENT` (node not on PATH) — 107 events
from never-recovered users, disproportionately macOS (60%)
- **#2 root cause**: compiler not found — 50 events
- **297 `reading 'kind'` unhandled errors** — now will include
server-side stack for future diagnosis

---------

Co-authored-by: Timothee Guerin <timothee.guerin@outlook.com>

Author: RodgeFu

.chronus/changes/tel-improve-compiler-2026-6-1-19-0-0.md

@@ -0,0 +1,7 @@
+---
+changeKind: fix
+packages:
+  - "@typespec/compiler"
+---
+
+[Language Server] Wrapped LSP server handlers with `wrapUnhandledError` to preserve server-side stack traces in error messages forwarded to the client. Previously, the JSON-RPC layer discarded the original stack trace, making unhandled errors in telemetry opaque.

.chronus/changes/tel-improve-vscode-2026-6-1-19-0-0.md

@@ -0,0 +1,7 @@
+---
+changeKind: fix
+packages:
+  - "typespec-vscode"
+---
+
+Improved telemetry instrumentation for `install-global-compiler-cli`, `preview-openapi3`, `start-server`, and `server-path-changed` events by adding missing `lastStep` tracking and error detail logging. Added actionable error message when compiler is found but neither `node` nor `tsp` is available on PATH, guiding users to fix common nvm/fnm/volta configuration issues.

packages/compiler/src/server/serverlib.ts

@@ -182,44 +182,99 @@ export function createServer(
   let isInitialized = false;
   let pendingMessages: ServerLog[] = [];
 
+  /**
+   * Wraps an LSP handler to preserve the server-side error details when it crashes.
+   *
+   * By default, the JSON-RPC layer (vscode-languageserver) catches handler errors and
+   * creates a new ResponseError using only `error.message`, discarding the original stack
+   * trace. On the client side, the telemetry framework then captures this as an unhandled
+   * error, but the `unhandled_error_stack` only shows the client-side message handling code:
+   *
+   * ```
+   * Error: Request textDocument/hover failed with message: Cannot read properties of undefined (reading 'kind')
+   *     at handleResponse (extension.cjs:2104:40)         // <-- client-side LSP message handler
+   *     at handleMessage (extension.cjs:1914:11)
+   *     at processMessageQueue (extension.cjs:1929:13)
+   *     at Immediate.<anonymous> (extension.cjs:1905:11)
+   * ```
+   *
+   * The actual server-side crash location (e.g., in the checker or parser) is completely lost.
+   *
+   * This wrapper catches the error first and re-throws a new Error whose message includes
+   * the full original error details (stack trace for Error instances, String() for others).
+   * The JSON-RPC layer then forwards this enriched message to the client, so the
+   * telemetry `unhandled_error_message` will contain the server-side crash location:
+   *
+   * ```
+   * [getHover] TypeError: Cannot read properties of undefined (reading 'kind')
+   *     at Checker.getTypeForNode (checker.ts:1234:15)    // <-- actual crash location
+   *     at getHover (serverlib.ts:826:52)
+   *     ...
+   * ```
+   */
+  function wrapUnhandledError<T extends (...args: any[]) => any>(fn: T): T {
+    const name = fn.name || "anonymous";
+    return (async (...args: any[]) => {
+      try {
+        return await fn(...args);
+      } catch (e) {
+        if (e instanceof Error) {
+          const detail = e.stack ? `${e.message}\n${e.stack}` : e.message;
+          throw new Error(`[${name}] ${detail}`, { cause: e });
+        } else if (typeof e === "string") {
+          throw new Error(`[${name}] ${e}`, { cause: e });
+        } else if (typeof e === "object" && e !== null) {
+          let detail: string;
+          try {
+            detail = JSON.stringify(e);
+          } catch {
+            throw e;
+          }
+          throw new Error(`[${name}] ${detail}`, { cause: e });
+        }
+        throw e;
+      }
+    }) as T;
+  }
+
   return {
     get pendingMessages() {
       return pendingMessages;
     },
     get workspaceFolders() {
       return workspaceFolders;
     },
-    compile,
-    initialize,
-    initialized,
-    workspaceFoldersChanged,
-    watchedFilesChanged,
-    formatDocument,
-    gotoDefinition,
-    documentClosed,
-    documentOpened,
-    complete,
-    findReferences,
-    findDocumentHighlight,
-    prepareRename,
-    rename,
-    renameFiles,
-    getSemanticTokens: getSemanticTokensForDocument,
-    buildSemanticTokens,
-    checkChange,
-    getFoldingRanges,
-    getHover,
-    getSignatureHelp,
-    getDocumentSymbols,
-    getCodeActions,
-    resolveCodeAction,
+    compile: wrapUnhandledError(compile),
+    initialize: wrapUnhandledError(initialize),
+    initialized: wrapUnhandledError(initialized),
+    workspaceFoldersChanged: wrapUnhandledError(workspaceFoldersChanged),
+    watchedFilesChanged: wrapUnhandledError(watchedFilesChanged),
+    formatDocument: wrapUnhandledError(formatDocument),
+    gotoDefinition: wrapUnhandledError(gotoDefinition),
+    documentClosed: wrapUnhandledError(documentClosed),
+    documentOpened: wrapUnhandledError(documentOpened),
+    complete: wrapUnhandledError(complete),
+    findReferences: wrapUnhandledError(findReferences),
+    findDocumentHighlight: wrapUnhandledError(findDocumentHighlight),
+    prepareRename: wrapUnhandledError(prepareRename),
+    rename: wrapUnhandledError(rename),
+    renameFiles: wrapUnhandledError(renameFiles),
+    getSemanticTokens: wrapUnhandledError(getSemanticTokensForDocument),
+    buildSemanticTokens: wrapUnhandledError(buildSemanticTokens),
+    checkChange: wrapUnhandledError(checkChange),
+    getFoldingRanges: wrapUnhandledError(getFoldingRanges),
+    getHover: wrapUnhandledError(getHover),
+    getSignatureHelp: wrapUnhandledError(getSignatureHelp),
+    getDocumentSymbols: wrapUnhandledError(getDocumentSymbols),
+    getCodeActions: wrapUnhandledError(getCodeActions),
+    resolveCodeAction: wrapUnhandledError(resolveCodeAction),
     log,
-    reportDiagnostics,
+    reportDiagnostics: wrapUnhandledError(reportDiagnostics),
 
-    getInitProjectContext,
-    validateInitProjectTemplate,
-    initProject,
-    internalCompile,
+    getInitProjectContext: wrapUnhandledError(getInitProjectContext),
+    validateInitProjectTemplate: wrapUnhandledError(validateInitProjectTemplate),
+    initProject: wrapUnhandledError(initProject),
+    internalCompile: wrapUnhandledError(internalCompile),
   };
 
   async function initialize(params: InitializeParams): Promise<InitializeResult> {

packages/typespec-vscode/src/extension.ts

@@ -245,6 +245,7 @@ export async function activate(context: ExtensionContext) {
             await telemetryClient.doOperationWithTelemetry(
               TelemetryEventName.ServerPathSettingChanged,
               async (tel) => {
+                tel.lastStep = "Recreate LSP client for path change";
                 return await recreateLSPClient(context, tel.activityId);
               },
               undefined,
@@ -316,6 +317,7 @@ export async function activate(context: ExtensionContext) {
             }
             // client will be undefined only when we can't find compiler locally or globally
             // otherwise, the client should always be created though the start command may fail which is a different case
+            ssTel.lastStep = "Compiler not found (prompting to install)";
             const choice: "Yes" | "Ignore" | undefined = await vscode.window.showWarningMessage(
               "No TypeSpec compiler found which is required to start TypeSpec language server. Do you want to install TypeSpec compiler?",
               "Yes",
@@ -355,6 +357,8 @@ export async function activate(context: ExtensionContext) {
                 { showPopup: true },
               );
               ssTel.lastStep = "Failed to install TypeSpec compiler.";
+            } else {
+              ssTel.lastStep = "Install TypeSpec compiler cancelled.";
             }
             return installResult.code;
           },

packages/typespec-vscode/src/tsp-executable-resolver.ts

@@ -8,6 +8,7 @@ import { SettingName } from "./types.js";
 import {
   checkInstalledExecutable,
   checkInstalledNode,
+  checkInstalledTspCli,
   isFile,
   loadModule,
   useShellInExec,
@@ -159,10 +160,32 @@ export async function resolveTypeSpecServer(
     });
     return { command: "node", args: [serverPath, ...args], options };
   } else {
-    // otherwise the local compiler should be installed by standalone tsp cli
-    logger.debug("Start tsp server using standalone tsp cli");
+    const tspCliPath = await checkInstalledTspCli();
+    if (tspCliPath.length > 0) {
+      logger.debug("Start tsp server using standalone tsp cli");
+      telemetryClient.logOperationDetailTelemetry(activityId, {
+        compilerStartType: "standalone-tsp-cli",
+      });
+      return { command: "tsp", args: ["--server", serverPath, ...args], options };
+    }
+    // Neither node nor tsp is on PATH. Show an actionable error but still try tsp
+    // as a last resort — it may work in some environments where `which` fails but
+    // the shell can still resolve the command.
+    logger.error(
+      [
+        `TypeSpec compiler was found at '${serverPath}', but it cannot be started because neither 'node' nor 'tsp' is available in PATH.`,
+        "This commonly happens when Node.js is installed via a version manager (nvm, fnm, volta) whose PATH is not inherited by VS Code.",
+        "To fix this, try one of the following:",
+        " - Launch VS Code from a terminal where 'node' is available (e.g. run 'code .' after activating nvm).",
+        " - Set the 'typespec.tsp-server.path' setting to the full path of your tsp-server.js file.",
+        " - Install Node.js system-wide so it's available to all processes.",
+      ].join("\n"),
+      [],
+      { showPopup: true, showOutput: true },
+    );
     telemetryClient.logOperationDetailTelemetry(activityId, {
-      compilerStartType: "standalone-tsp-cli",
+      compilerStartType: "standalone-tsp-cli-fallback",
+      error: "Neither node nor tsp is available in PATH. Compiler found but cannot be started.",
     });
     return { command: "tsp", args: ["--server", serverPath, ...args], options };
   }

packages/typespec-vscode/src/vscode-cmd/install-tsp-compiler.ts

@@ -1,3 +1,4 @@
+import { inspect } from "util";
 import logger from "../log/logger.js";
 import telemetryClient from "../telemetry/telemetry-client.js";
 import { TelemetryEventName } from "../telemetry/telemetry-event.js";
@@ -11,6 +12,7 @@ export async function installCompilerGlobally(
     TelemetryEventName.InstallGlobalCompilerCli,
     async (tel) => {
       const showPopup = args?.silentMode !== true;
+      tel.lastStep = "Call installCompilerWithUi";
       const result = await installCompilerWithUi(
         {
           confirmNeeded: args?.confirm !== false,
@@ -20,13 +22,26 @@ export async function installCompilerGlobally(
         [] /*localPath, empty for global*/,
       );
       if (result.code === ResultCode.Success) {
+        tel.lastStep = "Compiler installed successfully";
         logger.info(`Compiler installed successfully`, [], { showPopup });
-      } else if (result.code === ResultCode.Fail || result.code === ResultCode.Timeout) {
-        logger.error(
-          `Installing compiler ${result.code === ResultCode.Fail ? "failed" : "timeout"}. Please check previous logs for details`,
-          [],
-          { showPopup },
-        );
+      } else if (result.code === ResultCode.Cancelled) {
+        tel.lastStep = "User cancelled installation";
+      } else if (result.code === ResultCode.Timeout) {
+        tel.lastStep = "Installation timeout";
+        telemetryClient.logOperationDetailTelemetry(tel.activityId, {
+          error: `Installing compiler globally timeout`,
+        });
+        logger.error(`Installing compiler timeout. Please check previous logs for details`, [], {
+          showPopup,
+        });
+      } else if (result.code === ResultCode.Fail) {
+        tel.lastStep = "Installation failed";
+        telemetryClient.logOperationDetailTelemetry(tel.activityId, {
+          error: `Installing compiler globally failed: ${inspect(result.details)}`,
+        });
+        logger.error(`Installing compiler failed. Please check previous logs for details`, [], {
+          showPopup,
+        });
       }
       return result;
     },

packages/typespec-vscode/src/vscode-cmd/openapi3-preview.ts

@@ -7,7 +7,7 @@ import { getBaseFileName, getDirectoryPath, joinPaths } from "../path-utils.js";
 import telemetryClient from "../telemetry/telemetry-client.js";
 import { OperationTelemetryEvent } from "../telemetry/telemetry-event.js";
 import { TspLanguageClient } from "../tsp-language-client.js";
-import { ResultCode } from "../types.js";
+import { Result, ResultCode } from "../types.js";
 import { getEntrypointTspFile, TraverseMainTspFileInWorkspace } from "../typespec-utils.js";
 import { createTempDir, throttle } from "../utils.js";
 
@@ -134,15 +134,13 @@ async function loadOpenApi3PreviewPanel(
     });
     panel.reveal();
   } else {
-    const getOpenApi3OutputFilePath = async (
-      selectOutput: boolean,
-    ): Promise<string | undefined> => {
+    const getOpenApi3OutputFilePath = async (selectOutput: boolean): Promise<Result<string>> => {
       return await vscode.window.withProgress(
         {
           location: vscode.ProgressLocation.Notification,
           title: "Loading OpenAPI3 files...",
         },
-        async (): Promise<string | undefined> => {
+        async (): Promise<Result<string>> => {
           const srcFolder = getDirectoryPath(mainTspFile);
           const outputFolder = await getOutputFolder(mainTspFile, tmpRoot);
           if (!outputFolder) {
@@ -153,7 +151,7 @@ async function loadOpenApi3PreviewPanel(
             telemetryClient.logOperationDetailTelemetry(tel.activityId, {
               error: "Failed to create temporary folder for OpenAPI3 files",
             });
-            return undefined;
+            return { code: ResultCode.Fail };
           }
           await clearOutputFolder(outputFolder);
 
@@ -163,27 +161,32 @@ async function loadOpenApi3PreviewPanel(
               "Failed to generate OpenAPI3 files.",
               result?.stderr ? [result.stderr] : [],
             );
-            return;
-          } else {
-            return await selectAndGetOpenApi3FilePath(
-              mainTspFile,
-              outputFolder,
-              selectOutput,
-              context,
-            );
+            telemetryClient.logOperationDetailTelemetry(tel.activityId, {
+              error: `Failed to compile OpenAPI3: exitCode=${result?.exitCode ?? "N/A"}, stderr=${result?.stderr ?? "N/A"}`,
+            });
+            return { code: ResultCode.Fail };
+          }
+          const filePath = await selectAndGetOpenApi3FilePath(
+            mainTspFile,
+            outputFolder,
+            selectOutput,
+            context,
+          );
+          if (filePath === undefined) {
+            return { code: ResultCode.Cancelled };
           }
+          return { code: ResultCode.Success, value: filePath };
         },
       );
     };
 
-    const filePath = await getOpenApi3OutputFilePath(true);
-    if (filePath === undefined) {
-      telemetryClient.logOperationDetailTelemetry(tel.activityId, {
-        error: "Failed to get generated OpenAPI3 file",
-      });
-      tel.lastStep = "Get OpenAPI3 output";
-      return ResultCode.Cancelled;
+    const outputResult = await getOpenApi3OutputFilePath(true);
+    if (outputResult.code !== ResultCode.Success) {
+      tel.lastStep =
+        outputResult.code === ResultCode.Fail ? "Compile OpenAPI3 failed" : "Get OpenAPI3 output";
+      return outputResult.code;
     }
+    const filePath = outputResult.value;
 
     const panel = vscode.window.createWebviewPanel(
       "webview",
@@ -199,11 +202,11 @@ async function loadOpenApi3PreviewPanel(
 
     const watch = vscode.workspace.createFileSystemWatcher("**/*.{tsp}");
     const throttledChangeHandler = throttle(async () => {
-      const outputFilePath = await getOpenApi3OutputFilePath(false);
-      if (outputFilePath) {
+      const refreshResult = await getOpenApi3OutputFilePath(false);
+      if (refreshResult.code === ResultCode.Success) {
         void panel.webview.postMessage({
           command: "load",
-          param: panel.webview.asWebviewUri(vscode.Uri.file(outputFilePath)).toString(),
+          param: panel.webview.asWebviewUri(vscode.Uri.file(refreshResult.value)).toString(),
         });
       }
     }, 1000);
@@ -233,6 +236,7 @@ async function loadOpenApi3PreviewPanel(
 
     loadHtml(context.extensionUri, panel);
   }
+  tel.lastStep = "Preview panel opened";
   return ResultCode.Success;
 }