fix(tunnel): add network-wait mode for transient DNS outages

The watchdog used to burn all 10 restart attempts during any network
outage longer than ~6 minutes and then permanently give up, leaving the
tunnel dead even after connectivity returned (common overnight: Wi-Fi
sleep, DHCP renewal, ISP DNS blip, EAI_NONAME on macOS).

Now DNS / connectivity errors route to a new network-wait state that
mirrors the existing auth-wait pattern: probes the DevTunnel host via
DNS every 60s and resumes automatically when reachable. Network errors
no longer consume restart attempts, and the 10-attempt ceiling
transitions into network-wait instead of a terminal giveup.

- src/tunnel/index.js: NETWORK_ERROR_PATTERNS, isNetworkError,
  isNetworkReachable, startNetworkWait/stopNetworkWait; new
  'network-lost' / 'network-restored' events
- test/tunnel/watchdog.test.js: unit tests for classifiers + events
- docs/architecture.md, docs/troubleshooting.md: describe new behavior

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: dorlugasigal

docs/architecture.md

@@ -117,10 +117,11 @@ Manages Azure DevTunnel lifecycle: login, create, host, cleanup. Includes a **wa
 - **Health check** — every 30 seconds, runs `devtunnel show` and parses the host connection count.
 - **Zombie detection** — if host connections drop to 0 for two consecutive checks (60s grace), the stale process is killed and a restart is initiated.
 - **Crash detection** — an `exit` handler on the child process triggers immediate restart if the process dies.
-- **Auto-restart** — exponential backoff (1s → 2s → 5s → 10s → 15s → 30s), up to 10 attempts before giving up.
+- **Auto-restart** — exponential backoff (1s → 2s → 5s → 10s → 15s → 30s), up to 10 attempts before transitioning to network-wait.
 - **Auth-wait system** — detects auth token expiry (Microsoft limitation), enters an auth-wait mode, polls for re-authentication via device code flow, and auto-reconnects once a fresh token is obtained.
+- **Network-wait system** — detects DNS / connectivity errors (e.g. `ENOTFOUND`, `EAI_AGAIN`, `nodename nor servname`), enters a network-wait mode, probes the DevTunnel host via DNS every 60 seconds, and auto-reconnects once the network is reachable again. Network errors do not consume restart attempts, so transient outages (Wi-Fi sleep, DHCP renewal, ISP DNS blips) no longer cause a permanent giveup.
 - **Token lifetime monitoring** — tracks the remaining lifetime of the DevTunnel auth token and emits warnings when less than 1 hour remains, giving the frontend time to prompt the user.
-- **Event emitter** — exports `tunnelEvents` (EventEmitter) with events: `connected`, `disconnected`, `reconnecting`, `failed`. The server subscribes for logging.
+- **Event emitter** — exports `tunnelEvents` (EventEmitter) with events: `connected`, `disconnected`, `reconnecting`, `network-lost`, `network-restored`, `failed`. The server subscribes for logging.
 
 Also exports `getLoginInfo()` (returns current auth provider and token expiry) and `parseLoginInfo()` (parses raw `devtunnel` CLI output into structured login metadata).
 

docs/troubleshooting.md

@@ -79,6 +79,12 @@ devtunnel user login
 !!! warning
 If a persisted tunnel stops working after ~30 days, delete `~/.termbeam/tunnel.json` and restart TermBeam to create a fresh tunnel.
 
+### Tunnel died overnight / after sleep
+
+If TermBeam is running as a long-lived service (`termbeam service`) and the tunnel stops working after a network interruption (laptop sleep, Wi-Fi drop, DHCP renewal, ISP DNS blip, etc.), the watchdog automatically enters a **network-wait** state once transient DNS / connectivity errors are detected. It probes the DevTunnel host every 60 seconds and reconnects as soon as the network is reachable again — no manual restart required.
+
+You'll see `[WARN] Tunnel paused — waiting for network connectivity` in the logs followed by `[INFO] Network connectivity restored — resuming tunnel` when it recovers. If the logs instead show repeated `Tunnel restart returned no URL` with no final giveup, the watchdog is still cycling through its 10 restart attempts; wait a few minutes for it to settle into network-wait.
+
 ---
 
 ## Authentication Issues

package-lock.json

@@ -2,6 +2,7 @@ const { execSync, execFileSync, execFile, spawn } = require('child_process');
 const path = require('path');
 const fs = require('fs');
 const os = require('os');
+const dns = require('dns');
 const EventEmitter = require('events');
 const log = require('../utils/logger');
 const { promptInstall } = require('./install');
@@ -26,15 +27,38 @@ let waitingForAuth = false;
 let authCheckInterval = null;
 let expiryWarned = false;
 
+// --- Network-wait state ---
+let waitingForNetwork = false;
+let networkWaitInterval = null;
+
 const HEALTH_CHECK_INTERVAL = 30_000; // 30s between checks
 const HEALTH_CHECK_GRACE = 2; // 2 consecutive failures before restart
 const MAX_RESTART_ATTEMPTS = 10;
 const BACKOFF_DELAYS = [1000, 2000, 5000, 10_000, 15_000, 30_000]; // then stays at 30s
 const AUTH_CHECK_INTERVAL = 30_000; // 30s between auth re-checks
+const NETWORK_CHECK_INTERVAL = 60_000; // 60s between network reachability probes
+const NETWORK_PROBE_HOST = 'global.rel.tunnels.api.visualstudio.com';
+const NETWORK_PROBE_TIMEOUT = 5_000;
 const TOKEN_EXPIRY_WARN_SECONDS = 3600; // warn at 1 hour remaining
 
 const AUTH_ERROR_PATTERNS = ['login required', 'not logged in', 'sign in required'];
 
+// DNS / transient network failures that should NOT burn restart attempts.
+// These typically resolve on their own once the host regains connectivity
+// (e.g. Wi-Fi sleep/wake, router reboot, upstream DNS hiccup).
+const NETWORK_ERROR_PATTERNS = [
+  'nodename nor servname',
+  'getaddrinfo',
+  'enotfound',
+  'eai_again',
+  'econnrefused',
+  'econnreset',
+  'etimedout',
+  'network is unreachable',
+  'no such host',
+  'temporary failure in name resolution',
+];
+
 const SAFE_ID_RE = /^[a-zA-Z0-9._-]+$/;
 
 const DEVICE_CODE_INITIAL_TIMEOUT = 15000;
@@ -45,6 +69,37 @@ function isAuthError(message) {
   return AUTH_ERROR_PATTERNS.some((p) => lower.includes(p));
 }
 
+function isNetworkError(message) {
+  const lower = (message || '').toLowerCase();
+  return NETWORK_ERROR_PATTERNS.some((p) => lower.includes(p));
+}
+
+function isNetworkReachable() {
+  return new Promise((resolve) => {
+    let settled = false;
+    const timer = setTimeout(() => {
+      if (!settled) {
+        settled = true;
+        resolve(false);
+      }
+    }, NETWORK_PROBE_TIMEOUT);
+    try {
+      dns.lookup(NETWORK_PROBE_HOST, (err) => {
+        if (settled) return;
+        settled = true;
+        clearTimeout(timer);
+        resolve(!err);
+      });
+    } catch {
+      if (!settled) {
+        settled = true;
+        clearTimeout(timer);
+        resolve(false);
+      }
+    }
+  });
+}
+
 function isLoggedIn() {
   try {
     const out = execFileSync(devtunnelCmd, ['user', 'show'], {
@@ -210,7 +265,7 @@ let isPersisted = false;
 // --- Watchdog: health check & auto-restart ---
 
 function checkTunnelHealth() {
-  if (!tunnelId || !tunnelProc || isRestarting || waitingForAuth) return;
+  if (!tunnelId || !tunnelProc || isRestarting || waitingForAuth || waitingForNetwork) return;
 
   const abortCtrl = new AbortController();
   const timer = setTimeout(() => abortCtrl.abort(), 10_000);
@@ -229,6 +284,17 @@ function checkTunnelHealth() {
           return;
         }
 
+        // Transient network errors (DNS, connection refused): the host has
+        // lost connectivity. Don't burn restart attempts — wait for network.
+        if (isNetworkError(err.message) || isNetworkError(err.stderr)) {
+          log.warn(`Tunnel health check: network unreachable — pausing until connectivity returns`);
+          stopHealthCheck();
+          killTunnelProc();
+          tunnelEvents.emit('disconnected');
+          startNetworkWait();
+          return;
+        }
+
         // "Tunnel not found" can mean the user's auth expired (CLI can't
         // query the tunnel without valid credentials). Check login status
         // to distinguish from a genuinely deleted tunnel.
@@ -389,13 +455,58 @@ function stopAuthWait() {
   }
 }
 
+function startNetworkWait() {
+  if (waitingForNetwork) return;
+  waitingForNetwork = true;
+  isRestarting = false;
+  restartAttempts = 0;
+  consecutiveFailures = 0;
+
+  log.warn('Tunnel paused — waiting for network connectivity.');
+  log.warn('Will auto-resume when the tunnel service is reachable.');
+  tunnelEvents.emit('network-lost');
+
+  const probe = async () => {
+    if (!waitingForNetwork) return;
+    // If auth expired while we were offline, switch to auth-wait instead.
+    if (!isLoggedIn()) {
+      log.warn('DevTunnel auth expired during network outage — waiting for re-authentication');
+      stopNetworkWait();
+      handleAuthExpiration();
+      return;
+    }
+    if (await isNetworkReachable()) {
+      log.info('Network connectivity restored — resuming tunnel');
+      stopNetworkWait();
+      tunnelEvents.emit('network-restored');
+      scheduleRestart();
+    }
+  };
+
+  networkWaitInterval = setInterval(probe, NETWORK_CHECK_INTERVAL);
+  networkWaitInterval.unref();
+  // Also probe once immediately in case the outage already cleared.
+  probe();
+}
+
+function stopNetworkWait() {
+  waitingForNetwork = false;
+  if (networkWaitInterval) {
+    clearInterval(networkWaitInterval);
+    networkWaitInterval = null;
+  }
+}
+
 function scheduleRestart() {
+  if (waitingForNetwork || waitingForAuth) return;
+
   if (restartAttempts >= MAX_RESTART_ATTEMPTS) {
-    log.error(
-      `Tunnel restart failed after ${MAX_RESTART_ATTEMPTS} attempts — giving up. Tunnel URL is unreachable.`,
+    log.warn(
+      `Tunnel restart failed after ${MAX_RESTART_ATTEMPTS} attempts — entering network-wait mode.`,
     );
     tunnelEvents.emit('failed', { attempts: restartAttempts });
     isRestarting = false;
+    startNetworkWait();
     return;
   }
 
@@ -428,11 +539,20 @@ function scheduleRestart() {
       } else {
         log.warn('Tunnel restart returned no URL');
         isRestarting = false;
+        // If the host appears to be offline, stop burning attempts.
+        if (!(await isNetworkReachable())) {
+          startNetworkWait();
+          return;
+        }
         scheduleRestart();
       }
     } catch (err) {
       log.error(`Tunnel restart error: ${err.message}`);
       isRestarting = false;
+      if (isNetworkError(err.message) || !(await isNetworkReachable())) {
+        startNetworkWait();
+        return;
+      }
       scheduleRestart();
     }
   }, delay);
@@ -635,9 +755,10 @@ async function startTunnel(port, options = {}) {
 }
 
 function cleanupTunnel() {
-  // Stop watchdog and auth-wait to prevent restart during cleanup
+  // Stop watchdog, auth-wait, and network-wait to prevent restart during cleanup
   stopHealthCheck();
   stopAuthWait();
+  stopNetworkWait();
   isRestarting = true; // prevent exit handler from restarting
   if (restartTimer) {
     clearTimeout(restartTimer);
@@ -674,4 +795,9 @@ module.exports = {
   tunnelEvents,
   getLoginInfo,
   parseLoginInfo,
+  // Exported for tests
+  _internal: {
+    isNetworkError,
+    isAuthError,
+  },
 };

src/tunnel/index.js

@@ -0,0 +1,67 @@
+const { describe, it } = require('node:test');
+const assert = require('node:assert/strict');
+
+const tunnel = require('../../src/tunnel');
+const { isNetworkError, isAuthError } = tunnel._internal;
+
+describe('tunnel watchdog classification', () => {
+  it('isNetworkError detects common DNS/connectivity failures', () => {
+    const cases = [
+      'nodename nor servname provided, or not known',
+      'getaddrinfo ENOTFOUND uks1.rel.tunnels.api.visualstudio.com',
+      'connect ECONNREFUSED 20.0.0.1:443',
+      'connect ETIMEDOUT',
+      'Network is unreachable',
+      'Temporary failure in name resolution',
+      'EAI_AGAIN some-host',
+    ];
+    for (const msg of cases) {
+      assert.equal(isNetworkError(msg), true, `expected network error: ${msg}`);
+    }
+  });
+
+  it('isNetworkError returns false for non-network messages', () => {
+    assert.equal(isNetworkError('login required'), false);
+    assert.equal(isNetworkError('tunnel not found'), false);
+    assert.equal(isNetworkError(''), false);
+    assert.equal(isNetworkError(null), false);
+    assert.equal(isNetworkError(undefined), false);
+  });
+
+  it('isAuthError detects auth-related messages', () => {
+    assert.equal(isAuthError('Login required'), true);
+    assert.equal(isAuthError('not logged in'), true);
+    assert.equal(isAuthError('Sign in required'), true);
+  });
+
+  it('isAuthError ignores network errors', () => {
+    assert.equal(isAuthError('getaddrinfo ENOTFOUND'), false);
+    assert.equal(isAuthError('ECONNREFUSED'), false);
+  });
+});
+
+describe('tunnel watchdog public surface', () => {
+  it('exposes tunnelEvents emitter', () => {
+    assert.ok(tunnel.tunnelEvents);
+    assert.equal(typeof tunnel.tunnelEvents.on, 'function');
+    assert.equal(typeof tunnel.tunnelEvents.emit, 'function');
+  });
+
+  it('network-lost/network-restored events are emitable', () => {
+    let lost = 0;
+    let restored = 0;
+    const onLost = () => lost++;
+    const onRestored = () => restored++;
+    tunnel.tunnelEvents.on('network-lost', onLost);
+    tunnel.tunnelEvents.on('network-restored', onRestored);
+    try {
+      tunnel.tunnelEvents.emit('network-lost');
+      tunnel.tunnelEvents.emit('network-restored');
+      assert.equal(lost, 1);
+      assert.equal(restored, 1);
+    } finally {
+      tunnel.tunnelEvents.off('network-lost', onLost);
+      tunnel.tunnelEvents.off('network-restored', onRestored);
+    }
+  });
+});