Bug: the mux front-end TCP port is shared across Node `cluster` workers, violating srt's own per-process proxy model → intermittent 407 in restricted mode

[Designerxsh]

Summary

srt is built around a one-process-one-proxy model: the mux backend binds a pid-scoped unix socket, and each process mints its own random proxyAuthToken (the code comment even says the token is set "only when this process owns the proxy"). But the mux front-end is bound with a plain server.listen(port, '127.0.0.1') — no exclusive flag. Under Node's cluster module (how PM2 exec_mode: cluster runs an app), net.Server.listen() is intercepted and the listen handle is shared across all workers, and the primary round-robins connections across them.

So a sandboxed command spawned by worker A (its env carries A's token) connects to the shared mux port, gets round-robined to worker B, and B's proxy checks A's token against B's proxyAuthToken → mismatch → 407:

HTTP/1.1 407 Proxy Authentication Required
Proxy-Authenticate: Basic realm="srt"

This is not a missing feature — it is an internal inconsistency: the front-end port sharing contradicts the per-process isolation the rest of the module (pid-scoped backend socket, per-process token) already assumes. The failure rate scales as ≈ (N-1)/N for N workers (e.g. ~67% at N = 3 in the reproduction below); a retry often lands on the right worker and succeeds. Single-process (fork / 1-worker) deployments are unaffected.

Root cause

srt's design intent is per-process isolation. Two places make this explicit:

1. The mux backend binds a pid-scoped unix socket — sandbox/mux-proxy.js:

   function unixSocketPath() {
     return join(tmpdir(), `srt-mux-${process.pid}-${(backendSeq++).toString(36)}.sock`);
   }

2. The auth token is per-process, and the comment frames it as process-owned — sandbox/sandbox-manager.js:

   // The auth token is only set when this process owns the proxy; an
   // external proxy (config.network.httpProxyPort) handles its own auth ...
   proxyAuthToken = config.network.httpProxyPort !== undefined
     ? undefined
     : randomBytes(16).toString('hex');   // distinct per worker

But the front-end binding breaks that assumption under cluster. In startMuxProxyServer() — sandbox/sandbox-manager.js:

await listenInRange(mux.server, p => mux.server.listen(p, '127.0.0.1'), portRange, ...);
// non-Windows: portRange is undefined -> listens on 127.0.0.1:0, no `exclusive` flag

net.Server.listen() under Node cluster defaults to exclusive: false, so the primary shares one listen handle across all workers. Every worker therefore reports the same muxPort, which each worker then bakes into the token URL for the commands it spawns — sandbox/sandbox-utils.js:

const auth = proxyAuthToken ? `srt:${proxyAuthToken}@` : '';
// -> http_proxy=http://srt:<this-worker-token>@localhost:<shared-muxPort>

Now worker A's command connects to the shared port, is round-robined to worker B, and B's auth check rejects A's token — sandbox/http-proxy.js:

const checkAuth = (got) => {
  if (!options.proxyAuthToken) return { ok: true };
  const m = /^basic\s+([a-z0-9+/=]+)\s*$/i.exec(got ?? '');
  if (!m) return { ok: false };
  const decoded = Buffer.from(m[1], 'base64').toString('utf8');
  const sep = decoded.indexOf(':');
  if (sep <= 0 || decoded.slice(sep + 1) !== options.proxyAuthToken) { // A's token !== B's token
    return { ok: false };
  }
  return { ok: true, encodedCommand: encodedCommandFromProxyUser(decoded.slice(0, sep)) };
};

→ 407, emitted at both the CONNECT and plain-HTTP paths with realm="srt".

The backend is isolated per pid and the token is minted per process, but the front-end port is the one piece that leaks across processes — so the design's own invariant (a command always reaches the proxy that issued its token) is violated whenever srt runs under cluster.

Minimal reproduction (pure @anthropic-ai/sandbox-runtime, no wrapper)

// repro.mjs — run: node repro.mjs   (in a project that has @anthropic-ai/sandbox-runtime installed)
import cluster from "node:cluster";
import { spawn } from "node:child_process";
import os from "node:os";
import { SandboxManager } from "@anthropic-ai/sandbox-runtime";
const N = 3, HOST = "example.com";
if (cluster.isPrimary) {
  let d = 0; for (let i = 0; i < N; i++) cluster.fork();
  cluster.on("exit", () => { if (++d === N) process.exit(0); });
} else {
  await SandboxManager.initialize({
    network: { allowedDomains: [HOST], deniedDomains: [] },
    filesystem: { allowWrite: [os.tmpdir()], denyWrite: [], denyRead: [] },
  }, undefined, false);
  const wrun = (cmd) => new Promise(async (res) => {
    const { argv, env } = await SandboxManager.wrapWithSandboxArgv(cmd, undefined, undefined, undefined);
    const p = spawn(argv[0], argv.slice(1), { shell: false, env });
    let o = ""; p.stdout.on("data", d => o += d); p.stderr.on("data", d => o += d); p.on("close", () => res(o));
  });
  const disc = await wrun("printenv http_proxy");
  const port = (disc.match(/@[^:]+:(\d+)/) || [])[1];
  const tok = (disc.match(/:([0-9a-f]{8})[0-9a-f]*@/) || [])[1];
  const cls = (o) => /407|realm="srt"/i.test(o) ? "407" : /< HTTP\/1\.1 200/.test(o) ? "OK" : "OTHER";
  const c = {};
  for (let r = 0; r < 3; r++) {
    const rs = await Promise.all(Array.from({ length: 8 }, () => wrun(`curl -sv --max-time 6 http://${HOST}/ 2>&1`)));
    for (const o of rs) c[cls(o)] = (c[cls(o)] ?? 0) + 1;
  }
  console.error(`worker pid=${process.pid} muxPort=${port} token8=${tok} => ${JSON.stringify(c)}`);
  process.exit(0);
}

Observed on 0.0.67 (3 workers):

worker pid=47253 muxPort=59010 token8=cef7c280 => {"407":16,"OK":8}
worker pid=47251 muxPort=59010 token8=11a649f1 => {"407":13,"OK":11}
worker pid=47252 muxPort=59010 token8=c8ed79fd => {"407":15,"OK":9}

All workers share muxPort 59010; each has a distinct token; ~61% (44/72) get 407. Running the same processes standalone (not via cluster) gives distinct ports and 0 × 407.

Still present on 0.0.71 (current latest)

The same repro.mjs, run against a clean install of 0.0.71, reproduces identically:

worker pid=8502 muxPort=53775 token8=f22dc2b0 => {"407":17,"OK":5,"OTHER":2}
worker pid=8501 muxPort=53775 token8=f8ef7089 => {"407":14,"OK":7,"OTHER":3}
worker pid=8503 muxPort=53775 token8=409a04e9 => {"407":17,"OK":5,"OTHER":2}

All three workers again share one muxPort (53775) with distinct tokens; ~67% (48/72) get 407.

The relevant code is byte-for-byte unchanged from 0.0.67 to 0.0.71: the mux.server.listen(p, '127.0.0.1') bind (no exclusive), the pid-scoped backend socket, the per-worker randomBytes(16) token, and the checkAuth token comparison. The only proxy-path change across 0.0.68–0.0.71 is that checkAuth now returns { ok, encodedCommand } (encoding the command into the proxy user field for attribution) — orthogonal to this bug; the password field is still strictly compared against the per-worker token, so cross-worker delivery still 407s. (Note 0.0.71 also added the same username/password auth to the SOCKS front-end, so a SOCKS deployment under cluster will exhibit the same cross-worker 407.)

Confirming the shared handle is the trigger

A minimal net server under cluster shows the handle-sharing directly, and that exclusive: true fixes it:

listen({ port: 0, host: "127.0.0.1", exclusive: false }) -> worker ports [58956,58956,58956,58956]  (distinct=1, shared)
listen({ port: 0, host: "127.0.0.1", exclusive: true  }) -> worker ports [58957,58958,58959,58960]  (distinct=4, private)

Suggested fix

Both options restore the per-process isolation the rest of the module already assumes, and neither changes the exposure surface — the front-end stays bound to 127.0.0.1 either way; exclusive only controls whether sibling cluster workers share the socket, not who can reach it (the token remains the access control).

Option 1 (minimal, verified): bind the mux front-end with exclusive: true so each worker gets its own handle/port even under cluster:

// sandbox-manager.js startMuxProxyServer()
await listenInRange(
  mux.server,
  p => mux.server.listen({ port: p, host: '127.0.0.1', exclusive: true }),
  portRange, ...
);

The exclusive: true check above confirms this yields per-worker distinct ports under cluster. Each worker then injects its own muxPort into its own commands → no cross-worker delivery → no 407. Keeps the current TCP architecture.

Option 2 (matches the existing design most closely): bind the mux front-end on a pid-scoped unix socket, exactly as the mux backend already does (srt-mux-${process.pid}-…​.sock). Node cluster does not share pipe servers bound to distinct paths, so workers stay isolated by construction — and the front-end would then follow the same per-pid convention as the backend.

Impact & workaround

Any consumer running srt in allowlist/restricted mode under Node cluster (PM2 exec_mode: cluster is a very common Node deployment) gets a high, intermittent 407 rate on all sandboxed egress — effectively unusable in cluster + restricted mode. Current workaround is to run one process per listener (PM2 fork mode behind a front LB, or one process per container/replica), which is a non-trivial topology change for cluster-based fleets. Upgrading through the latest release (0.0.71) does not address it.