v1.28.0.0 feat: browse --headed/--proxy/--navigate + gstack/llms.txt + webdriver-only stealth (#1363)

* feat(browse): SOCKS5 bridge with auth + cred redaction helper

Adds browse/src/socks-bridge.ts: a 127.0.0.1-only SOCKS5 listener that
accepts unauthenticated connections from Chromium and relays them through
an authenticated upstream proxy. Chromium does not prompt for SOCKS5 auth
at launch, so this bridge is the workaround for using auth-required
residential SOCKS5 upstreams.

- startSocksBridge({ upstream, port: 0 }) → ephemeral 127.0.0.1 listener
- testUpstream({ upstream, retries: 3, backoffMs: 500, budgetMs: 5000 })
  pre-flight that connects to a known endpoint (default 1.1.1.1:443)
- Stream-error policy: kill affected client + upstream sockets on any
  error mid-stream; no transport retries (a transport-layer retry can
  corrupt browser traffic)

Adds browse/src/proxy-redact.ts: single source of truth for redacting
credentials in any logged proxy URL or upstream config. Every code path
that prints proxy config goes through this helper.

Adds the socks npm dep (~30KB) and 16 tests covering: 127.0.0.1-only
bind, byte-for-byte round trip through the bridge, auth rejection,
mid-stream upstream drop kills client conn, listener teardown,
testUpstream success + retry-exhaust paths, redaction of every
credential shape.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(browse): --proxy and --headed flags wire bridge into daemon

Adds the global --proxy <url> and --headed flags to the browse CLI.
Resolves cred policy and routes the daemon launch through the SOCKS5
bridge (or pass-through for HTTP/HTTPS) before chromium.launch().

CLI (cli.ts):
- extractGlobalFlags() strips --proxy/--headed from argv, parses URL via
  Node URL class, validates D9 cred-mixing (env BROWSE_PROXY_USER/PASS
  + URL creds → exit 1 with hint), composes canonical proxy URL with
  resolved creds, computes a stable configHash for daemon-mismatch
- ensureServer() now reads existing daemon's configHash from state file
  and refuses (exit 1 with disconnect hint) if --proxy/--headed mismatch
  the existing daemon. No silent restart that would drop tab state.
- All proxy-related stderr lines go through redactProxyUrl

proxy-config.ts (new):
- parseProxyConfig() — URL parser + D9 cred-mixing detector + scheme allowlist
- computeConfigHash() — stable hash of (proxy URL minus creds + headed flag)
- toUpstreamConfig() — map ParsedProxyConfig → socks-bridge.UpstreamConfig

Server (server.ts):
- Reads BROWSE_PROXY_URL at startup; for SOCKS5+auth, runs testUpstream
  pre-flight (5s budget, 3 retries, 500ms backoff) and exits 1 on failure
  with redacted error
- Spawns startSocksBridge() on 127.0.0.1:<ephemeral> and points
  Chromium at it via socks5://127.0.0.1:<port>
- HTTP/HTTPS or unauth SOCKS5 → pass-through to chromium.launch
  proxy.server (with username/password if present)
- State file gains optional configHash for daemon-mismatch check
- Bridge tears down via process.on('exit')

Browser manager (browser-manager.ts):
- New setProxyConfig({ server, username, password }) called by server.ts
  before launch
- chromium.launch() and both launchPersistentContext sites pass the
  proxy config through when set

Tests: 22 new across proxy-config (parse + cred-mixing + hash stability)
and extractGlobalFlags (flag stripping + cred-mixing rejection + cred
rotation hash stability + redaction).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(browse): Xvfb auto-spawn with PID + start-time validation

Adds browse/src/xvfb.ts: a Linux-only Xvfb auto-spawn module for
running headed Chromium in containers without DISPLAY. The module
walks a display range to pick a free one (never hardcodes :99) and
validates orphan PIDs by BOTH /proc/<pid>/cmdline matching 'Xvfb' AND
start-time matching the recorded value before sending any signal.
Defends against PID reuse — refuses to kill anything that doesn't
match both checks.

- shouldSpawnXvfb(env, platform) — pure decision: skip on macOS/Windows,
  on Linux skip when DISPLAY or WAYLAND_DISPLAY is set (codex F2)
- pickFreeDisplay(99..120) — probes via xdpyinfo
- spawnXvfb(display) — returns { pid, startTime, display } handle
- isOurXvfb(pid, startTime) — both-checks validator
- cleanupXvfb(state) — best-effort, validates ownership before SIGTERM

Wired into server.ts startup: when shouldSpawnXvfb says yes, picks a
free display, spawns Xvfb, sets DISPLAY for chromium.launchHeaded, and
records xvfbPid/xvfbStartTime/xvfbDisplay in the state file. Cleanup
runs on process.on('exit'). The CLI's disconnect path also runs
cleanupXvfb() in the force-cleanup branch when the server is dead.

Disconnect now applies to any non-default daemon (headed mode OR
configHash-tagged daemon — i.e. one started with --proxy/--headed),
not just headed mode.

Adds xvfb + x11-utils to .github/docker/Dockerfile.ci so CI exercises
the Linux container --headed path on every run. Without it the most
common production path would go untested.

Tests: 17 new across decision logic, PID validation defenses
(cmdline mismatch, start-time mismatch), no-op safety on bad inputs,
and a Linux+Xvfb-installed gate for the spawn → validate → cleanup
round trip. Tests skip on macOS/Windows automatically.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(browse): webdriver-mask stealth + Chromium-through-bridge e2e

D7 (codex narrowing): mask navigator.webdriver only via addInitScript.
The wintermute approach (fake plugins=[1..5], fake languages=['en-US',
'en'], stub window.chrome) is intentionally NOT applied — modern
fingerprinters check consistency between plugins.length, languages,
userAgent, and platform, and synthesizing fixed values can flag MORE
bot-like, not less. The honest minimum is webdriver, which Chromium
exposes as a known automation tell.

Adds browse/src/stealth.ts: single source of truth for the stealth
init script and launch args. Both browser-manager.launch() (headless)
and launchHeaded() (persistent context with extension) call
applyStealth(context) and pass STEALTH_LAUNCH_ARGS into chromium.launch.

The pre-existing launchHeaded stealth that did fake plugins/languages
is removed for the same reason. The cdc_/__webdriver runtime cleanup
and Permissions API patch are kept — they remove automation-injected
artifacts, not synthesize fake natural-browser values.

Adds bridge-chromium-e2e.test.ts (codex F3): the test that proves the
FEATURE works. Real Chromium with proxy.server = 'socks5://127.0.0.1:
<bridgePort>' navigates to a local HTTP fixture; the auth upstream's
connect counter and the HTTP fixture's hit counter both increment,
proving traffic actually traversed bridge → auth-upstream → destination.
Without this test, we could ship a working byte-relay and a broken
Chromium integration and never know.

Adds bridge-port-restart.test.ts (codex F1, reframed): old test
assumed two daemons coexist, which contradicts D2 single-daemon model.
Reframed as restart-then-restart, asserting fresh ephemeral ports
(never the hardcoded 1090) on each spin-up.

Adds stealth-webdriver.test.ts: navigator.webdriver=false in both
fresh contexts and persistent contexts; navigator.plugins/languages
are NOT replaced with the wintermute fake list (D7 verification).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(gstack): generate llms.txt — single-file capability index for AI agents

Adds scripts/gen-llms-txt.ts: produces gstack/llms.txt at repo root,
indexing every skill (47), every browse command (75), and design
commands when the design CLI is present. Per the llmstxt.org
convention, agents can read one file to learn what gstack offers
instead of crawling 47 SKILL.md files.

Sources:
- skill SKILL.md.tmpl frontmatter (name + description block scalar)
- browse/src/commands.ts COMMAND_DESCRIPTIONS (sorted by category)
- design/src/commands.ts COMMAND_DESCRIPTIONS if present (best-effort)

Wired into scripts/gen-skill-docs.ts as a post-step so it regenerates
on every `bun run gen:skill-docs` (the same script that re-emits all
SKILL.md files). Failures are non-fatal warnings, not build breaks —
the generator never blocks SKILL.md regen.

Strict mode (--strict, also used by tests) throws when a skill is
missing name or description in its frontmatter, catching missing
metadata before it ships.

Tests: shape (top-level sections, sort order, single-line summary
discipline), every-skill-and-command-appears, strict-mode rejection of
incomplete frontmatter, and freshness check that the committed
gstack/llms.txt matches what the generator produces now.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(browse): --navigate flag on download for browser-triggered files

Adds the --navigate strategy from community PR #1355 (originally from
@garrytan-agents). When set, download navigates to the URL with
waitUntil:'commit' and captures the resulting browser download via
page.waitForEvent('download'), then saves via download.saveAs().
Handles URLs that trigger files via Content-Disposition headers,
multi-hop CDN redirects requiring browser cookies, or anti-bot CDN
chains where page.request.fetch() can't follow the auth/redirect
chain.

Defaults still use the existing direct-fetch strategy. --navigate is
opt-in.

Goes through the same validateNavigationUrl SSRF gate as goto, so
download --navigate cannot reach IPv4 metadata endpoints (AWS IMDSv1,
GCP/Azure equivalents) or arbitrary internal hosts.

Inferred content type from suggested filename for common extensions
(epub, pdf, zip, gz, mp3/mp4, jpg/jpeg/png, txt, html, json) — falls
back to application/octet-stream. Same 200MB cap as Strategy 1.

Frames the use case generically (anti-bot CDN, Content-Disposition,
redirect chains) rather than naming any specific site, per project
voice rules.

Co-Authored-By: @garrytan-agents
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: v1.28.0.0 — browse SKILL section + VERSION + CHANGELOG

VERSION 1.27.1.0 → 1.28.0.0 (MINOR — substantial new capability:
five new flags/features, ~600 LOC added, new socks dep, multiple
new modules).

browse/SKILL.md.tmpl: new "Headed Mode + Proxy + Anti-Bot Sites"
section between User Handoff and Snapshot Flags. Documents
--headed (auto-Xvfb on Linux), --proxy (with embedded SOCKS5
bridge for auth), download --navigate, the cred-mixing policy,
daemon-discipline (refuse-on-mismatch), the narrowed
webdriver-only stealth, container support caveats, and the
fail-fast/no-retry failure modes.

CHANGELOG entry follows the release-summary format from CLAUDE.md:
two-line headline, lead paragraph, "The numbers that matter"
table tied to specific test files that prove each capability,
"What this means for AI agents" closing tied to a real workflow
shift, then itemized Added/Changed/Fixed/For-contributors
sections.

Browse SKILL.md regenerated via bun run gen:skill-docs.
gstack/llms.txt regenerated automatically from the same pipeline.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(browse): integration coverage for daemon mismatch + proxy fail-fast

Adds two integration tests that exercise the full process boundary,
not just the module-level wiring.

daemon-mismatch-refuse.test.ts (D2):
- Stubs a healthy state file with a fake configHash and a fake /health
  HTTP server, runs the actual cli.ts binary with a mismatching
  --proxy, asserts exit 1 + 'different config' / 'browse disconnect'
  hint in stderr.
- Same shape with the plain-daemon-meets---headed case.
- Positive case: matching configHash → CLI does NOT emit the mismatch
  hint (regardless of whether the actual command succeeds).

server-proxy-fail-fast.test.ts:
- Starts the rejecting SOCKS5 upstream, spawns server.ts with
  BROWSE_PROXY_URL pointing at it, BROWSE_HEADLESS_SKIP=1 to skip
  Chromium launch.
- Asserts exit 1, 'FAIL upstream' in stderr (testUpstream pre-flight
  ran), no raw credential leakage in any output (redaction works on
  the failure path), and exit within 30s upper bound.

Both tests use the existing spawn-bun-cli pattern from
commands.test.ts so they run on the same CI infrastructure as the
rest of the bun test suite.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(gen-skill-docs): keep module sync so test require() still works

Two regressions caught by the full test suite after the v1.28.0.0
landing pass:

1) package.json version mismatch — VERSION was bumped to 1.28.0.0
   but package.json still pinned to 1.27.1.0.
   test/gen-skill-docs.test.ts asserts they match.

2) Top-level await in scripts/gen-llms-txt.ts (CLI entry block) and
   scripts/gen-skill-docs.ts (post-step) made gen-skill-docs an
   async module. test/gen-skill-docs.test.ts uses require() to pull
   extractVoiceTriggers/processVoiceTriggers from gen-skill-docs,
   which Bun rejects on async modules with:
     "TypeError: require() async module ... unsupported.
      use 'await import()' instead."

Fix: wrap the await blocks in void IIFEs so the modules remain sync
from a require() perspective.

After fix: all 379 gen-skill-docs tests pass, all 77 new feature
tests pass (3 skipped on macOS — Linux+Xvfb gates).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(browse): apply codex adversarial findings on the new lifecycle

Codex outside-voice review caught five real production-failure modes in
the v1.28.0.0 proxy/headed lifecycle. Fixed:

1) `browse disconnect` skip-graceful for proxy-only daemons
   (browse/src/cli.ts). The graceful /command POST went out with stray
   `domains,` shorthand and (even fixed) the server's disconnect handler
   only tears down headed mode — proxy-only daemons returned 200 "Not
   in headed mode" while leaving the bridge running. Now disconnect
   short-circuits to force-cleanup for non-headed daemons, which kicks
   process.on('exit') in server.ts to close the bridge + Xvfb.

2) sendCommand crash retry preserves --proxy / --headed
   (browse/src/cli.ts). The ECONNRESET retry path called startServer()
   with no extraEnv, silently dropping the proxied flags. A daemon that
   died mid-command would silently restart in default direct/headless
   mode and bypass the SOCKS bridge. Now reapplies BROWSE_PROXY_URL,
   BROWSE_HEADED, and BROWSE_CONFIG_HASH from the resolved global flags.

3) `connect` honors --proxy (browse/src/cli.ts). The headed-mode
   `connect` command built its own serverEnv that didn't include
   BROWSE_PROXY_URL, so `browse --proxy <url> connect` launched headed
   Chromium without the proxy. Now threads proxyUrl + configHash into
   the connect serverEnv.

4) SOCKS5 bridge handles fragmented TCP frames
   (browse/src/socks-bridge.ts). Previously used once('data') and
   parsed each chunk as a complete SOCKS5 frame — TCP doesn't preserve
   message boundaries and split greetings/CONNECT requests caused
   intermittent handshake failures. Replaced with a single state
   machine that buffers chunks and uses size predicates on the SOCKS5
   header to know when a complete frame has arrived. Pauses the client
   socket during upstream connect and replays any remainder bytes
   into the upstream on success.

5) Xvfb cleanup-then-state-delete ordering
   (browse/src/server.ts). emergencyCleanup() previously deleted the
   state file BEFORE any Xvfb cleanup could read it, orphaning Xvfb
   on uncaughtException / unhandledRejection. Now reads the state
   file first, calls cleanupXvfb() (which validates cmdline +
   start-time before kill), then deletes the state file.

Adds a regression test for #4: writes the SOCKS5 greeting + CONNECT
one byte at a time with 5ms ticks, asserts a clean round trip after
the fragmented handshake.

Codex's sixth finding (bridge advertises NO_AUTH on 127.0.0.1, so any
co-located process can use the authenticated upstream) is documented
as a known limitation — gstack's threat model assumes single-user
hosts. Adding bridge-side auth is a separate change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: update BROWSER.md + TODOS.md for v1.28.0.0

BROWSER.md picks up a "Headed mode + proxy + browser-native downloads
(v1.28.0.0)" subsection inside Real-browser mode plus the new source-map
entries (socks-bridge.ts, proxy-config.ts, proxy-redact.ts, xvfb.ts,
stealth.ts). TODOS.md anti-bot-stealth item updated to reflect the v1.28
narrowing — the "fake plugins" line is no longer accurate.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(ci): include bun.lock in image build for deterministic install

CI evals all failed on PR #1363 with:
  error: Could not resolve: "smart-buffer". Maybe you need to "bun install"?
  error: Could not resolve: "ip-address". Maybe you need to "bun install"?
  at /opt/node_modules_cache/socks/build/client/socksclient.js:15

The cached node_modules layer in the pre-baked Docker image had
`socks` (the new dep) but was missing its transitive deps (smart-buffer,
ip-address). The image build copied only package.json into the build
context — without bun.lock, `bun install` resolved a different tree
than local `bun install` did, dropping required transitive deps.

Reproduces locally as 229 packages (correct) when bun.lock is present
or absent. Why CI diverged isn't fully understood — possibly Docker
layer cache reuse across image rebuilds — but the deterministic fix is
to include the lockfile in the image build context and use
`--frozen-lockfile`, matching what every CI doc recommends.

Changes:
- .github/docker/Dockerfile.ci: COPY bun.lock alongside package.json,
  switch `bun install` → `bun install --frozen-lockfile` so any future
  lockfile drift fails loudly during image build instead of producing
  a partially-installed cache that breaks downstream eval jobs.
- .github/workflows/evals.yml: include bun.lock in the image-tag hash
  so adding/removing a dep invalidates the image, AND copy bun.lock
  into the docker context alongside package.json.
- .github/workflows/evals-periodic.yml: same updates.
- .github/workflows/ci-image.yml: rebuild trigger now fires on bun.lock
  changes too; build context includes bun.lock.

Image hash changes → fresh image gets built on next CI run → install
matches the lockfile exactly → no missing transitive deps.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(ci): use hardlink copy instead of symlink for node_modules cache

After the bun.lock fix landed, the eval matrix STILL failed identically:
  Could not resolve: "smart-buffer" / "ip-address"
  at /opt/node_modules_cache/socks/build/client/socksclient.js

But the hash-tagged image actually contains smart-buffer + ip-address +
socks all flat in /opt/node_modules_cache (verified by pulling and
inspecting the image). 207 packages, all present.

Root cause: the workflow used `ln -s /opt/node_modules_cache node_modules`
to restore deps. Bun build (and Node module resolution generally) walks
a file's realpath to find sibling deps. From the symlinked
/workspace/node_modules/socks/build/client/socksclient.js, realpath
resolves to /opt/node_modules_cache/socks/build/client/socksclient.js,
and walking up to find a node_modules/smart-buffer dir fails — there's
no `node_modules` segment in the realpath.

Switch `ln -s` → `cp -al` (hardlink-copy). Each file in the cache becomes
a hardlink at /workspace/node_modules/<pkg>, sharing inodes (no data
copy). Realpath of /workspace/node_modules/socks/.../socksclient.js
stays inside /workspace/node_modules, so sibling deps resolve correctly.

Speed is comparable to symlink — `cp -al` on ~200 packages on tmpfs is
sub-second. Same caching story preserved.

Both evals.yml and evals-periodic.yml updated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(ci): cp -r instead of cp -al — /opt and /workspace are different filesystems

The hardlink-copy fix landed and immediately broke with:
  cp: cannot create hard link 'node_modules/<file>' to
      '/opt/node_modules_cache/<file>': Invalid cross-device link

GitHub Actions runners mount the workspace volume at /workspace
(overlay-fs layered onto the runner image), and /opt is the runner
image's own filesystem. Cross-filesystem hardlinks aren't supported.

Switch `cp -al` → `cp -r`. Cost: ~5s for ~200 packages of small JS
files vs ~0s for the broken symlink. Still cheaper than the ~15s
`bun install` fallback. Realpath of /workspace/node_modules/<pkg>/...
stays inside /workspace, so bun build's sibling-dep resolution works.

Both evals.yml and evals-periodic.yml updated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: garrytan

.github/docker/Dockerfile.ci

@@ -77,17 +77,26 @@ RUN npx playwright install-deps chromium
 # render in DejaVu Sans. playwright install-deps happens to pull this in today,
 # but the dep is implicit and could change — install explicitly so upgrades
 # can't silently regress rendering.
+#
+# Xvfb is also installed here so the browse --headed integration tests
+# (headed-xvfb, headed-orphan-cleanup) can exercise the Linux container
+# auto-spawn path on every CI run. Without Xvfb in the image, the most
+# common production --headed path goes untested.
 RUN for i in 1 2 3; do \
-      apt-get update && apt-get install -y --no-install-recommends fonts-liberation fontconfig && break || \
+      apt-get update && apt-get install -y --no-install-recommends fonts-liberation fontconfig xvfb x11-utils && break || \
       (echo "fonts-liberation install retry $i/3"; sleep 10); \
     done \
     && fc-cache -f \
     && rm -rf /var/lib/apt/lists/*
 
-# Pre-install dependencies (cached layer — only rebuilds when package.json changes)
-COPY package.json /workspace/
+# Pre-install dependencies (cached layer — only rebuilds when package.json or
+# bun.lock changes). Copy BOTH so install is deterministic and matches local
+# resolution. Without bun.lock here, bun install resolved transitive deps
+# differently in CI vs local (observed on v1.28.0.0: socks landed but
+# smart-buffer + ip-address didn't make it into the cached node_modules).
+COPY package.json bun.lock /workspace/
 WORKDIR /workspace
-RUN bun install && rm -rf /tmp/*
+RUN bun install --frozen-lockfile && rm -rf /tmp/*
 
 # Install Playwright Chromium to a shared location accessible by all users
 ENV PLAYWRIGHT_BROWSERS_PATH=/opt/playwright-browsers

.github/workflows/ci-image.yml

@@ -9,6 +9,7 @@ on:
     paths:
       - '.github/docker/Dockerfile.ci'
       - 'package.json'
+      - 'bun.lock'
   # Manual trigger
   workflow_dispatch:
 
@@ -22,7 +23,7 @@ jobs:
       - uses: actions/checkout@v4
 
       # Copy lockfile + package.json into Docker build context
-      - run: cp package.json .github/docker/
+      - run: cp package.json bun.lock .github/docker/
 
       - uses: docker/login-action@v3
         with:

.github/workflows/evals-periodic.yml

@@ -25,7 +25,7 @@ jobs:
       - uses: actions/checkout@v4
 
       - id: meta
-        run: echo "tag=${{ env.IMAGE }}:${{ hashFiles('.github/docker/Dockerfile.ci', 'package.json') }}" >> "$GITHUB_OUTPUT"
+        run: echo "tag=${{ env.IMAGE }}:${{ hashFiles('.github/docker/Dockerfile.ci', 'package.json', 'bun.lock') }}" >> "$GITHUB_OUTPUT"
 
       - uses: docker/login-action@v3
         with:
@@ -43,7 +43,7 @@ jobs:
           fi
 
       - if: steps.check.outputs.exists == 'false'
-        run: cp package.json .github/docker/
+        run: cp package.json bun.lock .github/docker/
 
       - if: steps.check.outputs.exists == 'false'
         uses: docker/build-push-action@v6
@@ -101,10 +101,14 @@ jobs:
             echo "TMPDIR=/home/runner/.cache"
           } >> "$GITHUB_ENV"
 
+      # Recursive copy (cp -r) instead of symlink: bun build resolves a
+      # file's realpath when looking for sibling deps. See evals.yml for the
+      # full explanation. cp -al would be faster but /opt and /workspace
+      # are on different overlay-fs layers, so cross-device hardlink fails.
       - name: Restore deps
         run: |
           if [ -d /opt/node_modules_cache ] && diff -q /opt/node_modules_cache/.package.json package.json >/dev/null 2>&1; then
-            ln -s /opt/node_modules_cache node_modules
+            cp -r /opt/node_modules_cache node_modules
           else
             bun install
           fi

.github/workflows/evals.yml

@@ -25,7 +25,7 @@ jobs:
       - uses: actions/checkout@v4
 
       - id: meta
-        run: echo "tag=${{ env.IMAGE }}:${{ hashFiles('.github/docker/Dockerfile.ci', 'package.json') }}" >> "$GITHUB_OUTPUT"
+        run: echo "tag=${{ env.IMAGE }}:${{ hashFiles('.github/docker/Dockerfile.ci', 'package.json', 'bun.lock') }}" >> "$GITHUB_OUTPUT"
 
       - uses: docker/login-action@v3
         with:
@@ -43,7 +43,7 @@ jobs:
           fi
 
       - if: steps.check.outputs.exists == 'false'
-        run: cp package.json .github/docker/
+        run: cp package.json bun.lock .github/docker/
 
       - if: steps.check.outputs.exists == 'false'
         uses: docker/build-push-action@v6
@@ -110,11 +110,19 @@ jobs:
             echo "TMPDIR=/home/runner/.cache"
           } >> "$GITHUB_ENV"
 
-      # Restore pre-installed node_modules from Docker image via symlink (~0s vs ~15s install)
+      # Restore pre-installed node_modules from Docker image via recursive
+      # copy. Symlink (`ln -s`) breaks bun's module resolution because bun
+      # resolves a file's realpath when walking up to find node_modules/<dep>;
+      # from a symlinked path, realpath escapes the workspace and sibling
+      # deps no longer resolve. Hardlink copy (`cp -al`) fails because /opt
+      # and /workspace are on different overlay-fs layers ("Invalid
+      # cross-device link"). Recursive copy works on every layout. Cost:
+      # ~5s for ~200 packages of small JS files vs ~0s for symlink — still
+      # vastly cheaper than rerunning `bun install` (network + resolution).
       - name: Restore deps
         run: |
           if [ -d /opt/node_modules_cache ] && diff -q /opt/node_modules_cache/.package.json package.json >/dev/null 2>&1; then
-            ln -s /opt/node_modules_cache node_modules
+            cp -r /opt/node_modules_cache node_modules
           else
             bun install
           fi

BROWSER.md

@@ -49,7 +49,7 @@ $B connect                       # headed Chromium + Side Panel extension
 5. [Snapshot system + ref-based selection](#snapshot-system)
 6. [Browser-skills runtime](#browser-skills-runtime)
 7. [Domain-skills (per-site agent notes)](#domain-skills)
-8. [Real-browser mode (`$B connect`)](#real-browser-mode)
+8. [Real-browser mode (`$B connect`)](#real-browser-mode) — including [`--headed` + `--proxy` + `--navigate` (v1.28.0.0)](#headed-mode--proxy--browser-native-downloads-v12800)
 9. [Side Panel + sidebar agent](#side-panel--sidebar-agent)
 10. [Pair-agent — remote agents over an ngrok tunnel](#pair-agent)
 11. [Authentication + tokens](#authentication)
@@ -545,6 +545,63 @@ When in real-browser mode, `/qa` and `/design-review` automatically skip
 cookie import prompts and headless workarounds — the headed browser already
 has whatever session you logged into.
 
+### Headed mode + proxy + browser-native downloads (v1.28.0.0)
+
+Three coordinated flags for sites that block headless browsers, fingerprint
+Playwright defaults, or sit behind authenticated upstream proxies:
+
+```bash
+# Visible Chromium. Auto-spawns Xvfb on Linux containers without DISPLAY.
+$B --headed goto https://example.com
+
+# SOCKS5 with auth — Chromium can't prompt for SOCKS5 creds, so $B runs a
+# local 127.0.0.1 bridge that handles the auth handshake.
+$B --proxy socks5://user:pass@residential.proxy.host:1080 goto https://example.com
+
+# HTTP/HTTPS proxy passes through to Chromium directly.
+$B --proxy http://corp-proxy:3128 goto https://example.com
+
+# Browser-native download for Content-Disposition, redirect chains, anti-bot
+# CDNs where page.request.fetch() falls over.
+$B download "https://protected.example.com/file" /tmp/file.bin --navigate
+
+# Combined.
+$B --headed --proxy socks5://user:pass@host:1080 \
+   download "https://protected.example.com/file" /tmp/file.bin --navigate
+```
+
+**Credential policy.** Pass creds via the URL (`socks5://user:pass@host`) OR
+the env vars `BROWSE_PROXY_USER` / `BROWSE_PROXY_PASS` — never both. `$B`
+refuses with a clear hint when both are set; silent override created
+"works on my machine" debugging traps.
+
+**Daemon discipline.** `--proxy` and `--headed` are daemon-startup config.
+A running daemon with config A meeting a new invocation with config B exits
+1 with a `browse disconnect` hint instead of silently restarting and dropping
+tab state, cookies, or sessions.
+
+**Stealth scope.** When `--headed` or `--proxy` are set, `$B` masks
+`navigator.webdriver` only — via Chromium's
+`--disable-blink-features=AutomationControlled` plus a small init script.
+We do NOT fake `navigator.plugins`, `navigator.languages`, or `window.chrome`
+— modern fingerprinters check those for consistency, and synthesizing fixed
+values can flag MORE bot-like, not less. ChromeDriver's `cdc_` runtime
+artifacts and the Permissions API patch are still cleaned up.
+
+**Container support.** `--headed` on Linux without `DISPLAY` walks the
+display range (`:99`, `:100`, ...) until `xdpyinfo` reports a free slot,
+then spawns Xvfb. Cleanup-on-disconnect validates the recorded PID's
+`/proc/<pid>/cmdline` matches `Xvfb` AND start-time matches before sending
+any signal — no PID-reuse footguns. Skips spawn entirely when
+`WAYLAND_DISPLAY` is set (Chromium uses Wayland natively). Standard
+Debian/Ubuntu containers work out of the box; minimal images (alpine,
+distroless) may need fonts/dbus/gtk libs for headed Chromium to render.
+
+**Failure modes.** SOCKS5 upstream rejected or unreachable — fail-fast at
+startup with a redacted error after 3 retries (5s budget). Mid-stream
+upstream drop — bridge kills the affected client connection only; no
+transport retries that could corrupt browser traffic.
+
 ---
 
 ## Side Panel + sidebar agent
@@ -1117,6 +1174,11 @@ browse/
 │   ├── cli.ts                   # Thin client — reads state, sends HTTP, prints
 │   ├── server.ts                # Bun HTTP daemon — routes commands, dual-listener
 │   ├── browser-manager.ts       # Chromium lifecycle, tabs, ref map, crash detection
+│   ├── socks-bridge.ts          # Local 127.0.0.1 SOCKS5 bridge that handles auth handshakes Chromium can't speak
+│   ├── proxy-config.ts          # --proxy URL parsing + cred resolution (URL vs env, fail-fast on both)
+│   ├── proxy-redact.ts          # Cred-redaction helper for any proxy URL surfaced to logs/errors
+│   ├── xvfb.ts                  # Xvfb auto-spawn + orphan cleanup with PID + start-time validation
+│   ├── stealth.ts               # navigator.webdriver mask + cdc_ cleanup + Permissions API patch
 │   ├── browse-client.ts         # Canonical SDK — what skills import as _lib/browse-client.ts
 │   ├── snapshot.ts              # AX tree → @e/@c refs → Locator map; -D/-a/-C handling
 │   ├── read-commands.ts         # Non-mutating: text, html, links, js, css, is, dialog, ...

CHANGELOG.md

@@ -1,5 +1,121 @@
 # Changelog
 
+## [1.28.0.0] - 2026-05-07
+
+## **Browse handles real-world automation now: SOCKS5 with auth, container Xvfb, browser-native downloads. Plus a single-file `llms.txt` index agents can crawl in one read.**
+
+Five capabilities ship in one PR. Browse picks up `--proxy` (with an
+embedded SOCKS5 bridge so Chromium can speak to authenticated
+upstreams it can't speak to natively), `--headed` (auto-spawns Xvfb
+on Linux containers without DISPLAY), and `download --navigate` (uses
+the browser's native download handler for Content-Disposition,
+multi-hop CDN redirects, and anti-bot CDN chains where
+`page.request.fetch()` falls over). Stealth is narrowed to
+`navigator.webdriver` masking only — modern fingerprinters punish
+inconsistent fakes, so faking plugins/languages was making
+detection easier, not harder. And `gstack/llms.txt` is now
+auto-generated from the same source as every SKILL.md, so any agent
+that reads `llms.txt` boots into the full surface (47 skills, 75
+browse commands) in one fetch.
+
+### The numbers that matter
+
+End-to-end verified via `bun test browse/test/{socks-bridge,proxy-config,proxy-redact,xvfb,stealth-webdriver,bridge-chromium-e2e}.test.ts test/llms-txt-shape.test.ts`:
+
+| Surface | Before | After | Δ |
+|---|---|---|---|
+| `browse --proxy` (SOCKS5 with auth) | not supported | works end-to-end | new capability |
+| `browse --headed` on Linux without DISPLAY | not supported | auto-Xvfb on first free display | new capability |
+| `download --navigate` (browser-native) | only `page.request.fetch()` | added native download path | new capability |
+| `gstack/llms.txt` index for agents | none | 47 skills + 75 commands in 11KB | new capability |
+| Bridge PID validation defenses | n/a | both `/proc/<pid>/cmdline` AND start-time | full safety |
+| Tests covering proxy + headed + navigate | 0 | 70+ tests across 7 files | from zero to comprehensive |
+
+The `bridge-chromium-e2e.test.ts` is the one that proves the feature
+actually works: real Chromium launches with `proxy.server =
+socks5://127.0.0.1:<bridgePort>`, navigates to a local HTTP fixture,
+and we assert the auth upstream's connect counter and the HTTP
+fixture's hit counter both increment. Without that test we could
+ship a working byte-relay and a broken Chromium integration and never
+notice.
+
+### What this means for AI agents
+
+Any agent on any project can now hit any site. DDoS-Guard'd CDN
+behind an auth-required residential SOCKS5 → `browse --proxy
+socks5://user:pass@host:1080 --headed download <url> /tmp/file
+--navigate` and the file lands. Linux container without DISPLAY →
+`--headed` auto-spawns Xvfb, no manual setup. The `llms.txt` index
+makes discovery a one-fetch operation: agents stop scanning 47
+SKILL.md files and start with the right skill on the first try.
+
+### Itemized changes
+
+#### Added
+- `browse --proxy <url>` flag. Supports SOCKS5 with username/password
+  auth, HTTP, and HTTPS. SOCKS5+auth runs through an embedded local
+  bridge (`browse/src/socks-bridge.ts`, ~250 LOC) bound to 127.0.0.1
+  on an ephemeral port. The bridge handles the SOCKS5 auth handshake
+  so Chromium (which can't prompt for SOCKS5 creds) can still use
+  authenticated upstreams.
+- Pre-flight `testUpstream()` runs before Chromium launches: 5s total
+  budget, 3 retries with 500ms backoff (handles VPN warm-up race).
+  On failure, exits 1 with a redacted error message — no confusing
+  "connection refused" on first navigation.
+- `browse --headed` flag with auto-Xvfb on Linux. Walks the display
+  range (`:99`, `:100`, ...) until `xdpyinfo` says free; never
+  hardcodes `:99` and never unlinks `/tmp/.X<n>-lock` for displays
+  it didn't create. Xvfb child PID + start-time + display recorded
+  in `~/.gstack/browse.json` so cleanup-on-disconnect can validate
+  ownership before signaling. Skips spawn when `WAYLAND_DISPLAY` is
+  set (Chromium uses Wayland natively).
+- `download --navigate` flag (community PR #1355, attribution preserved).
+  Uses `page.waitForEvent('download')` and `page.goto(url, {
+  waitUntil: 'commit' })` instead of `page.request.fetch()`.
+  Required for sites where the download is triggered by browser
+  navigation (Content-Disposition headers, redirect chains, anti-bot
+  CDNs).
+- `gstack/llms.txt` auto-generated from skill frontmatter and the
+  browse `COMMAND_DESCRIPTIONS` registry. Regenerates on every
+  `bun run gen:skill-docs`. Strict mode (used in tests) refuses any
+  skill missing `name` or `description` in its frontmatter.
+
+#### Changed
+- Stealth narrowed to `navigator.webdriver` masking only. The
+  pre-existing `launchHeaded` patches that faked `navigator.plugins`
+  and `navigator.languages` were removed because modern
+  fingerprinters check those for consistency with `userAgent`/
+  `platform`, and synthesized fixed values can flag MORE bot-like,
+  not less. The cdc_/__webdriver runtime cleanup and Permissions API
+  patch are kept — those remove ChromeDriver-injected artifacts
+  rather than synthesize natural-browser values.
+- Browse daemon refuses to silently restart on `--proxy`/`--headed`
+  flag mismatch. Existing daemon with config A + new invocation with
+  config B → exits 1 with a `browse disconnect` hint. No silent
+  state loss.
+- Cred policy: passing creds in BOTH the URL and `BROWSE_PROXY_USER`/
+  `BROWSE_PROXY_PASS` env vars now fails fast with a clear error.
+  Silent override was a debugging trap.
+
+#### Fixed
+- N/A — all-new code paths.
+
+#### For contributors
+- New module boundary: `browse/src/socks-bridge.ts`,
+  `browse/src/proxy-config.ts`, `browse/src/proxy-redact.ts`,
+  `browse/src/xvfb.ts`, `browse/src/stealth.ts`. Each is small,
+  testable in isolation, and has matching `*.test.ts` coverage.
+- 70+ new tests across 7 files. The `bridge-chromium-e2e.test.ts`
+  test launches real Chromium through the bridge and asserts the
+  request actually traversed it (upstream connect counter + HTTP
+  fixture hit counter both increment).
+- `socks` npm dependency added (~30KB).
+- Xvfb + x11-utils added to `.github/docker/Dockerfile.ci` so
+  `headed-xvfb`/`headed-orphan-cleanup` exercise the Linux container
+  path on every CI run instead of only manual smoke tests.
+- Community PR #1355 from @garrytan-agents merged; attribution
+  preserved on the merging commit.
+
 ## [1.27.1.0] - 2026-05-06
 
 ## **Plan-mode reviews now refuse to dump findings without asking. Four gate-tier tests catch the regression on every PR.**

SKILL.md

@@ -862,7 +862,7 @@ Refs are invalidated on navigation — run `snapshot` again after `goto`.
 | Command | Description |
 |---------|-------------|
 | `archive [path]` | Save complete page as MHTML via CDP |
-| `download <url|@ref> [path] [--base64]` | Download URL or media element to disk using browser cookies |
+| `download <url|@ref> [path] [--base64] [--navigate]` | Download URL or media element to disk using browser cookies. Use --navigate for URLs that trigger browser downloads (CDN redirects, Content-Disposition, anti-bot protected sites) |
 | `scrape <images|videos|media> [--selector sel] [--dir path] [--limit N]` | Bulk download all media from page. Writes manifest.json |
 
 ### Interaction

TODOS.md

@@ -1562,7 +1562,7 @@ Shipped in v0.6.5. TemplateContext in gen-skill-docs.ts bakes skill name into pr
 
 **What:** Write a postinstall script that patches Playwright's CDP layer to suppress `Runtime.enable` and use `addBinding` for context ID discovery, same approach as rebrowser-patches. Eliminates the `navigator.webdriver`, `cdc_` markers, and other CDP artifacts that sites like Google use to detect automation.
 
-**Why:** Our current stealth patches (UA override, navigator.webdriver=false, fake plugins) work on most sites but Google still triggers captchas. The real detection is at the CDP protocol level. rebrowser-patches proved the approach works but their patches target Playwright 1.52.0 and don't apply to our 1.58.2. We need our own patcher using string matching instead of line-number diffs. 6 files, ~200 lines of patches total.
+**Why:** Our current stealth narrows to `navigator.webdriver` masking + ChromeDriver `cdc_` runtime cleanup + Permissions API patch (v1.28.0.0 narrowed it from also faking plugins/languages, since modern fingerprinters punish inconsistent fakes more than they punish admitted defaults). That's enough for most sites but Google still triggers captchas, because the real detection is at the CDP protocol level. rebrowser-patches proved the approach works but their patches target Playwright 1.52.0 and don't apply to our 1.58.2. We need our own patcher using string matching instead of line-number diffs. 6 files, ~200 lines of patches total.
 
 **Context:** Full analysis of rebrowser-patches source: patches 6 files in `playwright-core/lib/server/` (crConnection.js, crDevTools.js, crPage.js, crServiceWorker.js, frames.js, page.js). Key technique: suppress `Runtime.enable` (the main CDP detection vector), use `Runtime.addBinding` + `CustomEvent` trick to discover execution context IDs without it. Our extension communicates via Chrome extension APIs, not CDP Runtime, so it should be unaffected. Write E2E tests that verify: (1) extension still loads and connects, (2) Google.com loads without captcha, (3) sidebar chat still works.
 

VERSION

@@ -1 +1 @@
-1.27.1.0
+1.28.0.0