Stop suggesting redundant Playwright CLI mode (#58041)

* Review Pi Playwright CLI guidance

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

* Stop suggesting redundant Playwright CLI mode

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

* Fix remaining Playwright CLI guidance and Pi regression coverage

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

* Align Playwright compatibility guidance and trim scope

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

* Fix fallback AW file list sync for compat docs

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: Peli de Halleux <pelikhan@users.noreply.github.com>

Author: 3 people

.changeset/patch-playwright-cli-only-guidance.md

@@ -30,9 +30,12 @@ tools:
     version: "0.1.18"
 ```
 
-`mode: cli` is accepted but unnecessary. `mode: mcp` is not supported by the
-built-in tool. If MCP is required, configure and pin `@playwright/mcp` explicitly
-under `mcp-servers` and allow only the required tools.
+Omit `mode`; the built-in Playwright integration is CLI-only by default. The
+explicit `mode: cli` setting remains accepted for compatibility, but it is not
+needed and should be removed from workflows that still carry it. `mode: mcp` is
+not supported by the built-in tool. If MCP is required, configure and pin
+`@playwright/mcp` explicitly under `mcp-servers` and allow only the required
+tools.
 
 ## Configure network access
 

.github/aw/playwright.md

@@ -125,7 +125,6 @@ The `tools:` field configures which tools the coding agent may use.
   ```yaml
   tools:
     playwright:
-      mode: cli          # recommended: token-efficient CLI mode
       version: "0.1.11"  # optional: @playwright/cli npm package version
   ```
 - `timeout:` - Per-operation timeout in seconds for all tool and MCP calls (integer or expression, default: 60 s for all engines).

.github/aw/syntax-tools-imports.md

@@ -77,7 +77,7 @@ This will:
 
 ## Playwright Browser Access
 
-With **CLI mode** (`mode: cli`, recommended), `playwright-cli` runs directly on the runner — not in a Docker container. Use `localhost` directly to reach the dev server:
+With the built-in Playwright CLI integration, `playwright-cli` runs directly on the runner — not in a Docker container. Use `localhost` directly to reach the dev server:
 
 ```bash
 playwright-cli browser_navigate --url "http://localhost:4321/gh-aw/"
@@ -111,7 +111,7 @@ This will:
 ## Usage Notes
 
 - The server runs on `http://localhost:4321` and is accessible at `http://localhost:4321/gh-aw/` for curl/bash and playwright-cli
-- With CLI mode (`mode: cli`), use `localhost` directly for all playwright-cli commands — no bridge IP needed
+- With the built-in Playwright CLI integration, use `localhost` directly for all playwright-cli commands — no bridge IP needed
 - Always clean up the server when done to avoid orphan processes
 - If the server fails to start, check `/tmp/gh-aw/agent/preview.log` for errors
 - Node.js >= 22 is required; ensure `runtimes: node: version: "22"` is set in the workflow frontmatter

.github/workflows/shared/docs-server-lifecycle.md

@@ -90,7 +90,7 @@ Analyze the repository and create a concise daily status report covering:
 
 ## Capabilities and limitations
 
-Pi supports top-level `max-turns`, provider-prefixed models, and `engine.extensions`. Pi already runs in bare mode, so `engine.bare: true` is accepted but has no effect. Pi does not provide native MCP server integration, native `tools.web-search`, per-command bash allowlisting, `max-continuations`, native `engine.agent` selection, or custom `engine.harness` scripts. MCP-backed tools must be exposed through the required CLI proxy.
+Pi supports top-level `max-turns`, provider-prefixed models, and `engine.extensions`. Pi already runs in bare mode, so `engine.bare: true` is accepted but has no effect. The built-in `tools.playwright` integration works through `playwright-cli`; omit its `mode` field because CLI is the only built-in mode. Pi does not provide native MCP server integration, native `tools.web-search`, per-command bash allowlisting, `max-continuations`, native `engine.agent` selection, or custom `engine.harness` scripts. MCP-backed tools must be exposed through the required CLI proxy.
 
 See the [AI engine feature comparison](/gh-aw/reference/engines/#engine-feature-comparison) and [Pi extensions reference](/gh-aw/reference/engines/#pi-extensions-extensions).
 

docs/src/content/docs/engines/pi.md

@@ -9,12 +9,11 @@ Playwright enables headless browser control for accessibility testing, visual re
 
 ## Configuration
 
-The built-in Playwright tool uses CLI mode. It is token-efficient because it does not load MCP tool schemas into the agent context, avoids Docker overhead, and reaches local development servers through `localhost`.
+The built-in Playwright tool is CLI-only by default. It is token-efficient because it does not load MCP tool schemas into the agent context, avoids Docker overhead, and reaches local development servers through `localhost`. If an older workflow still sets `mode: cli`, it continues to work for compatibility, but omitting `mode` is preferred.
 
 ```yaml wrap
 tools:
   playwright:
-    mode: cli
 ```
 
 The compiler installs `@playwright/cli` as a global npm package, its skills, and
@@ -52,7 +51,6 @@ The `version` field pins the `@playwright/cli` npm package. Omit it to use the c
 ```yaml wrap
 tools:
   playwright:
-    mode: cli
     version: "0.1.18"
 ```
 
@@ -120,7 +118,8 @@ Capture `/tmp/home.png`, then call `upload_artifact` with
 
 ## Migrate from Playwright MCP
 
-Remove `mode: mcp` or replace it with `mode: cli`. The compiler now reports `mode: mcp` as an error.
+Remove `mode: mcp`. The built-in integration is CLI-only, so no replacement
+`mode` field is needed. The compiler now reports `mode: mcp` as an error.
 
 Replace MCP tool calls in prompts with equivalent `playwright-cli` commands run through bash:
 
@@ -173,7 +172,6 @@ on:
 
 tools:
   playwright:
-    mode: cli
 
 network:
   allowed:
@@ -238,7 +236,6 @@ steps:
 
 tools:
   playwright:
-    mode: cli
     version: "0.1.18"  # pins `@playwright/cli` npm package; see Configuration > Version
   bash:
     - "npm *"
@@ -289,7 +286,6 @@ on:
 
 tools:
   playwright:
-    mode: cli
   bash: [":*"]
 
 network:

docs/src/content/docs/reference/playwright.md

@@ -22,7 +22,6 @@ package workflow
 //
 //	tools:
 //	  playwright:
-//	    mode: cli
 
 import (
 	"strings"

pkg/workflow/playwright_cli.go

@@ -7,13 +7,12 @@
 //
 // # Migration
 //
-// To migrate from MCP mode to CLI mode:
+// To migrate from MCP mode to the built-in CLI integration:
 //
-//  1. Add `mode: cli` to your playwright tool configuration:
+//  1. Remove `mode: mcp` from your playwright tool configuration:
 //
 //     tools:
 //       playwright:
-//         mode: cli
 //
 //  2. Update prompts to use `playwright-cli <command>` via bash instead of
 //     MCP browser tool calls. For example:
@@ -60,15 +59,15 @@ func (c *Compiler) validatePlaywrightMode(workflowData *WorkflowData) error {
 				"tools.playwright.mode",
 				mode,
 				"mode must be a literal value; expressions are not allowed",
-				"Set mode to cli, or omit mode because CLI is the default",
+				"Remove mode because the built-in Playwright integration is CLI-only",
 			)
 		}
 		if mode, ok := config["mode"].(string); ok && strings.EqualFold(mode, "mcp") {
 			return NewValidationError(
 				"tools.playwright.mode",
 				mode,
 				"built-in Playwright MCP support has been removed",
-				"Remove `mode: mcp` or change it to `mode: cli`, then update prompts to run `playwright-cli <command>` from bash. If MCP is still required, configure Playwright explicitly under `mcp-servers`. See https://github.com/github/gh-aw/blob/main/docs/src/content/docs/reference/playwright.md",
+				"Remove `mode: mcp`, then update prompts to run `playwright-cli <command>` from bash. If MCP is still required, configure Playwright explicitly under `mcp-servers`. See https://github.com/github/gh-aw/blob/main/docs/src/content/docs/reference/playwright.md",
 			)
 		}
 		if browsers, ok := config["browsers"].([]any); ok {

pkg/workflow/playwright_validation.go

@@ -97,10 +97,48 @@ tools:
 
 	require.Error(t, err)
 	assert.Contains(t, err.Error(), "built-in Playwright MCP support has been removed")
+	assert.Contains(t, err.Error(), "Remove `mode: mcp`")
+	assert.NotContains(t, err.Error(), "mode: cli")
 	assert.Contains(t, err.Error(), "playwright-cli <command>")
 	assert.Contains(t, err.Error(), "mcp-servers")
 }
 
+func TestPiEngineAcceptsPlaywrightWithImplicitCLIMode(t *testing.T) {
+	tmpDir := t.TempDir()
+	mdPath := filepath.Join(tmpDir, "test-workflow.md")
+	content := `---
+name: pi-playwright-cli
+on: push
+engine: pi
+permissions:
+  contents: read
+  issues: read
+
+tools:
+  github:
+    mode: gh-proxy
+  cli-proxy: true
+  playwright:
+---
+
+# Test Workflow
+`
+	require.NoError(t, os.WriteFile(mdPath, []byte(content), 0o644))
+
+	compiler := NewCompiler()
+	require.NoError(t, compiler.CompileWorkflow(mdPath))
+
+	lockPath := filepath.Join(tmpDir, "test-workflow.lock.yml")
+	lockContent, err := os.ReadFile(lockPath)
+	require.NoError(t, err)
+	lockStr := string(lockContent)
+
+	assert.Contains(t, lockStr, "@playwright/cli")
+	assert.Contains(t, lockStr, "playwright-cli install --skills")
+	assert.NotContains(t, lockStr, "@playwright/mcp")
+	assert.NotContains(t, lockStr, "mode: cli")
+}
+
 // TestCompileWorkflowRejectsLegacyPlaywrightMCPModeWithArgs ensures that a legacy
 // configuration combining `mode: mcp` with the removed MCP-only `args` field still
 // surfaces the actionable migration error instead of a generic JSON schema