Skip to content

fix(ce-test-browser): prefer host-native browser drivers#1097

Open
tmchow wants to merge 2 commits into
mainfrom
tmchow/fix-ce-test-browser-native-driver
Open

fix(ce-test-browser): prefer host-native browser drivers#1097
tmchow wants to merge 2 commits into
mainfrom
tmchow/fix-ce-test-browser-native-driver

Conversation

@tmchow

@tmchow tmchow commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

ce-test-browser can now use a browser embedded in or directly owned by the active app harness instead of failing because every run was forced through agent-browser. CLI harnesses and environments without a qualifying native browser retain agent-browser as the portable fallback.

Pipeline and LFG runs are still unattended, but unattended no longer means invisible: an integrated browser can remain visible and non-blocking while the run suppresses questions. The original quality guard remains intact -- standalone Playwright, Puppeteer, separately configured browser extensions or MCPs, and ad hoc automation are still rejected.

Driver policy

  • A native browser qualifies only when it supports local navigation, rendered and interactive state inspection, click/fill/press actions, screenshots, and console-error inspection.
  • One driver owns the run's session, element state, screenshots, and authentication. Fallback is allowed only during initialization, before the first route is tested.
  • The CLI fallback loads version-matched agent-browser guidance and distinguishes a missing CLI from a browser-runtime launch failure.

Validation

  • bun test -- 1,913 pass, 0 fail
  • bun test tests/ce-test-browser-driver-policy.test.ts tests/skill-conventions.test.ts -- 206 pass, 0 fail
  • bun run release:validate -- release metadata in sync
  • Compounded-learning frontmatter and source-claim validators -- pass
  • Installed Codex in-app browser contract checked directly for navigation, DOM/state inspection, interaction, screenshots, visibility control, and console logs
  • Static routing scenarios covered app pipeline mode, CLI fallback, and an embedded API named Playwright; no live Cursor or CLI-only end-to-end run was performed

New concepts

Host-native capability with a portable fallback

A host-native capability is part of the harness running the agent, not merely another browser tool installed beside it. Preferring that surface lets the harness provide its intended observable experience without giving agents permission to improvise a new automation stack.

Layer Role Boundary
Harness-native Best integrated experience Must be embedded in or directly owned by the harness and satisfy the full test contract
Portable fallback Consistent coverage in CLI environments Use agent-browser when no qualifying native surface exists
Rejected substitute Uncontrolled third stack Do not install or switch to standalone Playwright, Puppeteer, extensions, or arbitrary MCP browsers

An API named Playwright can still be host-native when it is the harness's own browser API. Do not use this pattern when the native surface lacks a required capability; select the portable fallback before testing begins instead.


Compound Engineering
Codex

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f37fdbec63

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread skills/ce-test-browser/references/agent-browser-driver.md Outdated
- Make fallback setup guidance platform-neutral
- Pin semantic ce-setup skill wording in the policy test

Note: concurrent full-suite runs in another worktree contended on the shared repo-profile cache; the focused policy test passed 5/5.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant