Refactor/replace extension by cdp

AGENTS.md

@@ -25,7 +25,7 @@ ClawBench/
       runtime/
         shared/
           alex_green_personal_info.json
-        extension-server/         # Container-only uv project for server deps
+        runtime-server/           # Container-only uv project for server deps
         chrome-extension/         # Recording extension
         harnesses/                # Dockerfiles + setup/run scripts
   models/
@@ -139,7 +139,7 @@ test-output/<model>/<harness>-<case>-<model>-<timestamp>/
 ## Key Documentation
 
 - [README.md#-cli](README.md#-cli) -- CLI usage, batch runner flags, output format
-- [src/clawbench/runtime/extension-server/README.md](src/clawbench/runtime/extension-server/README.md) -- FastAPI server, endpoints, screen recording
+- [src/clawbench/runtime/runtime-server/README.md](src/clawbench/runtime/runtime-server/README.md) -- FastAPI server, endpoints, screen recording
 - [CONTRIBUTING.md](CONTRIBUTING.md) -- how to add new test cases
 - [eval/README.md](eval/README.md) -- evaluation guide and Claude Code prompt template
 - [eval/agentic_eval.md](eval/agentic_eval.md) -- evaluator rubric for PASS/FAIL judgment

CHANGELOG.md

@@ -6,6 +6,10 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/)
 and this project adheres to [Semantic Versioning](https://semver.org/).
 
+## [Unreleased]
+### Changed
+- Refactored the container runtime to move the action recording and screenshot capturing logic into the CDP server rather than the Chrome extension, making it possible to integrate remote browsers in the future.
+
 ## [0.6.0] - 2026-06-04
 ### Added
 - Added support for the [Harbor](https://github.com/harbor-framework/harbor) framework.

CONTRIBUTING.md

@@ -108,11 +108,11 @@ If the task requires additional context (e.g., a pre-filled profile, a specific
 
 ## Code changes
 
-For changes to the framework itself (test driver, extension server, Chrome extension, container):
+For changes to the framework itself (test driver, runtime server, Chrome extension, container):
 
 1. Read the relevant sub-README for component-specific documentation:
    - [README.md#-cli](README.md#-cli)
-   - [src/clawbench/runtime/extension-server/README.md](src/clawbench/runtime/extension-server/README.md)
+   - [src/clawbench/runtime/runtime-server/README.md](src/clawbench/runtime/runtime-server/README.md)
    - [src/clawbench/runtime/chrome-extension/README.md](src/clawbench/runtime/chrome-extension/README.md)
 2. Open an issue first for anything beyond a small bug fix so we can align on approach before you spend time.
 3. Open a PR with a clear description of the change and how you tested it.

README.md

@@ -583,23 +583,20 @@ ClawBench's niche: **live consumer websites, everyday tasks, end-to-end recordin
 ┌─────────────────────────────────────────────────┐
 │  Container (Docker / Podman)                    │
 │                                                 │
-│  ┌───────────┐   DOM events  ┌──────────────┐   │
-│  │ content.js├──────────────►│ background.js│   │
-│  │ (per tab) │               │  (service    │   │
-│  └───────────┘               │   worker)    │   │
-│                              └──┬──────┬────┘   │
-│                                 │      │        │
-│                         actions │      │ screenshots
-│                                 │      │        │
-│  ┌──────────┐            ┌──────▼──────▼────┐   │
+│  ┌──────────┐  CDP Fetch/Runtime/Page events    │
+│  │ Chromium ├─────────────────────────────┐     │
+│  │ :9222 CDP│                             │     │
+│  └──────────┘                             │     │
+│                                           │     │
+│  ┌──────────┐            ┌────────────────▼─┐   │
 │  │  Xvfb    │◄──ffmpeg──►│  FastAPI Server  │   │
 │  │ :99      │  x11grab   │  :7878           │   │
 │  └──────────┘            └──────────────────┘   │
 │                                  │              │
-│  ┌──────────┐            ┌───────▼─────────┐    │
-│  │ Chromium │            │     /data       │    │
-│  │ :9222 CDP│            │  actions.jsonl  │    │
-│  └──────────┘            │  requests.jsonl │    │
+│                          ┌───────▼─────────┐    │
+│                          │     /data       │    │
+│                          │  actions.jsonl  │    │
+│                          │  requests.jsonl │    │
 │                          │  screenshots/   │    │
 │                          │  recording.mp4  │    │
 │                          └─────────────────┘    │

docs/README.zh-CN.md

@@ -484,25 +484,22 @@ ClawBench 定位:**真实消费级网站、日常任务、端到端录制**。
 
 ```
 ┌─────────────────────────────────────────────────┐
-│  容器 (Docker / Podman)                          │
+│  容器 (Docker / Podman)                         │
 │                                                 │
-│  ┌───────────┐   DOM 事件    ┌──────────────┐   │
-│  │ content.js├──────────────►│ background.js│   │
-│  │ (每个标签)│               │  (service    │   │
-│  └───────────┘               │   worker)    │   │
-│                              └──┬──────┬────┘   │
-│                                 │      │        │
-│                            动作 │      │ 截图   │
-│                                 │      │        │
-│  ┌──────────┐            ┌──────▼──────▼────┐   │
+│  ┌──────────┐  CDP Fetch/Runtime/Page 事件      │
+│  │ Chromium ├─────────────────────────────┐     │
+│  │ :9222 CDP│                             │     │
+│  └──────────┘                             │     │
+│                                           │     │
+│  ┌──────────┐            ┌────────────────▼─┐   │
 │  │  Xvfb    │◄──ffmpeg──►│  FastAPI Server  │   │
 │  │ :99      │  x11grab   │  :7878           │   │
 │  └──────────┘            └──────────────────┘   │
 │                                  │              │
-│  ┌──────────┐            ┌───────▼─────────┐    │
-│  │ Chromium │            │     /data       │    │
-│  │ :9222 CDP│            │  actions.jsonl  │    │
-│  └──────────┘            │  requests.jsonl │    │
+│                          ┌───────▼─────────┐    │
+│                          │     /data       │    │
+│                          │  actions.jsonl  │    │
+│                          │  requests.jsonl │    │
 │                          │  screenshots/   │    │
 │                          │  recording.mp4  │    │
 │                          └─────────────────┘    │

src/README.md

@@ -45,15 +45,13 @@ src/
     utils/
       paths.py                   # Shared PROJECT_ROOT / HARNESS_ROOT discovery
       hf_upload.py               # Optional HuggingFace upload helpers
-  extension-server/
-    server.py                    # FastAPI server for actions, screenshots, recording, request interception
+  runtime-server/
+    server.py                    # CDP instrumentation server for actions, screenshots, requests, recording
     pyproject.toml               # Container-only uv project
     uv.lock
     README.md
   chrome-extension/
     manifest.json
-    content.js                   # DOM action capture
-    background.js                # Screenshot capture + server relay
     stealth.js                   # Browser fingerprint hardening patches
     setup.sh                     # Local extension launch helper
     README.md

src/clawbench/runtime/chrome-extension/README.md

@@ -1,30 +1,27 @@
 # ClawBench Chrome Extension
 
-This is the source code for the ClawBench Chrome Extension, which acts as the client for the ClawBench benchmarking framework.
+This is the source code for the ClawBench Chrome Extension, which provides browser fingerprint hardening for benchmark runs.
 
 The extension is responsible for the following tasks:
 
-- Collecting every browser action performed by the user or agent and sending it to the ClawBench server.
-- Taking screenshots after browser actions, with high-frequency events throttled.
-- Keeping the active tab aligned with the agent's work so server-side screen recording and screenshots capture the right browser state.
+- Injecting `stealth.js` at `document_start` in the page's `MAIN` world.
+- Applying anti-bot-detection patches for webdriver, plugins, WebGL, permissions, and related browser fingerprints.
 
-The extension should auto start when any non-built-in page is loaded, and should stop when the browser is closed. No UI or configuration is needed for the extension, as all configuration is done on the server side.
+Action capture, screenshot capture, request logging, request interception, and tab activation are handled by the runtime server over CDP. The extension does not send data to the server.
 
 A `setup.sh` script is provided to load the extension into Chrome. Linux and macOS are supported.
 
 ## Files
 
 | File | Description |
 |------|-------------|
-| `manifest.json` | Manifest V3 extension definition. Permissions: `activeTab`, `tabs`. Content scripts injected on all URLs. |
+| `manifest.json` | Manifest V3 extension definition. Loads `stealth.js` on all URLs. |
 | `stealth.js` | Anti-bot-detection patches. Runs at `document_start` in `MAIN` world. Overrides `navigator.webdriver`, plugins, WebGL, permissions, etc. |
-| `content.js` | Injected into every non-chrome:// page. Listens for DOM events, extracts metadata, sends to background. Runs at `document_idle` in `ISOLATED` world. |
-| `background.js` | Service worker. Relays actions to server via HTTP POST. Captures screenshots with `chrome.tabs.captureVisibleTab`. |
 | `setup.sh` | Detects Chrome/Chromium binary on macOS or Linux and launches with `--load-extension` and remote debugging enabled. |
 
 ## Event Capture
 
-### Captured Events
+Browser event capture now lives in `src/clawbench/runtime/runtime-server/server.py`. The server injects capture listeners through CDP and preserves the historical action schema:
 
 `click`, `keydown`, `keyup`, `input`, `scroll`, `change`, `submit`, plus a synthetic `pageLoad` on each navigation.
 
@@ -123,5 +120,3 @@ Run Chrome with the extension loaded:
 ```bash
 ./setup.sh https://example.com
 ```
-
-The server must be running on `http://localhost:7878` for the extension to send data.

src/clawbench/runtime/chrome-extension/manifest.json

@@ -2,24 +2,15 @@
   "manifest_version": 3,
   "name": "ClawBench",
   "version": "1.0",
-  "description": "Browser action recording for benchmarking",
-  "permissions": ["activeTab", "tabs"],
+  "description": "Anti-bot-detection browser configuration for benchmarking",
+  "permissions": [],
   "host_permissions": ["<all_urls>"],
-  "background": {
-    "service_worker": "background.js"
-  },
   "content_scripts": [
     {
       "matches": ["<all_urls>"],
       "js": ["stealth.js"],
       "run_at": "document_start",
       "world": "MAIN"
-    },
-    {
-      "matches": ["<all_urls>"],
-      "js": ["content.js"],
-      "run_at": "document_idle",
-      "all_frames": true
     }
   ]
 }

src/clawbench/runtime/harnesses/base/Dockerfile.base

@@ -23,12 +23,12 @@ RUN git clone --depth 1 --branch v1.6.0 https://github.com/novnc/noVNC.git /opt/
 COPY --from=ghcr.io/astral-sh/uv:0.11.6 /uv /usr/local/bin/uv
 
 WORKDIR /app
-COPY extension-server/pyproject.toml extension-server/uv.lock ./src/extension-server/
-WORKDIR /app/src/extension-server
+COPY runtime-server/pyproject.toml runtime-server/uv.lock ./src/runtime-server/
+WORKDIR /app/src/runtime-server
 RUN UV_PYTHON_PREFERENCE=only-system uv sync --frozen
 
 WORKDIR /app
-COPY extension-server/server.py ./src/extension-server/server.py
+COPY runtime-server/server.py ./src/runtime-server/server.py
 
 COPY chrome-extension/ ./src/chrome-extension/
 

src/clawbench/runtime/harnesses/base/entrypoint.sh

@@ -10,7 +10,7 @@ export DISPLAY=:99
 sleep 1
 
 # Start the server
-cd /app/src/extension-server
+cd /app/src/runtime-server
 uv run --no-sync uvicorn server:app --host 0.0.0.0 --port 7878 &
 sleep 1