feat: Add browser view (passive and interactive modes) - #41
Merged
Conversation
…plus a Playwright sample Actor Add `sample_actor_playwright`, a PlaywrightCrawler Actor built from Apify's ts-crawlee-playwright-chrome template that always runs Chrome headful under the base image's Xvfb, and a per-Actor "browser view" toggle (`POST /actor-runtime/browser-view/:actorId`, plus a console form) that gives every run of the Actor a live mirror of the X display its browser draws on, served by the console at `/runs/:runId/browser` (noVNC, bridged over a websocket to the run's sidecar). The mirror is a runtime-owned x11vnc sidecar container, imported from a rootfs tar baked into the runtime image (no pull), sharing only a tmpfs volume at /tmp/.X11-unix with the Actor's container. The Actor's container is otherwise byte-identical to an ordinary run's (same image, command, env, network, no published port), nothing is injected into the browser, and the sidecar only reads the framebuffer - so whether the mirror is on, off, or watched is not observable from inside the browser or by the sites it visits. The runtime never changes headless/headful mode; the sample is headful always so watching is never a behavioural difference. - storage: `localBrowserView` on the Actor (toggle) and run (interactive, vncHost, vncPort) records, never on /v2 - driver: startBrowserViewer/stopBrowserViewer, X-socket volume mount on the run container, orphan sweep of leftover sidecars and volumes - console: toggle form, run-detail link, viewer page, static noVNC (`@novnc/novnc`), websocket bridge with retrying TCP dial - tests: unit (validation, driver), integration (endpoint, form, run lifecycle, viewer page, bridge against a fake RFB server), e2e (`test/e2e/browser-view.test.ts`, requires Docker) - docs: README, CLAUDE.MD, requirements (actor-driver, api, console, storage, system, test) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D5pqNoJ2DDRCNXzQKrNZaG
…viewer opened before the sidecar is up Verified end to end against a real Docker daemon with the Playwright sample Actor: the sidecar saw the Actor's Xvfb socket and x11vnc connected to the display, but then died on X_ShmAttach (BadAccess) - its default MIT-SHM framebuffer grab needs a shared-memory segment the X server can attach, impossible across the two containers' IPC namespaces. With `-noshm` the mirror works: the console viewer connects, shows the Actor's Chrome live at the display's 1920x1080, survives a migration/reboot of the run's container, and the sidecar and volume are removed when the run ends. Also found in that test: a viewer opened straight from the run id (before the sidecar registered its address on the run record, a second or two after `apify call`) was told browser view is "not on" and the page 404ed. The bridge now waits for the address on a live run of a toggled Actor (within its dial budget), and the page renders the client for that starting state. - docker/browser-viewer.sh: x11vnc -noshm - console/browser-view-ws.ts, console/server.ts: "starting" handling shared by the page and the bridge - tests: pending-mirror and untoggled-run cases; ended-run wording - requirements: console.md starting state, actor-driver.md -noshm rationale Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D5pqNoJ2DDRCNXzQKrNZaG
…y; state the page's guarantees precisely x11vnc's default selection sync would make the browser answer a ConvertSelection request whenever it owns the X selection, and push a viewer's clipboard into the display in interactive mode. `-nosel` closes both directions (`-nobell` drops the XBell watch too), so the mirror asks the browser nothing in either mode. The viewer page now says exactly what crosses: view-only - nothing (no input, no clipboard); interactive - mouse and keyboard input only. Verified against a real daemon in both modes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D5pqNoJ2DDRCNXzQKrNZaG
…view e2e; document the headless cause of a black mirror `sample_actor_playwright_py` is the Crawlee for Python twin of `sample_actor_playwright`, built from Apify's `python-crawlee-playwright` template with one change: `headless=False`. The template hard-codes `headless=True`, and a headless browser draws nothing on the display - which is why browser view on an unmodified template Actor shows a black mirror (the runtime never changes headless mode by design). The `apify/actor-python-playwright` base image runs the Actor under Xvfb exactly like the Node image, so nothing runtime-side differs between the languages; verified end to end against a real daemon (mirror connected, page content visible, 25/25 items, sidecar and volume cleaned up). - e2e: browser-view case now runs for both samples; helper pulls both Playwright base images - docs: README/CLAUDE.MD/requirements name the headless default as the first thing to check, and the Python sample alongside the TypeScript one Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D5pqNoJ2DDRCNXzQKrNZaG
Reproduced against a real daemon: an unmodified Apify Python Crawlee Playwright template Actor (`headless=True`) under browser view connects fine and shows a black display - the headless browser draws nothing on the Xvfb it is given. The viewer's connected status now says exactly that and how to fix it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D5pqNoJ2DDRCNXzQKrNZaG
Comments only state what the code does not make obvious. The README section is a short how-to. The requirements say what the capability is (toggle, two modes, not observable by the browser, headful needed, lifetime), and one short entry each for the endpoint, console form, viewer page, and storage fields, with implementation detail removed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D5pqNoJ2DDRCNXzQKrNZaG
…e-push failure
The e2e files each start a runtime container on the fixed host ports, so they cannot share a daemon; CI now
runs them as a matrix of five jobs on separate runners instead of one 12-minute serial job. The browser-view
e2e is split per sample (`browser-view-ts`, `browser-view-py`, sharing `helpers/browser-view-suite.ts`) so each
job pulls only its own Playwright base image.
Also fixes the one CI failure: the "toggle cleared" case re-pushed the unchanged sample, which the CLI refuses
("already exists ... newer changes than your local copy"); it now reuses the Actor id from the first case.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D5pqNoJ2DDRCNXzQKrNZaG
…twice The previous commit's message described this fix but the edit had not applied. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D5pqNoJ2DDRCNXzQKrNZaG
…sh budget On the previous head the Python run once exceeded the test's 4-minute wait: a run at the default 1024 MB gets 0.25 core, on which headful Chromium crawling crawlee.dev is slow and variable. The e2e runs now start with 4096 MB (one core) and a 600 s run timeout, and the test waits up to 8 minutes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D5pqNoJ2DDRCNXzQKrNZaG
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Actors can be started in a special mode that lets you observe them in the Docker browser.
This is for developers to be able to see what the browser operating actors are actually doing and how does thge page looks like.
There are two modes:
Browser view can be specifically useful in combination with debug mode that allows you to stop the Actor at a specific point in time through breakpoints!