Skip to content

Commit a659914

Browse files
committed
docs(docker): update self-hosting & migration guides for 0.9.x
The self-hosting guide predated the 0.9.0 secure-by-default release and its quickstarts produced an unreachable server. Verified against a fresh 0.9.2 deployment. self-hosting.md: - Require CRAWL4AI_API_TOKEN in every quickstart (docker run, compose, manual build); explain the loopback-only bind and the resulting "connection reset" on published ports, plus startup-delay troubleshooting - Compose: .llm.env is required; the token must be set inside it (host `export` is not forwarded to the container) - Replace the removed inline-Python hooks API (~600 lines) with the declarative hooks reference: CRAWL4AI_HOOKS_ENABLED flag, action table, /hooks/info, and 0.8.x migration notes - Rewrite /screenshot and /pdf for the artifact flow; note output_path is removed and currently silently ignored - Fix dashboard URL (/dashboard, not /monitor); document the token bar in the playground and dashboard UIs - Fix stale versions (0.8.0 -> 0.9.2); drop the SDK function-hooks example MIGRATION.md: - Token setup instructions for compose (.llm.env) and docker run (explicit -e form); clarify the loopback failure symptom - Document CRAWL4AI_HOOKS_ENABLED and the silent failure modes of legacy hooks.code and output_path .llm.env.example: - Add CRAWL4AI_API_TOKEN with guidance; add CRAWL4AI_HOOKS_ENABLED
1 parent 2d8f673 commit a659914

3 files changed

Lines changed: 245 additions & 694 deletions

File tree

deploy/docker/.llm.env.example

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,14 @@
1+
# REQUIRED for a reachable server: API token for the Docker server (0.9.0+).
2+
# Without it the server binds loopback inside the container and the published
3+
# port answers with "connection reset". Any non-empty value works, but treat it
4+
# as a password — use a long random string (e.g. from: openssl rand -hex 32).
5+
# Note: with docker compose, the token MUST be set here — exporting it in your
6+
# shell does not reach the container.
7+
CRAWL4AI_API_TOKEN=
8+
9+
# Optional: enable declarative hooks support (disabled by default)
10+
# CRAWL4AI_HOOKS_ENABLED=true
11+
112
# LLM Provider Keys
213
OPENAI_API_KEY=your_openai_key_here
314
DEEPSEEK_API_KEY=your_deepseek_key_here

deploy/docker/MIGRATION.md

Lines changed: 29 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,11 +25,23 @@ loopback by default and will not expose itself without a credential.
2525
export CRAWL4AI_API_TOKEN="$(openssl rand -hex 32)"
2626
```
2727

28+
> ⚠️ **Docker Compose users:** `export` alone does **not** work — the shipped
29+
> `docker-compose.yml` does not forward host environment variables. Set the
30+
> token in the `.llm.env` file at the project root instead (the example file
31+
> ships an empty `CRAWL4AI_API_TOKEN=` line — fill it in).
32+
>
33+
> For plain `docker run`, pass it explicitly:
34+
> `-e CRAWL4AI_API_TOKEN="$CRAWL4AI_API_TOKEN"` (the value-less shorthand
35+
> `-e CRAWL4AI_API_TOKEN` silently passes empty from a shell where the variable
36+
> isn't set).
37+
2838
- With a token set, you may expose the server (put a TLS-terminating reverse
2939
proxy in front) and must send `Authorization: Bearer <token>` on every
3040
request except `GET /health`.
31-
- With **no** token set, the server binds `127.0.0.1` only and prints a one-off
32-
token at startup for local use.
41+
- With **no** token set, the server binds `127.0.0.1` only (the **container's**
42+
loopback — published ports answer with *connection reset* even though the
43+
container reports healthy) and prints a one-off token at startup for
44+
in-container use.
3345

3446
WebSocket clients (MCP, monitor) that can't set headers may pass `?token=...`.
3547

@@ -60,6 +72,10 @@ safe maximums.
6072

6173
### Hooks: declarative actions instead of code
6274

75+
Hooks are now **disabled by default** — enable them with
76+
`CRAWL4AI_HOOKS_ENABLED=true` in the container environment, or any request
77+
containing `hooks` returns HTTP 403.
78+
6379
`hooks.code` (Python strings) is replaced by a fixed set of declarative actions:
6480

6581
```jsonc
@@ -77,6 +93,13 @@ Available actions: `block_resources`, `add_cookies`, `set_headers`,
7793
`scroll_to_bottom`, `wait_for_timeout`. Call `GET /hooks/info` for the parameter
7894
schemas. Arbitrary hook code is available in a self-hosted in-process build.
7995

96+
> ⚠️ **Legacy `hooks.code` requests fail silently.** With hooks enabled, a
97+
> request in the old format returns HTTP 200 with
98+
> `"hooks": {"status": "success", "attached": []}` — the inline code is
99+
> dropped without error. If `attached` is empty, your hooks did not run.
100+
> (With hooks disabled, the same request returns the generic 403, whose
101+
> "enable hooks" hint will not make code hooks work either.)
102+
80103
### Screenshot / PDF: artifact id instead of `output_path`
81104

82105
`output_path` is removed. The server stores the result and returns an id + URL:
@@ -89,6 +112,10 @@ schemas. Arbitrary hook code is available in a self-hosted in-process build.
89112
Fetch the file with `GET /artifacts/{artifact_id}` (authenticated). Artifacts
90113
have a TTL and a storage quota.
91114

115+
> ⚠️ A request that still includes `output_path` is **silently ignored** — it
116+
> returns `success: true` with an artifact id, but no file is written to the
117+
> requested path. Update your code to fetch from `/artifacts/{artifact_id}`.
118+
92119
### LLM endpoints: provider by name
93120

94121
`base_url` is removed from `/md`, `/llm`, and `/llm/job`. Select a provider by

0 commit comments

Comments
 (0)