Skip to content

Commit 7320426

Browse files
committed
doc(BA-7321): align docker/README.md with the DOCKER install-mode defaults
1 parent 01679e7 commit 7320426

1 file changed

Lines changed: 62 additions & 21 deletions

File tree

docker/README.md

Lines changed: 62 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,12 @@ path the image's default command reads:
6060
| appproxy-coordinator | `/etc/backend.ai/proxy-coordinator.toml` | |
6161
| appproxy-worker | `/etc/backend.ai/proxy-worker.toml` | one container per worker: each needs its OWN toml with a unique `authority`, protocol (`http`/`tcp`), `api_bind_addr` port, and a non-overlapping `[proxy_worker.port_proxy] bind_port_range` (port-based frontends only) — and the compose port mappings must match |
6262

63+
The `/etc/backend.ai/*` targets are what the images' **default commands** read.
64+
The DOCKER install mode (see the reference compose file below) instead keeps
65+
every config in the parity-mounted install directory and overrides each
66+
service's `command:` to point there — either layout works; pick one per
67+
deployment.
68+
6369
Shared prerequisites:
6470

6571
| Item | Used by | Why |
@@ -79,7 +85,7 @@ together they amount to root-equivalent control of the host.
7985
| Requirement | manager | agent | Why |
8086
|---|---|---|---|
8187
| `network_mode: host` | optional || Agent: kernel↔agent ZMQ/service ports and agent RPC are advertised on host addresses; kernels spawned on the host network must reach them. Manager: convenience only — the bridge alternative works via the `announce-addr` / `announce-internal-addr` knobs |
82-
| `privileged: true` | || Agent: container/device management against the host daemon; sysfs reads for metrics. The manager does not need it — the Docker socket alone suffices for its (conditional) Docker use |
88+
| `privileged: true` | default || Agent: container/device management against the host daemon; sysfs reads for metrics. The manager does not strictly need it — the Docker socket alone suffices for its (conditional) Docker use — but the DOCKER install mode's generated compose grants it by default; remove the flag for a least-privilege deployment |
8389
| `/var/run/docker.sock` bind mount | conditional || DooD: containers are created by talking to the **host** Docker daemon. Manager: only when the `local` container registry is used |
8490
| `pid: host` ||| Host PID namespace visibility: the agent inspects and signals kernel processes by host PID |
8591
| `cgroup: host` (host cgroup namespace) ||| **Required, not optional** — see below |
@@ -110,15 +116,21 @@ the same absolute path on both sides. The paths are set by `agent.toml` —
110116
**every one of them must be an absolute path**, bind-mounted host↔container at
111117
the identical location:
112118

113-
| Config knob (`agent.toml`) | Reference value | Used for |
119+
The values below are the defaults the DOCKER install mode writes
120+
(`<install-dir>` is the install target directory); a hand-rolled deployment
121+
may choose any absolute paths as long as the parity rule holds.
122+
123+
| Config knob (`agent.toml`) | DOCKER-mode default | Used for |
114124
|---|---|---|
115-
| `[container] scratch-root` | `/var/lib/backend.ai/scratches` | Scratch roots of kernel containers |
116-
| `[agent] ipc-base-path` | `/tmp/backend.ai/ipc` (this document's chosen value; `configs/agent/sample.toml` ships `/var/run/backend.ai/ipc`) | Agent↔kernel IPC sockets |
117-
| `[agent] var-base-path` | `/var/lib/backend.ai` | Plugin state bind-mounted into kernels (e.g. accelerator hook caches) |
118-
| env `BACKENDAI_KRUNNER_SHARED` | `/var/lib/backend.ai/krunner` | Kernel-runner files: the image entrypoint copies them here so the host daemon can mount them into kernels. Covered automatically by the `/var/lib/backend.ai` parity mount; the entrypoint **refuses to start** without it — override the path via the `BACKENDAI_KRUNNER_SHARED` env var |
125+
| `[container] scratch-root` | `<install-dir>/scratches` | Scratch roots of kernel containers |
126+
| `[agent] mount-path` | `<install-dir>/vfolder/local` | Vfolder tree whose subdirectories become kernel bind-mount sources |
127+
| `[agent] ipc-base-path` | `<install-dir>/ipc/agent` | Agent↔kernel IPC sockets |
128+
| `[agent] var-base-path` | `<install-dir>/var/agent` | Plugin state bind-mounted into kernels (e.g. accelerator hook caches) |
129+
| `[agent] image-commit-path` | `<install-dir>/tmp/backend.ai/commit` | Session image-commit tarballs written by the host daemon |
130+
| env `BACKENDAI_KRUNNER_SHARED` | `/var/lib/backend.ai/krunner` | Kernel-runner files: the image entrypoint copies them here so the host daemon can mount them into kernels. Mounted as its **own** bind mount (the Docker daemon creates the host directory on first start); the entrypoint **refuses to start** without it |
119131

120-
With the reference values, two parity mounts cover everything:
121-
`/var/lib/backend.ai` and `/tmp/backend.ai`.
132+
With these defaults, two mounts cover everything: the `<install-dir>` parity
133+
mount and the fixed `/var/lib/backend.ai/krunner` krunner share.
122134

123135
Vfolder roots (e.g. `/vfroot/local/volume1`) follow the same rule on the
124136
**storage-proxy**: mount each volume at the identical absolute path on host and
@@ -135,20 +147,48 @@ namespace is invisible to the host daemon — use `hostdir`.
135147

136148
`docker-compose.monorepo.yml` at the repository root is a **partial, legacy
137149
example** — it uses different image names, includes no agent or storage-proxy,
138-
and runs on a bridge network. The fragment below is the authoritative
139-
reference for the two elevated services, verified against a live deployment.
140-
Replace `<version>` with a tag from the tagging scheme above. The `cgroup:`
141-
key requires Docker Compose v2.15+.
150+
and runs on a bridge network. The authoritative reference is the compose file
151+
the **DOCKER install mode** of `backend.ai-installer` generates at
152+
`<install-dir>/docker-compose.services.yml` (rendered from
153+
`src/ai/backend/install/configs/docker-compose.services.yml`). Its contract:
154+
155+
- Every service runs on the host network, so the generated configs use the
156+
same `127.0.0.1` addressing as a package-based install.
157+
- The install directory is bind-mounted into every container at the identical
158+
absolute path, and each service's `command:` reads its config from there —
159+
no `/etc/backend.ai` mounts.
160+
- `/etc/machine-id` is passed through read-only to the manager and agent so
161+
anything deriving a stable host identity sees the host's, not the
162+
container's.
163+
- All images are pinned to the installer's own version (the event-bus
164+
version-skew rule above).
165+
- The compose project name is fixed (`backendai-services`) so the file never
166+
shares a project with the halfstack file the installer places in the same
167+
directory.
168+
- One-off management commands run via `docker compose run` on a dedicated
169+
non-privileged `manager-cli` twin of the manager (no Docker socket, no
170+
restart policy; its `cli` profile keeps `up -d` from starting it).
171+
- No agent-watcher container ships in this mode, and the app-proxy data plane
172+
runs as an `appproxy-worker` / `appproxy-worker-tcp` pair.
173+
174+
The fragment below reproduces the two elevated services. Replace `<version>`
175+
with a tag from the tagging scheme above and `<install-dir>` with the install
176+
target directory. The `cgroup:` key requires Docker Compose v2.15+.
142177

143178
```yaml
179+
name: backendai-services
144180
services:
145181
manager:
146182
image: lablup/backend.ai-manager:<version>
147183
network_mode: host # optional — bridge works via the announce-addr knobs
184+
privileged: true # installer default; the socket alone suffices (see the matrix) — remove for least privilege
185+
working_dir: <install-dir>
186+
command: ["python", "-m", "ai.backend.manager.server", "--config", "<install-dir>/manager.toml"]
148187
volumes:
149-
- /var/run/docker.sock:/var/run/docker.sock # only when the `local` container registry is used
150-
- ./manager.toml:/etc/backend.ai/manager.toml:ro
151-
- ./fixtures:/app/fixtures # read-write: the entrypoint writes the RPC keypair here
188+
- /var/run/docker.sock:/var/run/docker.sock # needed only when the `local` container registry is used
189+
- /etc/machine-id:/etc/machine-id:ro
190+
# parity mount: configs, fixtures/ (RPC keypair, written relative to working_dir), vfolder/
191+
- <install-dir>:<install-dir>
152192
restart: unless-stopped
153193

154194
agent:
@@ -157,20 +197,21 @@ services:
157197
privileged: true
158198
pid: host
159199
cgroup: host # REQUIRED on cgroup v2 hosts; Docker defaults to private
160-
deploy: # GPU nodes only
161-
resources:
200+
deploy: # GPU nodes only (the installer currently rejects --accelerator
201+
resources: # until the published images bundle the accelerator plugins)
162202
reservations:
163203
devices:
164204
- driver: nvidia
165205
count: all
166206
capabilities: [gpu]
207+
working_dir: <install-dir>
208+
command: ["python", "-m", "ai.backend.agent.server", "-f", "<install-dir>/agent.toml"]
167209
volumes:
168210
- /var/run/docker.sock:/var/run/docker.sock
169-
- ./agent.toml:/etc/backend.ai/agent.toml:ro
211+
- /etc/machine-id:/etc/machine-id:ro
170212
# path-parity mounts: host path == container path
171-
# (the krunner share /var/lib/backend.ai/krunner is covered by the /var/lib/backend.ai mount)
172-
- /var/lib/backend.ai:/var/lib/backend.ai
173-
- /tmp/backend.ai:/tmp/backend.ai
213+
- <install-dir>:<install-dir>
214+
- /var/lib/backend.ai/krunner:/var/lib/backend.ai/krunner # created by the Docker daemon on first start
174215
restart: unless-stopped
175216
```
176217

0 commit comments

Comments
 (0)