You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
| appproxy-worker |`/etc/backend.ai/proxy-worker.toml`| one container per worker: each needs its OWN toml with a unique `authority`, protocol (`http`/`tcp`), `api_bind_addr` port, and a non-overlapping `[proxy_worker.port_proxy] bind_port_range` (port-based frontends only) — and the compose port mappings must match |
62
62
63
+
The `/etc/backend.ai/*` targets are what the images' **default commands** read.
64
+
The DOCKER install mode (see the reference compose file below) instead keeps
65
+
every config in the parity-mounted install directory and overrides each
66
+
service's `command:` to point there — either layout works; pick one per
67
+
deployment.
68
+
63
69
Shared prerequisites:
64
70
65
71
| Item | Used by | Why |
@@ -79,7 +85,7 @@ together they amount to root-equivalent control of the host.
79
85
| Requirement | manager | agent | Why |
80
86
|---|---|---|---|
81
87
|`network_mode: host`| optional | ✅ | Agent: kernel↔agent ZMQ/service ports and agent RPC are advertised on host addresses; kernels spawned on the host network must reach them. Manager: convenience only — the bridge alternative works via the `announce-addr` / `announce-internal-addr` knobs |
82
-
|`privileged: true`|—| ✅ | Agent: container/device management against the host daemon; sysfs reads for metrics. The manager does not need it — the Docker socket alone suffices for its (conditional) Docker use |
88
+
|`privileged: true`|default| ✅ | Agent: container/device management against the host daemon; sysfs reads for metrics. The manager does not strictly need it — the Docker socket alone suffices for its (conditional) Docker use — but the DOCKER install mode's generated compose grants it by default; remove the flag for a least-privilege deployment|
83
89
|`/var/run/docker.sock` bind mount | conditional | ✅ | DooD: containers are created by talking to the **host** Docker daemon. Manager: only when the `local` container registry is used |
84
90
|`pid: host`| — | ✅ | Host PID namespace visibility: the agent inspects and signals kernel processes by host PID |
85
91
|`cgroup: host` (host cgroup namespace) | — | ✅ |**Required, not optional** — see below |
@@ -110,15 +116,21 @@ the same absolute path on both sides. The paths are set by `agent.toml` —
110
116
**every one of them must be an absolute path**, bind-mounted host↔container at
111
117
the identical location:
112
118
113
-
| Config knob (`agent.toml`) | Reference value | Used for |
119
+
The values below are the defaults the DOCKER install mode writes
120
+
(`<install-dir>` is the install target directory); a hand-rolled deployment
121
+
may choose any absolute paths as long as the parity rule holds.
122
+
123
+
| Config knob (`agent.toml`) | DOCKER-mode default | Used for |
114
124
|---|---|---|
115
-
|`[container] scratch-root`|`/var/lib/backend.ai/scratches`| Scratch roots of kernel containers |
|`[agent] var-base-path`|`/var/lib/backend.ai`| Plugin state bind-mounted into kernels (e.g. accelerator hook caches) |
118
-
| env `BACKENDAI_KRUNNER_SHARED`|`/var/lib/backend.ai/krunner`| Kernel-runner files: the image entrypoint copies them here so the host daemon can mount them into kernels. Covered automatically by the `/var/lib/backend.ai` parity mount; the entrypoint **refuses to start** without it — override the path via the `BACKENDAI_KRUNNER_SHARED` env var |
125
+
|`[container] scratch-root`|`<install-dir>/scratches`| Scratch roots of kernel containers |
126
+
|`[agent] mount-path`|`<install-dir>/vfolder/local`| Vfolder tree whose subdirectories become kernel bind-mount sources |
|`[agent] var-base-path`|`<install-dir>/var/agent`| Plugin state bind-mounted into kernels (e.g. accelerator hook caches) |
129
+
|`[agent] image-commit-path`|`<install-dir>/tmp/backend.ai/commit`| Session image-commit tarballs written by the host daemon |
130
+
| env `BACKENDAI_KRUNNER_SHARED`|`/var/lib/backend.ai/krunner`| Kernel-runner files: the image entrypoint copies them here so the host daemon can mount them into kernels. Mounted as its **own** bind mount (the Docker daemon creates the host directory on first start); the entrypoint **refuses to start** without it |
119
131
120
-
With the reference values, two parity mounts cover everything:
121
-
`/var/lib/backend.ai` and `/tmp/backend.ai`.
132
+
With these defaults, two mounts cover everything: the `<install-dir>` parity
133
+
mount and the fixed `/var/lib/backend.ai/krunner` krunner share.
122
134
123
135
Vfolder roots (e.g. `/vfroot/local/volume1`) follow the same rule on the
124
136
**storage-proxy**: mount each volume at the identical absolute path on host and
@@ -135,20 +147,48 @@ namespace is invisible to the host daemon — use `hostdir`.
135
147
136
148
`docker-compose.monorepo.yml` at the repository root is a **partial, legacy
137
149
example** — it uses different image names, includes no agent or storage-proxy,
138
-
and runs on a bridge network. The fragment below is the authoritative
139
-
reference for the two elevated services, verified against a live deployment.
140
-
Replace `<version>` with a tag from the tagging scheme above. The `cgroup:`
141
-
key requires Docker Compose v2.15+.
150
+
and runs on a bridge network. The authoritative reference is the compose file
151
+
the **DOCKER install mode** of `backend.ai-installer` generates at
152
+
`<install-dir>/docker-compose.services.yml` (rendered from
153
+
`src/ai/backend/install/configs/docker-compose.services.yml`). Its contract:
154
+
155
+
- Every service runs on the host network, so the generated configs use the
156
+
same `127.0.0.1` addressing as a package-based install.
157
+
- The install directory is bind-mounted into every container at the identical
158
+
absolute path, and each service's `command:` reads its config from there —
159
+
no `/etc/backend.ai` mounts.
160
+
-`/etc/machine-id` is passed through read-only to the manager and agent so
161
+
anything deriving a stable host identity sees the host's, not the
162
+
container's.
163
+
- All images are pinned to the installer's own version (the event-bus
164
+
version-skew rule above).
165
+
- The compose project name is fixed (`backendai-services`) so the file never
166
+
shares a project with the halfstack file the installer places in the same
167
+
directory.
168
+
- One-off management commands run via `docker compose run` on a dedicated
169
+
non-privileged `manager-cli` twin of the manager (no Docker socket, no
170
+
restart policy; its `cli` profile keeps `up -d` from starting it).
171
+
- No agent-watcher container ships in this mode, and the app-proxy data plane
172
+
runs as an `appproxy-worker` / `appproxy-worker-tcp` pair.
173
+
174
+
The fragment below reproduces the two elevated services. Replace `<version>`
175
+
with a tag from the tagging scheme above and `<install-dir>` with the install
176
+
target directory. The `cgroup:` key requires Docker Compose v2.15+.
142
177
143
178
```yaml
179
+
name: backendai-services
144
180
services:
145
181
manager:
146
182
image: lablup/backend.ai-manager:<version>
147
183
network_mode: host # optional — bridge works via the announce-addr knobs
184
+
privileged: true # installer default; the socket alone suffices (see the matrix) — remove for least privilege
0 commit comments