Skip to content

Commit 9a23111

Browse files
committed
fix(docker): document CDI context mount
Signed-off-by: Evan Lezar <elezar@nvidia.com>
1 parent 2dd7d3c commit 9a23111

4 files changed

Lines changed: 21 additions & 16 deletions

File tree

architecture/compute-runtimes.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -185,10 +185,11 @@ For all in-tree drivers, this is equivalent to selecting a single GPU.
185185
For Docker GPU sandboxes, the driver treats CDI specs as runtime metadata for
186186
both outer injection and inner sandbox policy. It selects opaque CDI device IDs,
187187
passes them to Docker, mounts daemon-reported CDI spec directories into
188-
supervisor-only paths, and uploads a versioned CDI context before starting the
189-
container. The supervisor resolves that context inside the sandbox and derives
190-
Landlock paths and supplemental groups from CDI `containerEdits`. Host-side CDI
191-
spec paths are diagnostic only and are never treated as sandbox policy paths.
188+
supervisor-only paths, and bind-mounts a gateway-owned versioned CDI context
189+
read-only before creating the container. The supervisor resolves that context
190+
inside the sandbox and derives Landlock paths and supplemental groups from CDI
191+
`containerEdits`. Host-side CDI spec paths are diagnostic only and are never
192+
treated as sandbox policy paths.
192193
Kubernetes must not infer CDI device IDs from the `nvidia.com/gpu` resource
193194
request; it needs a node-local selected-device handoff before using the same
194195
supervisor resolver.

crates/openshell-driver-docker/Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,10 +24,10 @@ serde_json = { workspace = true }
2424
prost-types = { workspace = true }
2525
bollard = { version = "0.20" }
2626
url = { workspace = true }
27-
tar = "0.4"
2827

2928
[dev-dependencies]
3029
prost-types = { workspace = true }
30+
tar = "0.4"
3131
temp-env = "0.3"
3232
tempfile = "3"
3333

crates/openshell-driver-docker/README.md

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,7 @@ contract:
9191
| `restart_policy = unless-stopped` | Keeps managed sandboxes resumable across daemon or gateway restarts. |
9292
| `PidsLimit` | Enforces the sandbox PID budget at the Docker cgroup layer. Set `[openshell.drivers.docker].sandbox_pids_limit = 0` to inherit the Docker/runtime default. |
9393
| CDI GPU request | Uses opaque `driver_config.cdi_devices` values when set; otherwise selects the requested count of NVIDIA CDI GPUs in round-robin order when daemon CDI support is detected. Docker daemon `/info` can permit `nvidia.com/gpu=all` as a WSL2 all-only compatibility fallback, where it counts as one selectable device. Exact CDI device lists must not contain duplicates and must match the effective GPU count. |
94-
| CDI context upload | For GPU/CDI sandboxes only, mounts daemon-reported CDI spec directories read-only under `/run/openshell/supervisor/cdi-specs/<n>` and uploads `/run/openshell/supervisor/cdi-context.json` after container create and before start. |
94+
| CDI context mount | For GPU/CDI sandboxes only, creates a gateway-owned context file and bind-mounts it read-only at `/run/openshell/supervisor/cdi-context.json`; daemon-reported CDI spec directories are mounted read-only under `/run/openshell/supervisor/cdi-specs/<n>`. |
9595

9696
The agent child process does not retain these supervisor privileges.
9797

@@ -102,11 +102,13 @@ CDI device IDs from `driver_config.cdi_devices` or the daemon's discovered CDI
102102
inventory, then passes the same IDs to Docker with a CDI `DeviceRequest`.
103103

104104
When a GPU/CDI request is present, the driver also mounts the Docker
105-
daemon-reported `Info.CDISpecDirs` into supervisor-only paths and uploads a
106-
small versioned CDI context through Docker's container archive API. The context
107-
uses container-side spec paths for resolution and keeps host-side spec sources
108-
diagnostic-only. If the upload fails, the driver removes the created container
109-
and sandbox token file before reporting the failure.
105+
daemon-reported `Info.CDISpecDirs` into supervisor-only paths. Before container
106+
creation, it writes a small versioned CDI context in gateway-owned state and
107+
bind-mounts it read-only into the supervisor. The context uses container-side
108+
spec paths for resolution and keeps host-side spec sources diagnostic-only. If
109+
context or token creation fails, the driver removes any created state files; if
110+
container creation or start fails, it also removes the container and state
111+
files before reporting the failure.
110112

111113
The sandbox supervisor resolves the selected IDs from those mounted specs
112114
before it launches agent processes. CDI device nodes become read-write

docs/reference/sandbox-compute-drivers.mdx

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -158,11 +158,13 @@ For GPU-backed Docker sandboxes, configure Docker CDI before starting the gatewa
158158
For Docker GPU/CDI sandboxes, OpenShell uses Docker's selected CDI device IDs
159159
and daemon-reported CDI spec directories to build a supervisor-only CDI
160160
context. The driver mounts the spec directories read-only into the sandbox
161-
container, uploads `cdi-context.json` before starting the container, and removes
162-
the created container if that upload fails. The supervisor resolves the context
163-
inside the sandbox and derives the inner filesystem and supplemental group
164-
requirements from CDI specs. Non-GPU Docker sandboxes do not receive the CDI
165-
context, spec mounts, or CDI-derived policy changes.
161+
container. Before creation, it writes a gateway-owned `cdi-context.json` and
162+
bind-mounts it read-only into the supervisor. If context or token creation
163+
fails, the driver removes the created state files; if container creation or
164+
start fails, it also removes the container and state files. The supervisor
165+
resolves the context inside the sandbox and derives the inner filesystem and
166+
supplemental group requirements from CDI specs. Non-GPU Docker sandboxes do not
167+
receive the CDI context, spec mounts, or CDI-derived policy changes.
166168

167169
### Docker Driver Config Mounts
168170

0 commit comments

Comments
 (0)