Skip to content

Commit 98ef34f

Browse files
feat(drivers): inject local sandbox identity
Make Docker and Podman select a validated numeric agent identity and inject it into the supervisor, allowing userless images to run without a baked-in sandbox account.\n\nCloses #2331 Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
1 parent e3d26dd commit 98ef34f

22 files changed

Lines changed: 388 additions & 69 deletions

File tree

Cargo.lock

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

architecture/compute-runtimes.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,12 @@ when a sandbox create request asks for GPU resources.
3838

3939
Per-sandbox CPU and memory values currently enter the driver layer through
4040
template resource limits. Docker and Podman apply them as runtime limits.
41+
42+
Docker and Podman operators can configure `sandbox_uid` and `sandbox_gid` in
43+
their driver tables. When unset, the drivers resolve `10001:10001` and inject
44+
it for the agent child; the root supervisor remains the container entry process.
45+
Rootless Podman requires that the chosen values are available in its subordinate
46+
UID/GID mappings.
4147
Kubernetes mirrors each limit into the matching request. VM accepts the fields
4248
but currently ignores them.
4349

architecture/sandbox.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,13 @@ container-granted capabilities. This is fail-closed: the supervisor retains
2121
aborts unless the bounding set ends up empty. A `setpcap` `EPERM` is tolerated
2222
only when the set is already empty; any other outcome fails the spawn.
2323

24+
The compute runtime selects the agent's numeric UID and GID; sandbox creators
25+
do not control it. Docker and Podman start the supervisor as root, inject the
26+
driver-resolved identity, and the supervisor drops only the agent child to that
27+
identity. A container image therefore need not contain a matching passwd or
28+
group entry. Agent children receive a stable `HOME=/sandbox`; numeric identities
29+
use the UID as their `USER` and `LOGNAME` presentation value.
30+
2431
## Startup Flow
2532

2633
1. The compute runtime starts the workload with sandbox identity, callback

crates/openshell-core/src/sandbox_env.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,7 +103,7 @@ pub const PROVIDER_SPIFFE_WORKLOAD_API_SOCKET: &str =
103103
/// Resolved sandbox UID used to override `run_as_user` when the policy
104104
/// specifies a numeric value instead of the hardcoded "sandbox" user name.
105105
///
106-
/// Set by compute drivers (Kubernetes, Docker, VM) from resolved config or
106+
/// Set by compute drivers (Kubernetes, Docker, Podman, and VM) from resolved config or
107107
/// cluster autodetection. The supervisor reads this at startup and uses it
108108
/// directly with `setuid()` / `chown()` without requiring an `/etc/passwd`
109109
/// entry in the sandbox image.

crates/openshell-driver-docker/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ repository.workspace = true
1212

1313
[dependencies]
1414
openshell-core = { path = "../openshell-core", default-features = false }
15+
openshell-policy = { path = "../openshell-policy" }
1516

1617
tokio = { workspace = true }
1718
tonic = { workspace = true }

crates/openshell-driver-docker/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ contract:
3232
| `apparmor=unconfined` | Avoids Docker's default profile blocking required mount operations. |
3333
| `restart_policy = unless-stopped` | Keeps managed sandboxes resumable across daemon or gateway restarts. |
3434
| `PidsLimit` | Enforces the sandbox PID budget at the Docker cgroup layer. Set `[openshell.drivers.docker].sandbox_pids_limit = 0` to inherit the Docker/runtime default. |
35+
| `sandbox_uid` / `sandbox_gid` | Operator-controlled numeric identity for the agent child. The supervisor still starts as root; the UID defaults to `10001` and the GID defaults to the resolved UID. |
3536
| CDI GPU request | Uses opaque `driver_config.cdi_devices` values when set; otherwise selects the requested count of NVIDIA CDI GPUs in round-robin order when daemon CDI support is detected. Docker daemon `/info` can permit `nvidia.com/gpu=all` as a WSL2 all-only compatibility fallback, where it counts as one selectable device. Exact CDI device lists must not contain duplicates and must match the effective GPU count. |
3637

3738
The agent child process does not retain these supervisor privileges.

crates/openshell-driver-docker/src/lib.rs

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@ const SUPERVISOR_PATH: &str = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
7777
const HOST_OPENSHELL_INTERNAL: &str = "host.openshell.internal";
7878
const HOST_DOCKER_INTERNAL: &str = "host.docker.internal";
7979
const DOCKER_NETWORK_DRIVER: &str = "bridge";
80+
const DEFAULT_SANDBOX_UID: u32 = 10_001;
8081

8182
/// Queried by the Docker driver to decide when a sandbox's supervisor
8283
/// relay is live. Implementations return `true` once a sandbox has an
@@ -136,6 +137,14 @@ pub struct DockerComputeConfig {
136137
/// Set to `0` to leave Docker's runtime/default PID limit unchanged.
137138
pub sandbox_pids_limit: i64,
138139

140+
/// Numeric identity used for the agent process. The root supervisor
141+
/// retains UID 0 while preparing the sandbox.
142+
pub sandbox_uid: Option<u32>,
143+
144+
/// Numeric group used for the agent process. Defaults to the resolved
145+
/// sandbox UID when unset.
146+
pub sandbox_gid: Option<u32>,
147+
139148
/// Allow sandbox requests to attach host bind mounts through
140149
/// `template.driver_config`.
141150
#[serde(default)]
@@ -158,6 +167,8 @@ impl Default for DockerComputeConfig {
158167
host_gateway_ip: String::new(),
159168
ssh_socket_path: "/run/openshell/ssh.sock".to_string(),
160169
sandbox_pids_limit: DEFAULT_SANDBOX_PIDS_LIMIT,
170+
sandbox_uid: None,
171+
sandbox_gid: None,
161172
enable_bind_mounts: false,
162173
}
163174
}
@@ -187,6 +198,8 @@ struct DockerDriverRuntimeConfig {
187198
supports_gpu: bool,
188199
allow_all_default_gpu: bool,
189200
sandbox_pids_limit: i64,
201+
sandbox_uid: u32,
202+
sandbox_gid: u32,
190203
enable_bind_mounts: bool,
191204
}
192205

@@ -307,6 +320,7 @@ impl DockerComputeDriver {
307320
docker_config: &DockerComputeConfig,
308321
supervisor_readiness: Arc<dyn SupervisorReadiness>,
309322
) -> CoreResult<Self> {
323+
let sandbox_identity = resolve_sandbox_identity(docker_config)?;
310324
let docker = Docker::connect_with_local_defaults()
311325
.map_err(|err| Error::execution(format!("failed to create Docker client: {err}")))?;
312326
let version = docker.version().await.map_err(|err| {
@@ -370,6 +384,8 @@ impl DockerComputeDriver {
370384
supports_gpu,
371385
allow_all_default_gpu,
372386
sandbox_pids_limit: docker_config.sandbox_pids_limit,
387+
sandbox_uid: sandbox_identity.0,
388+
sandbox_gid: sandbox_identity.1,
373389
enable_bind_mounts: docker_config.enable_bind_mounts,
374390
},
375391
events: broadcast::channel(WATCH_BUFFER).0,
@@ -2162,6 +2178,14 @@ fn build_environment(sandbox: &DriverSandbox, config: &DockerDriverRuntimeConfig
21622178
openshell_core::sandbox_env::TELEMETRY_ENABLED.to_string(),
21632179
openshell_core::telemetry::enabled_env_value().to_string(),
21642180
);
2181+
environment.insert(
2182+
openshell_core::sandbox_env::SANDBOX_UID.to_string(),
2183+
config.sandbox_uid.to_string(),
2184+
);
2185+
environment.insert(
2186+
openshell_core::sandbox_env::SANDBOX_GID.to_string(),
2187+
config.sandbox_gid.to_string(),
2188+
);
21652189
// The root supervisor executes namespace helpers during bootstrap; keep
21662190
// their search path driver-owned even when the template/spec set PATH.
21672191
environment.insert("PATH".to_string(), SUPERVISOR_PATH.to_string());
@@ -2619,6 +2643,22 @@ fn validate_sandbox_pids_limit(value: i64) -> CoreResult<()> {
26192643
Ok(())
26202644
}
26212645

2646+
fn resolve_sandbox_identity(config: &DockerComputeConfig) -> CoreResult<(u32, u32)> {
2647+
let uid = config.sandbox_uid.unwrap_or(DEFAULT_SANDBOX_UID);
2648+
let gid = config.sandbox_gid.unwrap_or(uid);
2649+
for (field, value) in [("sandbox_uid", uid), ("sandbox_gid", gid)] {
2650+
if !(openshell_policy::MIN_SANDBOX_UID..=openshell_policy::MAX_SANDBOX_UID).contains(&value)
2651+
{
2652+
return Err(Error::config(format!(
2653+
"docker {field} must be in [{}, {}]",
2654+
openshell_policy::MIN_SANDBOX_UID,
2655+
openshell_policy::MAX_SANDBOX_UID,
2656+
)));
2657+
}
2658+
}
2659+
Ok((uid, gid))
2660+
}
2661+
26222662
fn docker_pids_limit(value: i64) -> Result<Option<i64>, Status> {
26232663
if value < 0 {
26242664
return Err(Status::failed_precondition(

crates/openshell-driver-docker/src/tests.rs

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,8 @@ fn runtime_config() -> DockerDriverRuntimeConfig {
113113
supports_gpu: false,
114114
allow_all_default_gpu: false,
115115
sandbox_pids_limit: DEFAULT_SANDBOX_PIDS_LIMIT,
116+
sandbox_uid: 10_001,
117+
sandbox_gid: 10_001,
116118
enable_bind_mounts: false,
117119
}
118120
}
@@ -551,6 +553,50 @@ fn docker_compute_config_disables_bind_mounts_by_default() {
551553
assert!(!cfg.enable_bind_mounts);
552554
}
553555

556+
#[test]
557+
fn docker_sandbox_identity_defaults_to_10001() {
558+
assert_eq!(
559+
resolve_sandbox_identity(&DockerComputeConfig::default()).unwrap(),
560+
(10_001, 10_001)
561+
);
562+
}
563+
564+
#[test]
565+
fn docker_sandbox_identity_uses_resolved_uid_for_default_gid() {
566+
let config = DockerComputeConfig {
567+
sandbox_uid: Some(12_345),
568+
..DockerComputeConfig::default()
569+
};
570+
assert_eq!(resolve_sandbox_identity(&config).unwrap(), (12_345, 12_345));
571+
}
572+
573+
#[test]
574+
fn docker_sandbox_identity_allows_configured_gid_without_uid() {
575+
let config = DockerComputeConfig {
576+
sandbox_gid: Some(12_346),
577+
..DockerComputeConfig::default()
578+
};
579+
assert_eq!(resolve_sandbox_identity(&config).unwrap(), (10_001, 12_346));
580+
}
581+
582+
#[test]
583+
fn docker_sandbox_identity_rejects_system_uid() {
584+
let config = DockerComputeConfig {
585+
sandbox_uid: Some(999),
586+
..DockerComputeConfig::default()
587+
};
588+
assert!(resolve_sandbox_identity(&config).is_err());
589+
}
590+
591+
#[test]
592+
fn docker_sandbox_identity_rejects_gid_above_policy_range() {
593+
let config = DockerComputeConfig {
594+
sandbox_gid: Some(openshell_policy::MAX_SANDBOX_UID + 1),
595+
..DockerComputeConfig::default()
596+
};
597+
assert!(resolve_sandbox_identity(&config).is_err());
598+
}
599+
554600
#[test]
555601
fn container_create_body_sets_driver_owned_pids_limit() {
556602
let body = build_container_create_body(&test_sandbox(), &runtime_config()).unwrap();
@@ -567,6 +613,28 @@ fn build_environment_sets_docker_tls_paths() {
567613
assert!(env.contains(&"TEMPLATE_ENV=template".to_string()));
568614
assert!(env.contains(&"SPEC_ENV=spec".to_string()));
569615
assert!(env.contains(&"OPENSHELL_SANDBOX_COMMAND=sleep infinity".to_string()));
616+
assert!(env.contains(&"OPENSHELL_SANDBOX_UID=10001".to_string()));
617+
assert!(env.contains(&"OPENSHELL_SANDBOX_GID=10001".to_string()));
618+
}
619+
620+
#[test]
621+
fn build_environment_keeps_sandbox_identity_driver_controlled() {
622+
let mut sandbox = test_sandbox();
623+
let spec = sandbox.spec.as_mut().unwrap();
624+
spec.environment.insert(
625+
openshell_core::sandbox_env::SANDBOX_UID.to_string(),
626+
"1234".to_string(),
627+
);
628+
spec.template.as_mut().unwrap().environment.insert(
629+
openshell_core::sandbox_env::SANDBOX_GID.to_string(),
630+
"1234".to_string(),
631+
);
632+
633+
let env = build_environment(&sandbox, &runtime_config());
634+
assert!(env.contains(&"OPENSHELL_SANDBOX_UID=10001".to_string()));
635+
assert!(env.contains(&"OPENSHELL_SANDBOX_GID=10001".to_string()));
636+
assert!(!env.contains(&"OPENSHELL_SANDBOX_UID=1234".to_string()));
637+
assert!(!env.contains(&"OPENSHELL_SANDBOX_GID=1234".to_string()));
570638
}
571639

572640
#[test]

crates/openshell-driver-podman/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ path = "src/main.rs"
1616

1717
[dependencies]
1818
openshell-core = { path = "../openshell-core", default-features = false }
19+
openshell-policy = { path = "../openshell-policy" }
1920

2021
tokio = { workspace = true }
2122
tonic = { workspace = true, features = ["transport"] }

crates/openshell-driver-podman/README.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,12 @@ The container spec in `container.rs` sets these security-critical fields:
5050

5151
The restricted agent child does not retain these supervisor privileges.
5252

53+
The gateway operator may set `[openshell.drivers.podman].sandbox_uid` and
54+
`sandbox_gid` for the agent child. The supervisor remains `0:0` during setup;
55+
the UID defaults to `10001` and the GID defaults to the resolved UID. In
56+
rootless mode, configure values that are available in Podman's subordinate
57+
UID/GID mapping.
58+
5359
## Driver Config Mounts
5460

5561
The gateway forwards the `podman` block from `--driver-config-json` to this

0 commit comments

Comments
 (0)