Skip to content

Commit 6461677

Browse files
authored
feat(policy): accept numeric UIDs for sandbox process identity (#1973)
* feat(policy): accept numeric UIDs in sandbox process identity validation Allow run_as_user and run_as_group to be either the literal 'sandbox' or a numeric UID/GID within [1000, 2_000_000_000]. This removes the hard dependency on a baked-in 'sandbox' user in container images, enabling compute drivers to inject resolved UIDs at sandbox creation. Phase 1 of #1959. Signed-off-by: Seth Jennings <sjenning@redhat.com> * feat(supervisor): accept numeric UIDs for process identity dropping Allow run_as_user and run_as_group to be numeric UIDs/GIDs, removing the hard dependency on a baked-in 'sandbox' user in container images. Changes: - validate_sandbox_user(): accepts numeric UIDs without passwd lookup (logs OCSF event); keeps passwd check for "sandbox" name; rejects non-numeric non-sandbox strings that fail passwd lookup - prepare_filesystem(): passes numeric UIDs/GIDs directly to chown() instead of requiring a passwd entry - drop_privileges(): resolves numeric UIDs/GIDs directly via UID::from_raw / Gid::from_raw; skips initgroups when target uid matches current euid; uses guard conditions before setgid/setuid calls - session_user_and_home(): falls back to ("{uid}", "/sandbox") for numeric UIDs, avoiding a passwd lookup that will fail Re-exports MIN_SANDBOX_UID and MAX_SANDBOX_UID from openshell-policy so callers have consistent range constants. Phase 2 of #1959. Signed-off-by: Seth Jennings <sjenning@redhat.com> * feat(driver-kubernetes): resolve sandbox UID/GID from config or OpenShift SCC annotations Phase 3 of the numeric-UID plan: allow operators to specify explicit sandbox_uid/sandbox_gid in Kubernetes driver config, auto-detect from OpenShift SCC namespace annotations, and propagate resolved values to supervisor container env vars and PVC init container securityContext. Changes: - Add sandbox_uid/sandbox_gid fields to KubernetesComputeConfig - Add SANDBOX_UID/SANDBOX_GID env var constants to openshell-core - Implement resolve_sandbox_identity() to fetch namespace annotations and auto-detect OpenShift SCC UID ranges (sa.scc.uid-range) - Pass resolved UID/GID through SandboxPodParams to pod spec builder - Inject SANDBOX_UID/SANDBOX_GID env vars into supervisor container - Update PVC init container securityContext with resolved UID/GID instead of hard-coded root - Add comprehensive unit tests for resolution logic and annotation parsing (resolve_sandbox_uid, resolve_sandbox_gid, OpenShift SCC annotation parsing) Signed-off-by: Seth Jennings <sjenning@redhat.com> * feat(driver-vm): add configurable sandbox UID/GID and update docs/examples Phase 4 of the numeric-UID plan: replace hardcoded SANDBOX_UID (10001) in VM rootfs preparation with configurable sandbox_uid/sandbox_gid fields. Changes: - Add sandbox_uid/sandbox_gid to VmDriverConfig with serde derives - Pass resolved UID/GID through prepare_sandbox_rootfs_from_image_root to ensure_sandbox_guest_user which writes /etc/passwd/group/gshadow - Update BYOC Dockerfile: remove groupadd/useradd, document runtime UID injection and the ability to skip baked-in sandbox user - Update gateway-config.mdx: document sandbox_uid/sandbox_gid for both Kubernetes (with OpenShift SCC autodetection) and VM drivers - Update sandbox-compute-drivers.mdx: add Sandbox User Identity section explaining numeric UID support across all compute drivers - Update rootfs tests to use non-default UIDs, verify config passthrough Signed-off-by: Seth Jennings <sjenning@redhat.com> * code review changes * fix(supervisor): harden tests for restricted CI container environments Guard tests against CI-specific constraints: root without CAP_SETPCAP, UIDs with no /etc/passwd entry, and restricted /proc access. Signed-off-by: Seth Jennings <sjennings@nvidia.com> Signed-off-by: Seth Jennings <sjenning@redhat.com> --------- Signed-off-by: Seth Jennings <sjenning@redhat.com> Signed-off-by: Seth Jennings <sjennings@nvidia.com>
1 parent 5f9bf9c commit 6461677

21 files changed

Lines changed: 1661 additions & 109 deletions

File tree

Cargo.lock

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

crates/openshell-core/src/sandbox_env.rs

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,3 +71,18 @@ pub const K8S_SA_TOKEN_FILE: &str = "OPENSHELL_K8S_SA_TOKEN_FILE";
7171
/// exchanges without using SPIFFE for gateway authentication.
7272
pub const PROVIDER_SPIFFE_WORKLOAD_API_SOCKET: &str =
7373
"OPENSHELL_PROVIDER_SPIFFE_WORKLOAD_API_SOCKET";
74+
75+
/// Resolved sandbox UID used to override `run_as_user` when the policy
76+
/// specifies a numeric value instead of the hardcoded "sandbox" user name.
77+
///
78+
/// Set by compute drivers (Kubernetes, Docker, VM) from resolved config or
79+
/// cluster autodetection. The supervisor reads this at startup and uses it
80+
/// directly with `setuid()` / `chown()` without requiring an `/etc/passwd`
81+
/// entry in the sandbox image.
82+
pub const SANDBOX_UID: &str = "OPENSHELL_SANDBOX_UID";
83+
84+
/// Resolved sandbox GID paired with [`SANDBOX_UID`].
85+
///
86+
/// Used alongside UID for PVC init container `chown` operations and when the
87+
/// supervisor drops privileges to a group other than the UID's primary group.
88+
pub const SANDBOX_GID: &str = "OPENSHELL_SANDBOX_GID";

crates/openshell-driver-kubernetes/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ path = "src/main.rs"
1616

1717
[dependencies]
1818
openshell-core = { path = "../openshell-core", default-features = false }
19+
openshell-policy = { path = "../openshell-policy" }
1920

2021
tokio = { workspace = true }
2122
tonic = { workspace = true, features = ["transport"] }

crates/openshell-driver-kubernetes/src/config.rs

Lines changed: 275 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -241,6 +241,19 @@ pub struct KubernetesComputeConfig {
241241
deserialize_with = "deserialize_provider_spiffe_workload_api_socket_path"
242242
)]
243243
pub provider_spiffe_workload_api_socket_path: String,
244+
/// UID used for privilege-drop operations and workspace init container
245+
/// ownership. The supervisor container always runs as UID 0 (root) to
246+
/// create network namespaces and configure Landlock/seccomp; the
247+
/// `sandbox_uid` is injected as the `SANDBOX_UID` environment variable so
248+
/// the supervisor knows which UID to drop to for child processes.
249+
/// When empty, the driver auto-detects from `OpenShift` SCC annotations on
250+
/// the target namespace; if those are also absent, falls back to `1000`.
251+
#[serde(default, skip_serializing_if = "Option::is_none")]
252+
pub sandbox_uid: Option<u32>,
253+
/// GID used alongside `sandbox_uid` for PVC init container operations.
254+
/// When empty and `sandbox_uid` is set, defaults to the resolved UID.
255+
#[serde(default, skip_serializing_if = "Option::is_none")]
256+
pub sandbox_gid: Option<u32>,
244257
}
245258

246259
/// Lower bound enforced by kubelet for projected SA tokens.
@@ -251,6 +264,18 @@ pub const MIN_SA_TOKEN_TTL_SECS: i64 = 600;
251264
/// pod start).
252265
pub const MAX_SA_TOKEN_TTL_SECS: i64 = 86_400;
253266

267+
/// Default sandbox UID used when neither config nor `OpenShift` SCC annotations
268+
/// provide a resolved value.
269+
pub(crate) const DEFAULT_SANDBOX_UID: u32 = 1000;
270+
271+
/// The annotation key for the `OpenShift` `ServiceAccount` UID range.
272+
/// Format: `<start>/<size>` (e.g. `1000000000/10000`).
273+
pub const ANNOTATION_SCC_UID_RANGE: &str = "openshift.io/sa.scc.uid-range";
274+
275+
/// The annotation key for the `OpenShift` `ServiceAccount` supplemental groups.
276+
/// Format: `<start>/<size>` (e.g. `1000000000/10000`).
277+
pub const ANNOTATION_SCC_SUPPLEMENTAL_GROUPS: &str = "openshift.io/sa.scc.supplemental-groups";
278+
254279
impl Default for KubernetesComputeConfig {
255280
fn default() -> Self {
256281
Self {
@@ -277,6 +302,8 @@ impl Default for KubernetesComputeConfig {
277302
default_runtime_class_name: String::new(),
278303
sa_token_ttl_secs: 3600,
279304
provider_spiffe_workload_api_socket_path: String::new(),
305+
sandbox_uid: None,
306+
sandbox_gid: None,
280307
}
281308
}
282309
}
@@ -308,6 +335,84 @@ impl KubernetesComputeConfig {
308335
&self.provider_spiffe_workload_api_socket_path,
309336
)
310337
}
338+
339+
/// Resolve the sandbox UID/GID pair.
340+
///
341+
/// Resolution order:
342+
/// 1. Configured `sandbox_uid` / `sandbox_gid` (explicit override)
343+
/// 2. `OpenShift` SCC namespace annotations (`sa.scc.uid-range`,
344+
/// `sa.scc.supplemental-groups`) — passed in as the optional
345+
/// `namespace_annotations` map
346+
/// 3. Fallback defaults: UID=`1000`, GID=UID
347+
pub fn resolve_sandbox_uid(
348+
&self,
349+
namespace_annotations: Option<&std::collections::BTreeMap<String, String>>,
350+
) -> u32 {
351+
if let Some(uid) = self.sandbox_uid {
352+
return uid;
353+
}
354+
if let Some(anns) = namespace_annotations
355+
&& let Some(range) = anns.get(ANNOTATION_SCC_UID_RANGE)
356+
&& let Some(uid) = Self::from_open_shift_uid_range(range)
357+
{
358+
return uid;
359+
}
360+
DEFAULT_SANDBOX_UID
361+
}
362+
363+
pub fn resolve_sandbox_gid(
364+
&self,
365+
resolved_uid: u32,
366+
_namespace_annotations: Option<&std::collections::BTreeMap<String, String>>,
367+
) -> u32 {
368+
self.sandbox_gid
369+
.or(self.sandbox_uid)
370+
.unwrap_or(resolved_uid)
371+
}
372+
373+
/// Parse `OpenShift` SCC `sa.scc.uid-range` annotation.
374+
///
375+
/// Format: `<start>/<size>` (e.g. `1000000000/10000`).
376+
pub fn from_open_shift_uid_range(annotation: &str) -> Option<u32> {
377+
let (start, _) = annotation.split_once('/')?;
378+
start.trim().parse::<u32>().ok().filter(|&uid| {
379+
(openshell_policy::MIN_SANDBOX_UID..=openshell_policy::MAX_SANDBOX_UID).contains(&uid)
380+
})
381+
}
382+
383+
/// Parse `OpenShift` SCC `sa.scc.supplemental-groups` annotation.
384+
pub fn from_open_shift_supplemental_groups(annotation: &str) -> Option<u32> {
385+
let (start, _) = annotation.split_once('/')?;
386+
start.trim().parse::<u32>().ok().filter(|&gid| {
387+
(openshell_policy::MIN_SANDBOX_UID..=openshell_policy::MAX_SANDBOX_UID).contains(&gid)
388+
})
389+
}
390+
391+
/// Validate that configured `sandbox_uid` and `sandbox_gid` fall within
392+
/// the policy-enforced UID/GID range. Called during driver initialization
393+
/// before any pod parameters are rendered.
394+
pub fn validate_sandbox_identity_config(&self) -> Result<(), String> {
395+
let range = openshell_policy::MIN_SANDBOX_UID..=openshell_policy::MAX_SANDBOX_UID;
396+
if let Some(uid) = self.sandbox_uid
397+
&& !range.contains(&uid)
398+
{
399+
return Err(format!(
400+
"sandbox_uid {uid} is outside the allowed range [{}, {}]",
401+
openshell_policy::MIN_SANDBOX_UID,
402+
openshell_policy::MAX_SANDBOX_UID,
403+
));
404+
}
405+
if let Some(gid) = self.sandbox_gid
406+
&& !range.contains(&gid)
407+
{
408+
return Err(format!(
409+
"sandbox_gid {gid} is outside the allowed range [{}, {}]",
410+
openshell_policy::MIN_SANDBOX_UID,
411+
openshell_policy::MAX_SANDBOX_UID,
412+
));
413+
}
414+
Ok(())
415+
}
311416
}
312417

313418
fn validate_provider_spiffe_workload_api_socket_path_value(
@@ -345,6 +450,7 @@ fn validate_provider_spiffe_workload_api_socket_path_value(
345450
#[cfg(test)]
346451
mod tests {
347452
use super::*;
453+
use std::collections::BTreeMap as HashMap;
348454

349455
#[test]
350456
fn default_workspace_storage_size_is_2gi() {
@@ -515,4 +621,173 @@ mod tests {
515621
let cfg: KubernetesComputeConfig = serde_json::from_value(json).unwrap();
516622
assert_eq!(cfg.image_pull_secrets, ["regcred", "backup-regcred"]);
517623
}
624+
625+
#[test]
626+
fn default_sandbox_uid_and_gid_are_none() {
627+
let cfg = KubernetesComputeConfig::default();
628+
assert_eq!(cfg.sandbox_uid, None);
629+
assert_eq!(cfg.sandbox_gid, None);
630+
}
631+
632+
#[test]
633+
fn serde_override_sandbox_uid() {
634+
let json = serde_json::json!({
635+
"sandbox_uid": 1500
636+
});
637+
let cfg: KubernetesComputeConfig = serde_json::from_value(json).unwrap();
638+
assert_eq!(cfg.sandbox_uid, Some(1500));
639+
}
640+
641+
#[test]
642+
fn serde_override_sandbox_gid() {
643+
let json = serde_json::json!({
644+
"sandbox_gid": 2000
645+
});
646+
let cfg: KubernetesComputeConfig = serde_json::from_value(json).unwrap();
647+
assert_eq!(cfg.sandbox_gid, Some(2000));
648+
}
649+
650+
#[test]
651+
fn parse_openshift_uid_range() {
652+
assert_eq!(
653+
KubernetesComputeConfig::from_open_shift_uid_range("1000000000/10000"),
654+
Some(1_000_000_000)
655+
);
656+
assert_eq!(
657+
KubernetesComputeConfig::from_open_shift_uid_range("1000/50000"),
658+
Some(1000)
659+
);
660+
}
661+
662+
#[test]
663+
fn parse_openshift_uid_range_rejects_below_min() {
664+
// 999 is below MIN_SANDBOX_UID (1000) — should be rejected.
665+
assert_eq!(
666+
KubernetesComputeConfig::from_open_shift_uid_range("999/50000"),
667+
None
668+
);
669+
}
670+
671+
#[test]
672+
fn parse_openshift_uid_range_rejects_above_max() {
673+
// u32::MAX is well above MAX_SANDBOX_UID — should be rejected.
674+
assert_eq!(
675+
KubernetesComputeConfig::from_open_shift_uid_range("4294967295/10000"),
676+
None
677+
);
678+
}
679+
680+
#[test]
681+
fn validate_sandbox_identity_config_accepts_valid_range() {
682+
let cfg = KubernetesComputeConfig {
683+
sandbox_uid: Some(1000),
684+
sandbox_gid: Some(1000),
685+
..KubernetesComputeConfig::default()
686+
};
687+
assert!(cfg.validate_sandbox_identity_config().is_ok());
688+
}
689+
690+
#[test]
691+
fn validate_sandbox_identity_config_rejects_uid_zero() {
692+
let cfg = KubernetesComputeConfig {
693+
sandbox_uid: Some(0),
694+
..KubernetesComputeConfig::default()
695+
};
696+
let err = cfg.validate_sandbox_identity_config().unwrap_err();
697+
assert!(err.contains("sandbox_uid"));
698+
}
699+
700+
#[test]
701+
fn validate_sandbox_identity_config_rejects_gid_above_max() {
702+
let cfg = KubernetesComputeConfig {
703+
sandbox_gid: Some(openshell_policy::MAX_SANDBOX_UID + 1),
704+
..KubernetesComputeConfig::default()
705+
};
706+
let err = cfg.validate_sandbox_identity_config().unwrap_err();
707+
assert!(err.contains("sandbox_gid"));
708+
}
709+
710+
#[test]
711+
fn validate_sandbox_identity_config_accepts_none_fields() {
712+
let cfg = KubernetesComputeConfig::default();
713+
assert!(cfg.validate_sandbox_identity_config().is_ok());
714+
}
715+
716+
#[test]
717+
fn parse_openshift_supplemental_groups() {
718+
assert_eq!(
719+
KubernetesComputeConfig::from_open_shift_supplemental_groups("1000/50000"),
720+
Some(1000)
721+
);
722+
}
723+
724+
#[test]
725+
fn resolve_sandbox_uid_prefers_config() {
726+
let cfg = KubernetesComputeConfig {
727+
sandbox_uid: Some(5000),
728+
..KubernetesComputeConfig::default()
729+
};
730+
// Config value should win even when annotations are present.
731+
let mut anns: HashMap<String, String> = HashMap::new();
732+
anns.insert(
733+
ANNOTATION_SCC_UID_RANGE.to_string(),
734+
"1000000000/10000".to_string(),
735+
);
736+
assert_eq!(cfg.resolve_sandbox_uid(Some(&anns)), 5000);
737+
}
738+
739+
#[test]
740+
fn resolve_sandbox_uid_falls_back_to_openshift_annotation() {
741+
let cfg = KubernetesComputeConfig::default();
742+
let mut anns: HashMap<String, String> = HashMap::new();
743+
anns.insert(
744+
ANNOTATION_SCC_UID_RANGE.to_string(),
745+
"1000000000/10000".to_string(),
746+
);
747+
assert_eq!(cfg.resolve_sandbox_uid(Some(&anns)), 1_000_000_000);
748+
}
749+
750+
#[test]
751+
fn resolve_sandbox_uid_falls_back_to_default() {
752+
let cfg = KubernetesComputeConfig::default();
753+
// No config, no annotations.
754+
assert_eq!(cfg.resolve_sandbox_uid(None), DEFAULT_SANDBOX_UID);
755+
// Empty annotations map.
756+
let anns: HashMap<String, String> = HashMap::new();
757+
assert_eq!(cfg.resolve_sandbox_uid(Some(&anns)), DEFAULT_SANDBOX_UID);
758+
}
759+
760+
#[test]
761+
fn resolve_sandbox_gid_prefers_config() {
762+
let cfg = KubernetesComputeConfig {
763+
sandbox_uid: Some(5000),
764+
sandbox_gid: Some(6000),
765+
..KubernetesComputeConfig::default()
766+
};
767+
assert_eq!(
768+
cfg.resolve_sandbox_gid(cfg.resolve_sandbox_uid(None), None),
769+
6000
770+
);
771+
}
772+
773+
#[test]
774+
fn resolve_sandbox_gid_falls_back_to_uid() {
775+
let cfg = KubernetesComputeConfig {
776+
sandbox_uid: Some(5000),
777+
..KubernetesComputeConfig::default()
778+
};
779+
// sandbox_gid is None, should fall back to sandbox_uid.
780+
assert_eq!(
781+
cfg.resolve_sandbox_gid(cfg.resolve_sandbox_uid(None), None),
782+
5000
783+
);
784+
}
785+
786+
#[test]
787+
fn resolve_sandbox_gid_falls_back_to_resolved_uid() {
788+
let cfg = KubernetesComputeConfig::default();
789+
// Both are None, should use the resolved UID.
790+
let uid = cfg.resolve_sandbox_uid(None);
791+
assert_eq!(cfg.resolve_sandbox_gid(uid, None), uid);
792+
}
518793
}

0 commit comments

Comments
 (0)