fix(auth): harden sandbox gateway authentication

Address PR review feedback on the per-sandbox authentication changes.

Remove the implicit permissive user fallback once sandbox or user auth is configured. Missing credentials now fail closed unless an explicit local mode is selected. Keep mTLS user auth as a local single-player option for Docker, Podman, and VM gateways, reject it for Kubernetes, and add an explicit unsafe unauthenticated-user switch for trusted local Skaffold/Kubernetes development.

Deliver sandbox JWTs through driver-owned token files for Docker, Podman, and VM sandboxes instead of placing raw bearers in container or guest environment metadata. Strip token env overrides from user-provided sandbox environments and update debug-rpc helpers to print token fingerprints, expiry, and claims rather than raw bearer values.

Make certgen upgrades recover existing TLS-only installs by creating just the missing gateway JWT signing material while preserving existing TLS certificates and keys. Keep partial-state failures for inconsistent TLS or JWT sets.

Improve supervisor token refresh behavior for short JWT TTLs by removing the 60-second refresh floor, using shorter retry backoff, and re-running the Kubernetes ServiceAccount bootstrap path after unauthenticated refresh failures.

Update Helm defaults, Skaffold values, e2e gateway setup, Python gateway metadata handling, architecture notes, published docs, and generated chart docs to describe the new auth modes and local development behavior.

Validation: mise run pre-commit; Docker smoke e2e; Podman smoke e2e; Kubernetes smoke e2e.

Author: TaylorMutch

architecture/gateway.md

@@ -23,8 +23,12 @@ identity.
 ## Protocol and Auth
 
 The gateway listens on one service port and multiplexes gRPC and HTTP traffic.
-The default deployment mode is mTLS: clients and sandbox workloads present a
-certificate signed by the deployment CA before reaching application handlers.
+The default local single-user deployment mode is mTLS user authentication:
+clients present a certificate signed by the local deployment CA, and the
+gateway maps the verified certificate subject to a user principal. Kubernetes
+deployments use mTLS for transport only and require OIDC or a trusted access
+proxy for user authentication unless the explicit unsafe local-development
+`allow_unauthenticated_users` switch is enabled.
 When that service port is bound to loopback, the listener can also accept
 plaintext HTTP on the same port for sandbox service subdomains only. That local
 browser path is enabled by default and disabled with
@@ -37,14 +41,15 @@ Supported auth modes:
 
 | Mode | Use |
 |---|---|
-| mTLS | Default direct gateway access for CLI, SDK, TUI, and sandbox callbacks. |
+| mTLS user auth | Local single-user Docker, Podman, and VM gateway access. |
 | Plaintext | Local development or a trusted reverse proxy boundary. |
+| Unauthenticated local users | Trusted Kubernetes dev or fully trusted proxy deployments only. |
 | Cloudflare JWT | Edge-authenticated deployments where Cloudflare Access supplies identity. |
 | OIDC | Bearer-token auth for users, with browser PKCE or client credentials login. |
 
-Sandbox supervisor RPCs authenticate with either mTLS material or a sandbox
-secret depending on the runtime and deployment mode. User-facing mutations are
-authorized by role policy when OIDC or edge identity is enabled.
+Sandbox supervisor RPCs authenticate with gateway-minted sandbox JWTs when that
+authenticator is configured; mTLS does not grant sandbox identity. User-facing
+mutations are authorized by role policy when OIDC or edge identity is enabled.
 
 Sandbox secrets are gateway-signed JWTs bound to a single sandbox ID. Docker,
 Podman, and VM drivers deliver the initial token through supervisor-only

crates/openshell-bootstrap/src/metadata.rs

@@ -25,7 +25,7 @@ pub struct GatewayMetadata {
     #[serde(skip_serializing_if = "Option::is_none", default)]
     pub resolved_host: Option<String>,
 
-    /// Auth mode: `None` or `"mtls"` = mTLS (default), `"plaintext"` = direct HTTP,
+    /// Auth mode: `None` or `"mtls"` = mTLS, `"plaintext"` = direct HTTP,
     /// `"cloudflare_jwt"` = CF JWT.
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub auth_mode: Option<String>,

crates/openshell-cli/src/main.rs

@@ -122,9 +122,9 @@ fn resolve_gateway_name(gateway_flag: &Option<String>) -> Option<String> {
 
 /// Apply authentication token from local storage based on gateway auth mode.
 ///
-/// Handles both Cloudflare Access (`edge_token`) and OIDC (`oidc_token`)
-/// auth modes by loading the stored token and setting it on `TlsOptions`.
-/// For OIDC, automatically refreshes the token if it's near expiry.
+/// Handles Cloudflare Access and OIDC auth modes by loading the stored token
+/// and setting it on `TlsOptions`. For OIDC, automatically refreshes the token
+/// if it's near expiry.
 fn apply_auth(tls: &mut TlsOptions, gateway_name: &str) {
     let Some(meta) = get_gateway_metadata(gateway_name) else {
         return;

crates/openshell-cli/src/tls.rs

@@ -100,7 +100,7 @@ impl TlsOptions {
         }
     }
 
-    /// Returns `true` when using bearer token auth (edge or OIDC).
+    /// Returns `true` when using bearer token auth.
     pub fn is_bearer_auth(&self) -> bool {
         self.edge_token.is_some() || self.oidc_token.is_some()
     }
@@ -397,9 +397,9 @@ pub async fn build_channel(server: &str, tls: &TlsOptions) -> Result<Channel> {
         .keep_alive_while_idle(true);
 
     let tls_config = if tls.oidc_token.is_some() {
-        // OIDC bearer auth over HTTPS: use mTLS certs for the transport layer
-        // when available (server may still require client certs), and layer
-        // the Bearer token on top via the interceptor.
+        // Bearer auth over HTTPS: use mTLS certs for the transport layer when
+        // available (server may still require client certs), and layer the
+        // Bearer token on top via the interceptor.
         require_tls_materials(server, tls).map_or_else(
             |_| {
                 let resolved = tls.with_default_paths(server);

crates/openshell-core/src/auth.rs

@@ -7,9 +7,9 @@ use miette::Result;
 
 /// Interceptor that injects authentication headers into every outgoing gRPC request.
 ///
-/// Supports OIDC Bearer tokens (standard `authorization` header) and
-/// Cloudflare Access tokens (custom headers). When no token is set, acts
-/// as a no-op. OIDC takes precedence over edge tokens.
+/// Supports application-layer Bearer tokens (standard `authorization`
+/// header) and Cloudflare Access tokens (custom headers). When no token is
+/// set, acts as a no-op. OIDC takes precedence over edge tokens.
 #[derive(Clone)]
 #[allow(clippy::struct_field_names)]
 pub struct EdgeAuthInterceptor {
@@ -21,14 +21,14 @@ pub struct EdgeAuthInterceptor {
 impl EdgeAuthInterceptor {
     /// Create an interceptor from optional token strings.
     ///
-    /// OIDC bearer token takes precedence over edge token. Returns a no-op
-    /// interceptor when neither token is provided.
+    /// OIDC bearer tokens take precedence over edge tokens. Returns a no-op
+    /// interceptor when no token is provided.
     pub fn new(oidc_token: Option<&str>, edge_token: Option<&str>) -> Result<Self> {
         if let Some(token) = oidc_token {
             let bearer: tonic::metadata::MetadataValue<tonic::metadata::Ascii> =
                 format!("Bearer {token}")
                     .parse()
-                    .map_err(|_| miette::miette!("invalid OIDC token value"))?;
+                    .map_err(|_| miette::miette!("invalid bearer token value"))?;
             return Ok(Self {
                 bearer_value: Some(bearer),
                 header_value: None,

crates/openshell-core/src/config.rs

@@ -205,6 +205,17 @@ pub struct Config {
     #[serde(default)]
     pub oidc: Option<OidcConfig>,
 
+    /// Gateway user authentication behavior.
+    #[serde(default)]
+    pub auth: GatewayAuthConfig,
+
+    /// mTLS user authentication configuration. When enabled, a verified TLS
+    /// client certificate can authenticate CLI/SDK callers as a
+    /// `Principal::User`. This is for local single-user gateways only;
+    /// sandbox identity is always carried by gateway-minted sandbox JWTs.
+    #[serde(default)]
+    pub mtls_auth: MtlsAuthConfig,
+
     /// Gateway-minted sandbox JWT configuration. When `Some`, the gateway
     /// loads the signing key from disk and accepts gateway-issued sandbox
     /// JWTs as `Principal::Sandbox`. Required for the per-sandbox identity
@@ -320,6 +331,27 @@ pub struct OidcConfig {
     pub scopes_claim: String,
 }
 
+/// mTLS user authentication for local, single-user gateways.
+#[derive(Debug, Clone, Default, Serialize, Deserialize)]
+pub struct MtlsAuthConfig {
+    /// When true, the gateway maps a verified TLS client certificate into a
+    /// user principal. Keep disabled for Kubernetes deployments because
+    /// Kubernetes sandbox pods and external users must not share user auth.
+    #[serde(default)]
+    pub enabled: bool,
+}
+
+/// Gateway user authentication settings.
+#[derive(Debug, Clone, Default, Serialize, Deserialize)]
+pub struct GatewayAuthConfig {
+    /// When true, unauthenticated user/CLI calls are accepted as a local
+    /// developer principal. This is an unsafe local-development escape hatch
+    /// for trusted, non-shared gateways. Sandbox supervisor calls still use
+    /// gateway-minted sandbox JWTs.
+    #[serde(default)]
+    pub allow_unauthenticated_users: bool,
+}
+
 const fn default_jwks_ttl_secs() -> u64 {
     3600
 }
@@ -378,6 +410,8 @@ impl Config {
             log_level: default_log_level(),
             tls,
             oidc: None,
+            auth: GatewayAuthConfig::default(),
+            mtls_auth: MtlsAuthConfig::default(),
             gateway_jwt: None,
             database_url: String::new(),
             compute_drivers: vec![],
@@ -606,6 +640,12 @@ mod tests {
         assert!(cfg.health_bind_address.is_none());
     }
 
+    #[test]
+    fn config_disables_unauthenticated_users_by_default() {
+        let cfg = Config::new(None);
+        assert!(!cfg.auth.allow_unauthenticated_users);
+    }
+
     #[test]
     fn service_routing_allows_loopback_plaintext_http_by_default() {
         let cfg = Config::new(None);

crates/openshell-core/src/lib.rs

@@ -26,7 +26,10 @@ pub mod sandbox_env;
 pub mod settings;
 pub mod time;
 
-pub use config::{ComputeDriverKind, Config, GatewayJwtConfig, OidcConfig, TlsConfig};
+pub use config::{
+    ComputeDriverKind, Config, GatewayAuthConfig, GatewayJwtConfig, MtlsAuthConfig, OidcConfig,
+    TlsConfig,
+};
 pub use error::{ComputeDriverError, Error, Result};
 pub use metadata::{GetResourceVersion, ObjectId, ObjectLabels, ObjectName, SetResourceVersion};
 

crates/openshell-driver-docker/src/lib.rs

@@ -55,6 +55,7 @@ const SUPERVISOR_MOUNT_PATH: &str = "/opt/openshell/bin/openshell-sandbox";
 const TLS_CA_MOUNT_PATH: &str = "/etc/openshell/tls/client/ca.crt";
 const TLS_CERT_MOUNT_PATH: &str = "/etc/openshell/tls/client/tls.crt";
 const TLS_KEY_MOUNT_PATH: &str = "/etc/openshell/tls/client/tls.key";
+const SANDBOX_TOKEN_MOUNT_PATH: &str = "/etc/openshell/auth/sandbox.jwt";
 const SANDBOX_COMMAND: &str = "sleep infinity";
 const SUPERVISOR_PATH: &str = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin";
 const HOST_OPENSHELL_INTERNAL: &str = "host.openshell.internal";
@@ -417,6 +418,7 @@ impl DockerComputeDriver {
             .and_then(|spec| spec.template.as_ref())
             .expect("validated sandbox has template");
         self.ensure_image_available(&template.image).await?;
+        let token_file_created = write_sandbox_token_file(sandbox, &self.config).await?;
 
         let container_name = container_name_for_sandbox(sandbox);
         let create_body = build_container_create_body(sandbox, &self.config)?;
@@ -431,6 +433,9 @@ impl DockerComputeDriver {
             )
             .await
             .map_err(|err| {
+                if token_file_created {
+                    cleanup_sandbox_token_file(sandbox, &self.config);
+                }
                 create_status_from_docker_error("create docker sandbox container", err)
             })?;
 
@@ -450,6 +455,9 @@ impl DockerComputeDriver {
                     "Failed to clean up Docker container after start failure"
                 );
             }
+            if token_file_created {
+                cleanup_sandbox_token_file(sandbox, &self.config);
+            }
             return Err(create_status_from_docker_error(
                 "start docker sandbox container",
                 err,
@@ -482,8 +490,14 @@ impl DockerComputeDriver {
             )
             .await
         {
-            Ok(()) => Ok(true),
-            Err(err) if is_not_found_error(&err) => Ok(false),
+            Ok(()) => {
+                cleanup_sandbox_token_file_by_id(sandbox_id, &self.config);
+                Ok(true)
+            }
+            Err(err) if is_not_found_error(&err) => {
+                cleanup_sandbox_token_file_by_id(sandbox_id, &self.config);
+                Ok(false)
+            }
             Err(err) => Err(internal_status("delete docker sandbox container", err)),
         }
     }
@@ -905,7 +919,10 @@ impl ComputeDriver for DockerComputeDriver {
     }
 }
 
-fn build_binds(config: &DockerDriverRuntimeConfig) -> Vec<String> {
+fn build_binds(
+    sandbox: &DriverSandbox,
+    config: &DockerDriverRuntimeConfig,
+) -> Result<Vec<String>, Status> {
     let mut binds = vec![format!(
         "{}:{}:ro,z",
         config.supervisor_bin.display(),
@@ -920,7 +937,101 @@ fn build_binds(config: &DockerDriverRuntimeConfig) -> Vec<String> {
         ));
         binds.push(format!("{}:{}:ro,z", tls.key.display(), TLS_KEY_MOUNT_PATH));
     }
-    binds
+    if sandbox
+        .spec
+        .as_ref()
+        .is_some_and(|spec| !spec.sandbox_token.is_empty())
+    {
+        binds.push(format!(
+            "{}:{}:ro,z",
+            sandbox_token_host_path(sandbox, config)?.display(),
+            SANDBOX_TOKEN_MOUNT_PATH
+        ));
+    }
+    Ok(binds)
+}
+
+fn sandbox_token_host_path(
+    sandbox: &DriverSandbox,
+    config: &DockerDriverRuntimeConfig,
+) -> Result<PathBuf, Status> {
+    sandbox_token_host_path_by_id(&sandbox.id, config)
+}
+
+fn sandbox_token_host_path_by_id(
+    sandbox_id: &str,
+    config: &DockerDriverRuntimeConfig,
+) -> Result<PathBuf, Status> {
+    let base = openshell_core::paths::xdg_state_dir().map_err(|err| {
+        Status::internal(format!(
+            "resolve sandbox token state directory failed: {err}"
+        ))
+    })?;
+    Ok(base
+        .join("openshell")
+        .join("docker-sandbox-tokens")
+        .join(config.sandbox_namespace.replace(['/', '\\'], "-"))
+        .join(sandbox_id)
+        .join("sandbox.jwt"))
+}
+
+async fn write_sandbox_token_file(
+    sandbox: &DriverSandbox,
+    config: &DockerDriverRuntimeConfig,
+) -> Result<bool, Status> {
+    let Some(spec) = sandbox.spec.as_ref() else {
+        return Ok(false);
+    };
+    if spec.sandbox_token.is_empty() {
+        return Ok(false);
+    }
+    let path = sandbox_token_host_path(sandbox, config)?;
+    if let Some(parent) = path.parent() {
+        openshell_core::paths::create_dir_restricted(parent).map_err(|err| {
+            Status::internal(format!(
+                "create sandbox token directory {} failed: {err}",
+                parent.display()
+            ))
+        })?;
+    }
+    tokio::fs::write(&path, format!("{}\n", spec.sandbox_token))
+        .await
+        .map_err(|err| {
+            Status::internal(format!(
+                "write sandbox token file {} failed: {err}",
+                path.display()
+            ))
+        })?;
+    openshell_core::paths::set_file_owner_only(&path).map_err(|err| {
+        Status::internal(format!(
+            "restrict sandbox token file {} failed: {err}",
+            path.display()
+        ))
+    })?;
+    Ok(true)
+}
+
+fn cleanup_sandbox_token_file(sandbox: &DriverSandbox, config: &DockerDriverRuntimeConfig) {
+    cleanup_sandbox_token_file_by_id(&sandbox.id, config);
+}
+
+fn cleanup_sandbox_token_file_by_id(sandbox_id: &str, config: &DockerDriverRuntimeConfig) {
+    let Ok(path) = sandbox_token_host_path_by_id(sandbox_id, config) else {
+        return;
+    };
+    if let Err(err) = std::fs::remove_file(&path)
+        && err.kind() != std::io::ErrorKind::NotFound
+    {
+        warn!(
+            sandbox_id = %sandbox_id,
+            path = %path.display(),
+            error = %err,
+            "Failed to remove Docker sandbox token file"
+        );
+    }
+    if let Some(dir) = path.parent() {
+        let _ = std::fs::remove_dir(dir);
+    }
 }
 
 fn build_environment(sandbox: &DriverSandbox, config: &DockerDriverRuntimeConfig) -> Vec<String> {
@@ -979,16 +1090,17 @@ fn build_environment(sandbox: &DriverSandbox, config: &DockerDriverRuntimeConfig
         );
     }
 
-    // Gateway-minted sandbox JWT (PR 3 of the per-sandbox identity series).
-    // Passed via env var since Docker has no native secret mount that is
-    // simpler than the existing bind-mount pattern; the trust boundary
-    // (`docker inspect` access) is already equivalent to the TLS key mount.
+    environment.remove(openshell_core::sandbox_env::SANDBOX_TOKEN);
+    environment.remove(openshell_core::sandbox_env::SANDBOX_TOKEN_FILE);
+
+    // Gateway-minted sandbox JWT. Keep the raw bearer out of container
+    // metadata; the supervisor reads it from this driver-owned bind mount.
     if let Some(spec) = sandbox.spec.as_ref()
         && !spec.sandbox_token.is_empty()
     {
         environment.insert(
-            openshell_core::sandbox_env::SANDBOX_TOKEN.to_string(),
-            spec.sandbox_token.clone(),
+            openshell_core::sandbox_env::SANDBOX_TOKEN_FILE.to_string(),
+            SANDBOX_TOKEN_MOUNT_PATH.to_string(),
         );
     }
 
@@ -1052,7 +1164,7 @@ fn build_container_create_body(
             nano_cpus: resource_limits.nano_cpus,
             memory: resource_limits.memory_bytes,
             device_requests: docker_gpu_device_requests(spec.gpu, &spec.gpu_device),
-            binds: Some(build_binds(config)),
+            binds: Some(build_binds(sandbox, config)?),
             restart_policy: Some(RestartPolicy {
                 name: Some(RestartPolicyNameEnum::UNLESS_STOPPED),
                 maximum_retry_count: None,