Skip to content

Commit 9b6d904

Browse files
authored
feat(compute): delegate sandbox authentication to drivers (#2968)
* feat(compute): delegate sandbox authentication to drivers Signed-off-by: Drew Newberry <385+drew@users.noreply.github.com> * fix(kubernetes): align sandbox identity annotation Signed-off-by: Drew Newberry <385+drew@users.noreply.github.com> * test(auth): restore sandbox bootstrap coverage Signed-off-by: Drew Newberry <385+drew@users.noreply.github.com> * fix(kubernetes): satisfy ownership test lint Signed-off-by: Drew Newberry <385+drew@users.noreply.github.com> --------- Signed-off-by: Drew Newberry <385+drew@users.noreply.github.com>
1 parent c27a3a3 commit 9b6d904

30 files changed

Lines changed: 917 additions & 1357 deletions

File tree

.agents/skills/debug-openshell-cluster/SKILL.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -392,7 +392,7 @@ kubectl -n openshell get pod -l app.kubernetes.io/name=helm-chart -o jsonpath="{
392392
```
393393

394394
Sandbox pods using provider token grants should have an
395-
`openshell.io/sandbox-id` annotation, an `openshell.ai/managed-by=openshell`
395+
`openshell.ai/sandbox-id` annotation, an `openshell.ai/managed-by=openshell`
396396
label, supervisor env vars `OPENSHELL_K8S_SA_TOKEN_FILE` and
397397
`OPENSHELL_PROVIDER_SPIFFE_WORKLOAD_API_SOCKET`, plus both the projected
398398
`openshell-sa-token` volume and the `spiffe-workload-api` CSI volume.
@@ -467,7 +467,10 @@ Then inspect sandbox resources in that namespace.
467467
Check the configured sandbox service account when TokenReview bootstrap or
468468
sandbox registration fails. Helm creates a dedicated sandbox service account by
469469
default and writes it to `[openshell.drivers.kubernetes].service_account_name`;
470-
the gateway rejects projected tokens from other service accounts.
470+
the selected Kubernetes compute driver rejects projected tokens from other
471+
service accounts. For an external driver, inspect its logs and confirm it
472+
advertises `supports_sandbox_authentication`; the gateway delegates the opaque
473+
credential over the driver socket and never interprets Kubernetes settings.
471474

472475
```bash
473476
helm -n openshell get values openshell | grep -A3 sandboxServiceAccount

.agents/skills/helm-dev-environment/SKILL.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -259,7 +259,9 @@ annotations to `spiffe://openshell.local/openshell/sandbox/<sandbox-id>`.
259259
OpenShell mounts the SPIFFE CSI Workload API socket at
260260
`/spiffe-workload-api/spire-agent.sock` into sandbox pods for provider token
261261
grants. Supervisor-to-gateway authentication remains on the Kubernetes
262-
ServiceAccount bootstrap and gateway-minted sandbox JWT path.
262+
ServiceAccount bootstrap and gateway-minted sandbox JWT path; the selected
263+
Kubernetes compute driver validates the projected token before the gateway
264+
mints its JWT.
263265

264266
---
265267

architecture/compute-runtimes.md

Lines changed: 19 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -177,6 +177,15 @@ The driver reports this behavior through
177177
in-process and external drivers. Older drivers omit the field and retain the
178178
conservative operator-managed behavior.
179179

180+
Drivers that can verify a platform-native sandbox credential advertise
181+
`GetCapabilities.supports_sandbox_authentication`. On the path-scoped
182+
`IssueSandboxToken` exchange, the gateway forwards the opaque bearer credential
183+
to that selected driver through `AuthenticateSandbox`. The driver returns only
184+
the authenticated sandbox ID. The gateway then verifies that its durable
185+
sandbox record exists and mints the gateway JWT. The driver socket is therefore
186+
a sandbox-identity trust boundary, but it does not grant user or administrator
187+
authority.
188+
180189
## Deletion Lifecycle
181190

182191
Lifecycle requests use per-sandbox gates to serialize stop, start, and
@@ -465,20 +474,23 @@ watcher emits only sandbox CR changes, not platform events.
465474

466475
### SA Token Authentication
467476

468-
The gateway's `K8sServiceAccountAuthenticator` adapts its `NamespaceValidator`
469-
per mode (`crates/openshell-server/src/auth/k8s_sa.rs`):
477+
The Kubernetes driver's `AuthenticateSandbox` implementation applies its named
478+
`[openshell.drivers.kubernetes]` configuration per mode:
470479

471480
- **Shared:** `Exact` — accepts only the single configured namespace.
472481
- **Managed:** `Prefix` — accepts any namespace starting with `openshell-{gateway_id}-`.
473482
- **Operator:** `Allowlist` — accepts namespaces present in the dynamic
474483
`BTreeSet` populated by the label/file watchers. Starts empty (fail-closed)
475484
until the first watcher update.
476485

477-
These checks rely on an ownership invariant. In shared and managed modes, the
478-
gateway and its trusted Agent Sandbox controller exclusively administer the
479-
sandbox namespace, Sandbox CRs, sandbox pods, and configured sandbox
480-
ServiceAccount. Other principals must not create or mutate those resources or
481-
use that ServiceAccount. In operator mode, the platform operator retains
486+
It validates the projected token with Kubernetes `TokenReview`, checks the live
487+
pod UID, and verifies the pod's controlling Sandbox CR UID and sandbox ID before
488+
returning the identity to the gateway. These checks rely on an ownership
489+
invariant. In shared and managed modes, the Kubernetes driver and its trusted
490+
Agent Sandbox controller exclusively administer the sandbox namespace, Sandbox
491+
CRs, sandbox pods, and configured sandbox ServiceAccount. Other principals must
492+
not create or mutate those resources or use that ServiceAccount. In operator
493+
mode, the platform operator retains
482494
namespace lifecycle ownership, but must preserve the same exclusive control of
483495
Sandbox CRs and the pods and ServiceAccount used for sandbox token bootstrap.
484496
An allowlisted namespace is therefore a trust grant, not a tenant isolation

architecture/gateway.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -235,11 +235,12 @@ identity inspection without client-side token decoding.
235235
Sandbox secrets are gateway-signed JWTs bound to a single sandbox ID. Docker,
236236
Podman, and VM drivers deliver the initial token through supervisor-only
237237
runtime material; Kubernetes supervisors exchange a projected ServiceAccount
238-
token through `IssueSandboxToken`. The gateway validates that projected token
239-
with Kubernetes `TokenReview`, requires the configured sandbox service account,
240-
checks the returned pod binding against the live pod UID, and verifies the pod's
241-
controlling `Sandbox` ownerReference against the live Sandbox CR UID and
242-
sandbox-id label before minting the gateway JWT. The bootstrap path accepts
238+
token through `IssueSandboxToken`. The gateway delegates that opaque credential
239+
to the selected compute driver's `AuthenticateSandbox` RPC. A capable driver is
240+
trusted to return the authenticated sandbox ID, while the gateway still requires
241+
a matching durable sandbox record before minting a JWT. The Kubernetes driver
242+
uses its own named configuration to run TokenReview and verify the live pod and
243+
controlling Sandbox CR. The bootstrap path accepts
243244
both `agents.x-k8s.io/v1beta1` ownerReferences from newer Agent Sandbox
244245
controllers and `agents.x-k8s.io/v1alpha1` ownerReferences from existing
245246
deployments. Supervisors renew gateway JWTs in memory before expiry only while

crates/openshell-driver-docker/src/lib.rs

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -635,6 +635,7 @@ impl DockerComputeDriver {
635635
driver_version: self.config.daemon_version.clone(),
636636
default_image: self.config.default_image.clone(),
637637
gateway_manages_lifecycle: true,
638+
supports_sandbox_authentication: false,
638639
}
639640
}
640641

@@ -1740,6 +1741,19 @@ impl DockerComputeDriver {
17401741
impl ComputeDriver for ComputeDriverService {
17411742
type WatchSandboxesStream = WatchStream;
17421743

1744+
async fn authenticate_sandbox(
1745+
&self,
1746+
request: Request<openshell_core::proto::compute::v1::AuthenticateSandboxRequest>,
1747+
) -> Result<Response<openshell_core::proto::compute::v1::AuthenticateSandboxResponse>, Status>
1748+
{
1749+
self.trace_rpc(
1750+
"driver.authenticate_sandbox",
1751+
"authenticate_sandbox",
1752+
ComputeDriver::authenticate_sandbox(&self.driver, request),
1753+
)
1754+
.await
1755+
}
1756+
17431757
async fn get_capabilities(
17441758
&self,
17451759
request: Request<GetCapabilitiesRequest>,
@@ -1900,6 +1914,16 @@ impl ComputeDriver for ComputeDriverService {
19001914

19011915
#[tonic::async_trait]
19021916
impl ComputeDriver for DockerComputeDriver {
1917+
async fn authenticate_sandbox(
1918+
&self,
1919+
_request: Request<openshell_core::proto::compute::v1::AuthenticateSandboxRequest>,
1920+
) -> Result<Response<openshell_core::proto::compute::v1::AuthenticateSandboxResponse>, Status>
1921+
{
1922+
Err(Status::unimplemented(
1923+
"docker does not authenticate sandbox credentials",
1924+
))
1925+
}
1926+
19031927
type WatchSandboxesStream = WatchStream;
19041928

19051929
async fn get_capabilities(

crates/openshell-driver-kubernetes/README.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ annotation. An OTLP-enabled Agent Sandbox controller can therefore attach its
3333
asynchronous reconciliation spans to the originating OpenShell create trace.
3434

3535
Workspace namespace modes assume exclusive control of the sandbox identity
36-
resource chain. In shared and managed modes, only the gateway and its trusted
36+
resource chain. In shared and managed modes, only the driver and its trusted
3737
Agent Sandbox controller may administer the sandbox namespace, Sandbox CRs,
3838
sandbox pods, or configured sandbox ServiceAccount. In operator mode, the
3939
platform operator owns namespace lifecycle but must prevent other principals
@@ -103,7 +103,9 @@ Sandbox pods run as `service_account_name` and keep
103103
`automountServiceAccountToken: false`. The only Kubernetes token exposed to the
104104
supervisor is an explicit, audience-bound projected token mounted at
105105
`/var/run/secrets/openshell/token` for the one-shot `IssueSandboxToken`
106-
bootstrap exchange.
106+
bootstrap exchange. The Kubernetes driver authenticates that token through the
107+
compute-driver protocol using its own `service_account_name` and workspace-mode
108+
namespace policy; the gateway receives only the verified sandbox ID.
107109

108110
The gateway uses the supervisor relay for connect, exec, and file sync. Sandbox
109111
pods do not need direct external ingress for SSH.

crates/openshell-driver-kubernetes/src/config.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -303,7 +303,7 @@ pub struct KubernetesComputeConfig {
303303
#[serde(default, skip_serializing_if = "Option::is_none")]
304304
pub operator_namespace_file: Option<String>,
305305
/// Kubernetes `ServiceAccount` assigned to sandbox pods and accepted by
306-
/// the gateway's `TokenReview` bootstrap authenticator.
306+
/// the driver's `TokenReview` bootstrap authenticator.
307307
pub service_account_name: String,
308308
pub default_image: String,
309309
pub image_pull_policy: String,

0 commit comments

Comments
 (0)