Skip to content

buildkite/cleanroom

BranchesTags

Repository files navigation

👩‍🔬 Cleanroom

Cleanroom runs untrusted code in microVMs with deny-by-default network policy. It is self-hosted, enforces repository-scoped egress rules, and keeps credentials on the host side of the VM boundary.

Agent sandboxing tools are proliferating fast. Most focus on isolation alone. Cleanroom adds policy-controlled network access so you decide exactly what the sandbox can reach.

Why Cleanroom?

Deny-by-default egress. A cleanroom.yaml policy file in your repo controls exactly which hosts the sandbox can reach. Everything else is blocked.

MicroVM isolation. Each sandbox is a hardware-virtualized microVM (Firecracker on Linux, Virtualization.framework on macOS), not a container. A VM boundary is stronger than namespaces, seccomp, or gVisor -- a kernel vulnerability in the guest doesn't compromise the host.

Self-hosted. Runs on your infrastructure. Your code and data never leave your machines.

Credentials stay on the host. A host-side gateway proxies git clones and package fetches, injecting credentials on the upstream leg. Tokens never enter the sandbox.

Standard OCI images. Use any OCI image from any registry as your sandbox base. Digest-pinned in policy for reproducibility. No custom VM image format or vendor-specific base images. Same image works across backends.

Docker inside the sandbox. Enable a guest Docker daemon with a policy flag (services.docker.required: true) or explicitly for a repo-agnostic sandbox with cleanroom sandbox create --docker. Build and run containers inside the microVM.

Coming soon: package registry proxy with lockfile enforcement, Docker pull caching, content caching for hermetic offline builds, and structured audit logging. See the spec for the full roadmap.

Install

Install the latest release:

curl -fsSL https://raw.githubusercontent.com/buildkite/cleanroom/main/scripts/install.sh | bash

Install a specific version:

curl -fsSL https://raw.githubusercontent.com/buildkite/cleanroom/main/scripts/install.sh | \
  bash -s -- --version vX.Y.Z

By default this installs to /usr/local/bin. Override with --install-dir or CLEANROOM_INSTALL_DIR.

Install the locally built binaries from this checkout into /usr/local/bin:

mise run install:global

Quick start

Initialize runtime config and check host prerequisites:

cleanroom config init
cleanroom doctor

Start the server (all CLI commands need a running server):

cleanroom serve &

The server listens on unix://$XDG_RUNTIME_DIR/cleanroom/cleanroom.sock by default.

Install as a daemon:

# macOS: installs a user LaunchAgent (user-scope only)
cleanroom daemon install

# Linux (systemd)
sudo cleanroom daemon install

Use --force to overwrite an existing service file. On macOS, --system is unsupported; --user is accepted for explicitness.

Manage the daemon lifecycle:

cleanroom daemon status
cleanroom daemon start
cleanroom daemon stop
cleanroom daemon uninstall

The system daemon socket is root-owned (unix:///var/run/cleanroom/cleanroom.sock), so client commands against that daemon should be run with sudo unless you configure an alternate endpoint. User-scope daemons listen on the runtime socket (unix://$XDG_RUNTIME_DIR/cleanroom/cleanroom.sock when XDG_RUNTIME_DIR is set).

Run a command in a sandbox:

cleanroom exec -- npm test
cleanroom exec -e OPENAI_API_KEY -- codex app-server

When cleanroom.yaml includes a repository bootstrap block, the top-level commands become repo-aware: Cleanroom resolves the current git remote and local HEAD, materializes that checkout in the sandbox, and starts commands in the configured guest path.

Pre-create a long-running sandbox using repo policy:

SANDBOX_ID="$(cleanroom create)"
cleanroom exec --in "$SANDBOX_ID" -- npm run lint

Override the sandbox image per command (remote tag/digest or local Docker image name):

cleanroom sandbox create --image ghcr.io/buildkite/cleanroom-base/alpine:latest
cleanroom exec --image ghcr.io/buildkite/cleanroom-base/alpine:latest -- npm test
cleanroom console --image my-local-image:dev -- sh
cleanroom exec -e OPENAI_API_KEY -e CODEX_HOME=/workspace/.codex -- codex app-server

Pre-create a repo-agnostic sandbox without reading cleanroom.yaml:

cleanroom sandbox create

cleanroom sandbox create stays generic. It does not inspect the local git repository or read cleanroom.yaml. It creates a sandbox with a built-in deny-by-default policy and resolves ghcr.io/buildkite/cleanroom-base/alpine:latest to a digest unless --image is provided.

To start the guest Docker service for a repo-agnostic sandbox, pass --docker.

To disable egress filtering for a repo-agnostic sandbox, pass --dangerously-allow-all.

cleanroom exec and cleanroom console create ephemeral sandboxes by default. Reuse an existing sandbox with --in, or keep a newly created sandbox with --keep.

List sandboxes and run more commands:

cleanroom sandbox ls
cleanroom exec --in <id> -- npm run lint
cleanroom exec --in <id> -- npm run build

Keep a sandbox created by exec:

cleanroom exec --keep -- npm test

Run against a snapshot:

cleanroom exec --from snap_... -- npm test
cleanroom console --from snap_...

Interactive console:

cleanroom console -- bash

Policy file

A cleanroom.yaml in your repo defines the sandbox policy for policy-aware commands such as cleanroom create, cleanroom exec, and cleanroom console. Cleanroom also checks .buildkite/cleanroom.yaml as a fallback. cleanroom sandbox create does not read either path.

version: 1
sandbox:
  image:
    ref: ghcr.io/buildkite/cleanroom-base/alpine@sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
  network:
    default: deny
    allow:
      - host: api.github.com
        ports: [443]
      - host: registry.npmjs.org
        ports: [443]

Enable Docker as a guest service:

sandbox:
  services:
    docker:
      required: true

Validate policy without running anything:

cleanroom policy validate

Repository-aware bootstrap is the default for the top-level commands when you run them from inside a git repository.

The implicit defaults are:

repository:
  remote: origin
  path: /workspace
  submodules: false

Use the optional repository block only to override those defaults or disable the behavior:

repository:
  enabled: false

or:

repository:
  path: /work
  submodules: true

With the default behavior:

  • cleanroom create reads repo policy and creates a sandbox with the current repo checked out at local HEAD
  • cleanroom exec -- <cmd> checks out the repo, runs <cmd> from /workspace, and tears the sandbox down unless --keep is set
  • cleanroom console -- bash opens a shell in /workspace and tears the sandbox down unless --keep is set
  • dirty working trees print a warning and use committed HEAD; uncommitted changes are not copied in
  • cleanroom sandbox create remains explicit and repo-agnostic, using a built-in policy instead of repo policy (default image, deny-by-default networking unless --dangerously-allow-all is set, optional guest Docker via --docker)

Repository bootstrap needs the remote host in sandbox.network.allow, for example:

sandbox:
  network:
    default: deny
    allow:
      - host: github.com
        ports: [443]

Backend support

Host OS Backend Status Notes
Linux firecracker Full support Persistent sandboxes, per-sandbox TAP + guest IP identity, file download, egress allowlist enforcement
macOS darwin-vz Supported with gaps Persistent sandboxes, vmnet-shared by default on macOS 26+, experimental custom subnet host reachability, no file download, no egress filtering, no TAP parity

Backend capabilities are exposed in cleanroom doctor --json under capabilities. See isolation model for enforcement and persistence details.

Network model differs significantly by backend:

  • firecracker creates a dedicated TAP interface and host/guest IP pair per sandbox, which enables host-side identity and firewall enforcement.
  • darwin-vz now defaults to vmnet-shared on supported macOS 26+ hosts. It is still NAT-backed for outbound access, but the helper owns an explicit vmnet logical network and can provide guest IP metadata.
  • darwin-vz still does not expose Firecracker-style TAP devices or host firewall enforcement semantics. Explicit nat remains available as a fallback mode.
  • Current vmnet work and remaining gaps are tracked in docs/plans/darwin-vz-vmnet-mode.md.

Select a backend explicitly:

cleanroom exec --backend firecracker -- npm test
cleanroom exec --backend darwin-vz -- npm test

Architecture

  • Server: cleanroom serve (required for all operations)
  • Client: CLI and ConnectRPC clients
  • Transport: unix socket (default), HTTPS with mTLS, or Tailscale
  • RPC services: cleanroom.v1.SandboxService, cleanroom.v1.ExecutionService (API design)

Go Client (Public API)

Use github.com/buildkite/cleanroom/client from external Go modules.

import (
  "context"
  "os"

  "github.com/buildkite/cleanroom/client"
)

func example() error {
  c := client.Must(client.NewFromEnv())

  sb, err := c.EnsureSandbox(context.Background(), "thread:abc123", client.EnsureSandboxOptions{
    Backend: "firecracker",
    Policy: client.PolicyFromAllowlist(
      "ghcr.io/buildkite/cleanroom-base/alpine@sha256:...",
      "sha256:...",
      client.Allow("api.github.com", 443),
      client.Allow("registry.npmjs.org", 443),
    ),
  })
  if err != nil { return err }

  result, err := c.ExecAndWait(context.Background(), sb.ID, []string{"bash", "-lc", "echo hello"}, client.ExecOptions{
    Stdout: os.Stdout,
    Stderr: os.Stderr,
  })
  if err != nil { return err }
  _ = result
  return nil
}

client exposes:

  • client.Client for RPC calls
  • protobuf request/response/event types (for example client.CreateExecutionRequest)
  • status enums (client.SandboxStatus_*, client.ExecutionStatus_*)
  • ergonomic wrappers (client.NewFromEnv, client.EnsureSandbox, client.ExecAndWait)

Images

Cleanroom uses digest-pinned OCI images as sandbox bases. Images are pulled from any OCI registry and materialized into ext4 rootfs files for the VM backend.

cleanroom image pull ghcr.io/buildkite/cleanroom-base/alpine@sha256:...
cleanroom image ls
cleanroom image rm sha256:...
cleanroom image import ghcr.io/buildkite/cleanroom-base/alpine@sha256:... ./rootfs.tar.gz
cleanroom image bump-ref    # resolve :latest tag to digest and update cleanroom.yaml

ghcr.io/buildkite/cleanroom-base/alpine, ghcr.io/buildkite/cleanroom-base/alpine-docker, and ghcr.io/buildkite/cleanroom-base/alpine-agents are published from this repo on pushes to main.

Build these locally with mise:

mise run build:images
# or individually:
mise run build:image:alpine
mise run build:image:alpine-docker
mise run build:image:alpine-agents

Runtime config

Config path: $XDG_CONFIG_HOME/cleanroom/config.yaml (typically ~/.config/cleanroom/config.yaml).

cleanroom config init

On macOS this defaults default_backend to darwin-vz. On Linux it defaults to firecracker. If default_backend is omitted or blank in an existing config, Cleanroom falls back to the same host default at load time.

Optional endpoint override precedence is --host, then CLEANROOM_HOST, then control_host from runtime config, then defaults (macOS: user runtime socket; Linux: system socket when present, otherwise user runtime socket).

default_backend: firecracker
control_host: ""             # optional override for client endpoint resolution
backends:
  firecracker:
    binary_path: firecracker
    kernel_image: ""    # auto-managed when unset
    privileged_helper_path: /usr/local/sbin/cleanroom-root-helper
    vcpus: 2
    memory_mib: 1024
    launch_seconds: 30
  darwin-vz:
    kernel_image: ""    # auto-managed when unset
    rootfs: ""          # derived from sandbox.image.ref when unset
    network:
      mode: nat         # optional fallback; default is vmnet-shared on macOS 26+
    vcpus: 2
    memory_mib: 1024
    launch_seconds: 30

When kernel_image is unset, Cleanroom auto-downloads a managed kernel. Set it explicitly for offline operation.

When rootfs is unset, Cleanroom derives one from sandbox.image.ref and injects the guest runtime. This requires mkfs.ext4 and debugfs on the host (macOS: brew install e2fsprogs).

Host requirements

Linux (firecracker):

  • /dev/kvm available and writable
  • Firecracker binary installed
  • mkfs.ext4 for OCI-to-ext4 materialization
  • debugfs for runtime rootfs preparation
  • sudo -n access to /usr/local/sbin/cleanroom-root-helper for host networking

macOS (darwin-vz):

  • cleanroom-darwin-vz helper signed with com.apple.security.virtualization entitlement
  • vmnet-shared additionally needs com.apple.developer.networking.vmnet and a matching provisioning profile
  • explicit nat remains available as a compatibility fallback
  • mkfs.ext4 and debugfs (brew install e2fsprogs)

Diagnostics

cleanroom doctor              # check host prerequisites
cleanroom doctor --json       # machine-readable with capabilities map
cleanroom sandbox inspect <sandbox-id>
cleanroom execution inspect --sandbox-id <sandbox-id> --last
cleanroom execution inspect --sandbox-id <sandbox-id> <execution-id>
cleanroom status --last       # browse the newest retained execution artifacts
cleanroom status --execution-id <execution-id>
cleanroom version

Failure flow:

  • cleanroom exec and cleanroom console print sandbox_id and execution_id on failure when available.
  • cleanroom sandbox inspect <sandbox-id> shows sandbox state plus last_execution_id and active_execution_id.
  • cleanroom execution inspect ... is the control-plane view for execution status, retained stdout/stderr, image metadata, and observability.
  • cleanroom status ... is the local artifact view under $XDG_STATE_HOME/cleanroom/executions.

Further reading

About

Cleanroom sandbox orchestration system

Resources

Readme
Activity
Custom properties

Stars

12 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases 4

v0.3.0 Latest
Mar 13, 2026
+ 3 releases

Packages

 
 
 

Contributors

Languages