kraze

Bring the simplicity of docker-compose to Kubernetes local development

Description

kraze is a Kubernetes development environment manager that brings the familiar docker-compose developer experience to local Kubernetes development. It manages kind (Kubernetes in Docker) clusters and orchestrates the installation, upgrade, and removal of services defined in a simple, declarative YAML configuration file.

Key Features:

Simple YAML Configuration - Define your entire development environment in one file
Automatic Cluster Management - Creates and configures kind clusters automatically
Helm & Manifest Support - Deploy Helm charts and raw Kubernetes manifests
Dependency Resolution - Services are installed in the correct order based on dependencies
Clean Teardown - Removes all resources including CRDS, namespaces, and PVCs
State Management - Cluster-stored state via ConfigMap for team-friendly workflows
docker-compose UX - Familiar commands: up, down, status
Corporate Network Support - Works behind proxies with TLS inspection and custom CAs
GPU Support - Run NVIDIA and AMD GPU workloads in kind clusters (v0.7.0+)
Portable Packages - Bundle your deployment into a .tar.gz to share with teammates or distribute to servers

Breaking Changes (v0.6.0)

State File Removed

kraze v0.6.0+ no longer uses local .kraze.state files. All state is now stored in a ConfigMap (kraze-metadata) in the cluster's kube-system namespace.

If upgrading from v0.5.x or earlier:

Destroy existing clusters created with old versions:
```
# Using old kraze version
kraze destroy
```
Upgrade kraze to v0.6.0+
Recreate your clusters:
```
# Using new kraze version
kraze up
```

Why this change?

Team-friendly: No local state files that don't sync across team members
Portable: Cluster state travels with the cluster, not tied to your machine
Reliable: Cluster is the single source of truth, eliminating drift
Simpler: No cache invalidation or stale state file issues

Technical Details:

State is stored in kubectl -n kube-system get cm kraze-metadata
Tracks service installation status, namespaces, and image hashes
Automatically created during kraze init or first kraze up
Deleted when running kraze destroy

Demo

Watch kraze orchestrate a full microservices stack with dependencies, Helm charts, and custom manifests:

The interactive UI shows real-time progress with:

Live status updates - All services visible at once with in-place updates
Animated spinners - Visual feedback during installation and resource loading
Dependency ordering - Services start automatically in the correct sequence
Clean output - No scrolling clutter, just clear status for each service

Tip: Use --plain for traditional scrolling output or -v for detailed verbose logging.

Installation

Prerequisites

Docker-compatible runtime - Docker Desktop, Colima, Podman, or Rancher Desktop must be running
Go 1.25+ - Required to build from source (not needed for Homebrew installation)

Homebrew (macOS)

brew install hjames9/kraze/kraze

Shell completions are automatically installed and configured.

Prebuilt Binaries

Download the latest release for your platform from the releases page.

Available for:

Linux (amd64, arm64)
macOS (amd64, arm64)
Windows (amd64, arm64)

# Example: Download and install on Linux/macOS
curl -L https://github.com/hjames9/kraze/releases/download/v0.5.4/kraze-v0.5.4-linux-amd64 -o kraze
chmod +x kraze
sudo mv kraze /usr/local/bin/

Build from Source

# Clone the repository
git clone https://github.com/hjames9/kraze.git
cd kraze

# Build the binary
make build

# The binary will be in ./build/kraze
./build/kraze --help

# Optional: Install to system path
make install

Shell Completion

kraze supports shell completion for bash, zsh, fish, and PowerShell.

Bash:

# Load completion for current session
source <(kraze completion bash)

# Install permanently (Linux)
kraze completion bash | sudo tee /etc/bash_completion.d/kraze

# Install permanently (macOS with Homebrew)
kraze completion bash > $(brew --prefix)/etc/bash_completion.d/kraze

Zsh:

# Enable completion if not already enabled
echo "autoload -U compinit; compinit" >> ~/.zshrc

# Install completion
mkdir -p ~/.zsh/completions
kraze completion zsh > ~/.zsh/completions/_kraze

# Add to .zshrc if not present
echo 'fpath=(~/.zsh/completions $fpath)' >> ~/.zshrc

# Restart shell or reload
source ~/.zshrc

Fish:

# Install completion
mkdir -p ~/.config/fish/completions
kraze completion fish > ~/.config/fish/completions/kraze.fish

PowerShell:

# Load for current session
kraze completion powershell | Out-String | Invoke-Expression

# Add to profile for all sessions
kraze completion powershell >> $PROFILE

Quick Start

Create a kraze.yml file in your project:

cluster:
  name: dev-cluster
  version: "1.34.0"
  config:
    - role: control-plane
      extraPortMappings:
        - containerPort: 30080
          hostPort: 8080
          protocol: TCP

services:
  # Deploy Redis from Helm chart
  redis:
    type: helm
    repo: oci://registry-1.docker.io/bitnamicharts
    chart: redis
    namespace: data
    create_namespace: true

  # Deploy PostgreSQL from Helm chart
  postgres:
    type: helm
    repo: oci://registry-1.docker.io/bitnamicharts
    chart: postgresql
    version: 18.1.9
    namespace: data
    create_namespace: true

  # Deploy custom app from Kubernetes manifests
  myapp:
    type: manifests
    path: ./k8s/manifests
    namespace: app
    depends_on:
      - redis
      - postgres

Run your environment:

# Start everything (creates cluster + installs services)
kraze up

# Check status
kraze status

# Output:
# Cluster: dev-cluster
#
# SERVICE    TYPE         INSTALLED  READY      MESSAGE
# --------------------------------------------------------------------------------
# redis      helm         Yes        Yes        deployed
# postgres   helm         Yes        Yes        deployed
# myapp      manifests    Yes        Yes        3 resources applied

# Stop and clean up everything
kraze down

# Destroy the cluster
kraze destroy

Usage

Commands

`kraze up [services...]`

Install and start services defined in kraze.yml. If no services are specified, all services are started.

# Start all services
kraze up

# Start specific services
kraze up redis postgres

# Use a different config file
kraze up -f dev.yml

# Merge multiple config files (services from all files combined into one cluster)
kraze up -f app/kraze.yml -f ml-stack/kraze.yml

# Wait for resources to be ready
kraze up --wait --timeout 5m

# See what would happen without executing
kraze up --dry-run

`kraze down [services...]`

Uninstall services. Automatically cleans up namespaces and PVCs that were created.

# Stop all services
kraze down

# Stop specific services
kraze down myapp

# Keep Custom Resource Definitions (CRDs)
kraze down --keep-crds

`kraze status`

Show the current status of all services.

kraze status

# Verbose output
kraze status -v

`kraze plan [services...]`

Show a detailed plan of what would be installed or changed without actually executing.

# Plan all services
kraze plan

# Plan specific services (includes dependencies)
kraze plan frontend

# Plan services with labels
kraze plan --label env=prod

# Use custom config
kraze plan -f dev.yml

Output includes:

Services to be added, changed, or unchanged
Dependency levels and parallel execution groups
Namespaces that would be created
Cluster status and configuration

Example output:

Cluster: deps-cluster
  Status: no, will be created

Services to install: 4

Level 0 (parallel installation):
  + postgres (add) - helm chart postgresql@15.5.38
    + Namespace: data (will be created)
  + redis (add) - helm chart redis@20.2.1
    + Namespace: data (will be created)

Level 1:
  + backend (add) - manifests from remote URL
    + Namespace: app (will be created)
      Depends on: postgres, redis

Plan: 2 to add

`kraze init`

Create and initialize a new kind cluster.

kraze init

# Use custom config
kraze init -f kraze.yml

`kraze destroy`

Delete the kind cluster and clean up all resources.

kraze destroy

# Use custom config
kraze destroy -f kraze.yml

`kraze validate`

Validate your kraze.yml configuration file.

kraze validate

# Validate specific file
kraze validate -f dev.yml

`kraze pack`

Bundle a kraze deployment into a portable .tar.gz archive for sharing.

At pack time, all local assets (charts, manifests, values files, CA certs) are included directly. Remote Helm charts (OCI/HTTPS) are pulled and bundled as .tgz files, and remote HTTP manifests are downloaded and bundled — so the recipient needs no access to Helm repos or manifest URLs. Container images are the only thing fetched at runtime (from Docker registries).

The commands up, validate, and plan all accept .tar.gz packages transparently via the -f flag.

# Pack the deployment in the current directory
kraze pack

# Pack with a specific config file and output path
kraze pack -f kraze.yml -o myapp.tar.gz

# The recipient can then run it directly — no internet access to Helm repos needed
kraze up -f myapp.tar.gz

# Or validate and plan before deploying
kraze validate -f myapp.tar.gz
kraze plan -f myapp.tar.gz

`kraze load-image <image...>`

Load local Docker images into the kind cluster.

# Load one or more images
kraze load-image myapp:latest myworker:v1.0

# Useful for local development workflow
docker build -t myapp:dev .
kraze load-image myapp:dev
kubectl rollout restart deployment/myapp

`kraze version`

Display version information.

kraze version

`kraze completion [bash|zsh|fish|powershell]`

Generate shell completion scripts.

# Bash
kraze completion bash > /etc/bash_completion.d/kraze

# Zsh
kraze completion zsh > ~/.zsh/completions/_kraze

# Fish
kraze completion fish > ~/.config/fish/completions/kraze.fish

# PowerShell
kraze completion powershell >> $PROFILE

See Shell Completion section for detailed installation instructions.

Configuration File Reference

The kraze.yml file defines your cluster and services:

# Cluster configuration
cluster:
  name: my-cluster                    # Name of the kind cluster
  version: "1.34.0"                   # Kubernetes version (optional)
  network: "dev"                      # Docker network name (optional, auto-detected if not specified)
  ipv4_address: "172.1.0.2"           # Static IPv4 for cluster container (optional)
  subnet: "172.1.0.0/16"              # Network subnet (optional, creates network if doesn't exist)
  config:                             # kind cluster configuration
    - role: control-plane
      extraPortMappings:              # Expose ports from cluster
        - containerPort: 30080
          hostPort: 8080
          protocol: TCP

  # Optional: Corporate network support
  # ca_certificates:                  # Trust custom CA certificates
  #   - /etc/ssl/certs/corporate-ca.crt
  # insecure_registries:              # Skip TLS verification for specific registries
  #   - registry.corp.com
  # proxy:                            # HTTP/HTTPS proxy (opt-in)
  #   enabled: true                   # Use HTTP_PROXY, HTTPS_PROXY, NO_PROXY from env
  #   # Or set explicit values (enabled field not needed):
  #   http_proxy: http://proxy:8080
  #   https_proxy: http://proxy:8080
  #   no_proxy: localhost,127.0.0.1

  # Optional: GPU support (v0.7.0+, kind clusters only)
  # gpu:
  #   nvidia:
  #     enabled: true                 # all GPUs exposed; use CUDA_VISIBLE_DEVICES in pods
  #   amd:
  #     enabled: true                 # all GPUs auto-discovered; use ROCR_VISIBLE_DEVICES in pods

  # Optional: Use existing cluster (Docker Desktop, Minikube, remote)
  # external:
  #   enabled: true
  #   kubeconfig: ~/.kube/config      # Optional - default: ~/.kube/config
  #   context: docker-desktop         # Optional - default: current-context

# Services to deploy
services:
  # Helm chart from OCI registry
  service-name:
    type: helm
    repo: oci://registry-1.docker.io/bitnamicharts
    chart: redis
    version: 23.2.6              # Optional - defaults to latest
    namespace: data
    create_namespace: true       # Defaults to true
    values: values.yml           # Single values file
    # OR multiple values files (merged in order, later overrides earlier):
    # values:
    #   - base-values.yml
    #   - prod-values.yml
    depends_on:                  # Optional - list of dependencies
      - other-service
    wait: true                   # Wait for resources to be ready (defaults to CLI flag)
    wait_timeout: "15m"          # Timeout for wait operations (defaults to CLI timeout)
    post_ready_delay: "5s"       # Delay after service is ready before continuing (defaults to 3s)

  # Helm chart from HTTP repository
  another-service:
    type: helm
    repo: https://charts.bitnami.com/bitnami
    chart: postgresql
    namespace: database

  # Local Helm chart
  local-chart:
    type: helm
    path: ./charts/myapp        # Path to local chart directory
    namespace: app

  # Kubernetes manifests
  manifest-service:
    type: manifests
    path: ./k8s                 # Directory or single YAML file
    namespace: app
    depends_on:
      - service-name

  # Explicit image list (supplements auto-detection)
  # Use when images are in non-standard locations that auto-detection cannot reach
  # (e.g., extraInitContainers YAML strings, operator-managed pods, ConfigMap-sourced images)
  service-with-extra-images:
    type: helm
    repo: oci://registry-1.docker.io/bitnamicharts
    chart: keycloak
    namespace: auth
    images:
      - myorg/custom-theme:latest   # loaded before helm install, merged with auto-detected images

Disabling Services

You can temporarily disable services without removing them from your configuration using the enabled field:

services:
  postgres:
    type: helm
    chart: postgresql
    repo: oci://registry-1.docker.io/bitnamicharts
    namespace: data
    enabled: true               # Explicitly enabled (default)

  redis:
    type: helm
    chart: redis
    repo: oci://registry-1.docker.io/bitnamicharts
    namespace: data
    enabled: false              # Disabled - will be skipped

  backend:
    type: manifests
    path: ./manifests/backend
    namespace: app
    # No 'enabled' field - defaults to true

Behavior:

Disabled services are completely skipped during kraze up (not installed)
Disabled services are ignored during kraze down (not uninstalled)
kraze status shows disabled services with "DISABLED" status
kraze plan shows disabled services as "skipped"
Validation: Enabled services cannot depend on disabled services (validation error)

Use Cases:

Temporarily disable services during development/testing
Keep service definitions as reference without deploying them
Quickly toggle services on/off without editing configuration structure

Example:

# With redis disabled in config:
kraze up        # Installs postgres and backend only
kraze status    # Shows redis as DISABLED
kraze plan      # Shows "1 skipped" in summary

Environment Variables

You can use environment variable substitution in your configuration:

services:
  myapp:
    type: helm
    repo: ${HELM_REPO:-https://charts.bitnami.com/bitnami}
    chart: ${APP_CHART:-redis}
    namespace: ${NAMESPACE:-default}

Corporate Network Support

kraze works seamlessly in corporate environments with TLS inspection proxies and custom certificate authorities.

Custom CA Certificates

If your network uses TLS inspection (MITM proxies), add your corporate CA certificate:

cluster:
  name: dev-cluster
  ca_certificates:
    - /etc/ssl/certs/corporate-ca.crt

kraze will mount the certificate into cluster nodes and configure containerd to trust it.

HTTP/HTTPS Proxy

Proxy support is opt-in. To use environment variables, enable proxy in your config:

cluster:
  proxy:
    enabled: true  # Use HTTP_PROXY, HTTPS_PROXY, NO_PROXY from environment

export HTTP_PROXY=http://proxy.corp.com:8080
export HTTPS_PROXY=http://proxy.corp.com:8080
export NO_PROXY=localhost,127.0.0.1,.svc,.cluster.local
kraze up

Or configure proxy settings directly in YAML (no enabled field needed):

cluster:
  proxy:
    http_proxy: http://proxy.corp.com:8080
    https_proxy: http://proxy.corp.com:8080
    no_proxy: localhost,127.0.0.1,.svc,.cluster.local

To disable proxy even when environment variables are set:

cluster:
  proxy:
    enabled: false

Insecure Registries

As a quick workaround, skip TLS verification for specific registries:

cluster:
  insecure_registries:
    - ghcr.io
    - registry.corp.com

Note: Using CA certificates is more secure than insecure registries.

See examples/corporate-network/ for complete examples and troubleshooting.

GPU Support

Requires kraze v0.7.0+. Only supported for kind clusters (not external clusters).

kraze can configure kind clusters to expose NVIDIA and/or AMD GPUs to pods. Both vendors can be enabled simultaneously.

NVIDIA GPUs

kraze uses Docker's --gpus all flag (via DeviceRequests) when creating kind node containers. This works with the standard Docker runtime — the default runtime does not need to be changed to nvidia.

All GPUs on the host are exposed to the cluster. Use CUDA_VISIBLE_DEVICES in pod specs to restrict individual workloads to specific GPUs.

Host prerequisites:

nvidia-container-toolkit installed: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html

Verify the setup:

nvidia-ctk --version
nvidia-container-runtime --version
docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi

Configuration:

cluster:
  name: gpu-cluster

  gpu:
    nvidia:
      enabled: true

After cluster creation, kraze registers the nvidia RuntimeClass so pods can use runtimeClassName: nvidia.

AMD GPUs

For AMD GPUs, kraze bind-mounts /dev/kfd (the ROCm Kernel Fusion Driver, shared across all GPUs) and all discovered /dev/dri/renderD* nodes into each kind node. No Docker runtime changes are needed — standard runc handles AMD device bind-mounts.

All GPUs on the host are exposed to the cluster. Use ROCR_VISIBLE_DEVICES or HIP_VISIBLE_DEVICES in pod specs to restrict individual workloads to specific GPUs.

Host prerequisites:

AMD GPU drivers and ROCm runtime installed
/dev/kfd and /dev/dri/renderD* devices present

Verify the setup:

ls /dev/kfd              # should exist
ls /dev/dri/renderD*     # should list one device per GPU

Configuration:

cluster:
  name: amd-gpu-cluster

  gpu:
    amd:
      enabled: true

Multi-node clusters

In a multi-node cluster, GPU mounts are applied only to worker nodes. In a single-node (default) cluster, they are applied to the control-plane node.

Mismatch detection

GPU configuration is stored in cluster state at creation time. If you change GPU settings after the cluster was created, kraze returns a clear error:

GPU configuration has changed since cluster 'gpu-cluster' was created.
GPU support requires cluster recreation. Run: kraze destroy && kraze up

See examples/nvidia-gpu/ and examples/amd-gpu/ for complete examples.

Wait Behavior and Dependencies

kraze automatically handles service dependencies and ensures services are ready before starting dependent services.

CLI Flags

--wait (default: true) - Wait for all resources to be ready before proceeding
--no-wait - Skip waiting for resources to be ready
--timeout <duration> (default: 10m) - Global timeout for wait operations

Per-Service Configuration

You can override wait behavior for individual services:

services:
  rabbitmq:
    type: helm
    chart: rabbitmq
    repo: oci://registry-1.docker.io/bitnamicharts
    namespace: messaging
    wait: true           # Ensure RabbitMQ is fully ready
    wait_timeout: "15m"  # Give it 15 minutes to start

  rabbitmq-migrator:
    type: helm
    chart: ./migrations
    namespace: messaging
    depends_on:
      - rabbitmq         # Will only start after rabbitmq is ready
    wait_timeout: "5m"   # Migrations are usually quick

Configuration Precedence:

Service-specific wait and wait_timeout (highest priority)
CLI flags --wait, --timeout
Defaults: wait=true, timeout="10m"

What "ready" means:

For Helm charts:

Helm's built-in wait functionality monitors all resources
Deployments: status.availableReplicas == spec.replicas
StatefulSets: status.readyReplicas == spec.replicas
DaemonSets: status.numberReady == status.desiredNumberScheduled
Jobs: status.succeeded > 0 or conditions show Complete
All readiness probes must pass

For Manifests:

kraze polls each applied resource until ready
Same readiness checks as Helm
For CRDs (like RabbitmqCluster): Checks status.conditions for Ready=True
Polls every 2 seconds until ready or timeout

Example workflow:

# Use defaults (wait=true, timeout=10m)
kraze up

# Override globally
kraze up --wait --timeout 20m

# Skip waiting entirely (fast but risky)
kraze up --no-wait

Global Flags

-f, --file - Path to configuration file; can be specified multiple times to merge configs (default: kraze.yml)
-v, --verbose - Enable verbose output
--dry-run - Show what would happen without executing

Examples

See the examples/ directory for complete working examples:

minimal/ - Simplest possible configuration (single service)
charts/ - All Helm chart sources (OCI, HTTPS, local)
manifests/ - All manifest sources (local files, directories, remote URLs)
dependencies/ - Multi-service with dependency management
external-cluster/ - Use existing clusters (Docker Desktop, Minikube, remote)
labels/ - Filter services by labels (env, tier, team)
corporate-network/ - Corporate environments (proxies, CA certificates, TLS inspection)
nvidia-gpu/ - NVIDIA GPU support for kind clusters (v0.7.0+)
amd-gpu/ - AMD GPU support for kind clusters (v0.7.0+)
amd-rocminfo/ - End-to-end GPU verification with rocminfo for AMD GPUs (v0.7.0+)
vllm-amd/ - vLLM on AMD GPU using the ROCm build (v0.7.0+)
cuda-vectoradd/ - End-to-end GPU verification with NVIDIA's CUDA vectorAdd sample (v0.7.0+)
vllm/ - vLLM serving an LLM with HuggingFace model download (v0.7.0+)
vllm-local/ - vLLM serving an LLM from a local model cache (v0.7.0+)
vllm-openwebui/ - vLLM + Open WebUI complete local AI stack (v0.7.0+)
vllm-rag/ - vLLM + pgvector + Open WebUI RAG stack for document-grounded chat (v0.7.0+)
vllm-rag-amd/ - vLLM + pgvector + Open WebUI RAG stack on AMD GPUs with ROCm (v0.7.0+)
multi-config/ - Multiple config files merged into one cluster (app stack + ML/GPU stack)
explicit-images/ - Supplement auto-detected images with an explicit list for edge cases like extraInitContainers

Validate all examples:

make validate-examples   # CLI validation (integration test)
make test                # Includes example unit tests

Development

# Install dependencies
make deps

# Build
make build

# Run tests
make test

# Format code
make fmt

# Run linter
make lint

# Clean build artifacts
make clean

License

Apache License 2.0 - see LICENSE for details

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Name		Name	Last commit message	Last commit date
Latest commit History 109 Commits
cmd/kraze		cmd/kraze
docs		docs
examples		examples
internal		internal
.gitignore		.gitignore
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
VERSION		VERSION
go.mod		go.mod
go.sum		go.sum

Folders and files

Latest commit

History

Repository files navigation

kraze

Table of Contents

Description

Breaking Changes (v0.6.0)

State File Removed

Demo

Installation

Prerequisites

Homebrew (macOS)

Prebuilt Binaries

Build from Source

Shell Completion

Quick Start

Usage

Commands

kraze up [services...]

kraze down [services...]

kraze status

kraze plan [services...]

kraze init

kraze destroy

kraze validate

kraze pack

kraze load-image <image...>

kraze version

kraze completion [bash|zsh|fish|powershell]

Configuration File Reference

Disabling Services

Environment Variables

Corporate Network Support

Custom CA Certificates

HTTP/HTTPS Proxy

Insecure Registries

GPU Support

NVIDIA GPUs

AMD GPUs

Multi-node clusters

Mismatch detection

Wait Behavior and Dependencies

CLI Flags

Per-Service Configuration

Global Flags

Examples

Development

License

Contributing

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 32

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

`kraze up [services...]`

`kraze down [services...]`

`kraze status`

`kraze plan [services...]`

`kraze init`

`kraze destroy`

`kraze validate`

`kraze pack`

`kraze load-image <image...>`

`kraze version`

`kraze completion [bash|zsh|fish|powershell]`

Packages