Skip to content

DEPLOYMENT.md

Latest commit

 

History

History
345 lines (256 loc) · 9.97 KB

DEPLOYMENT.md

File metadata and controls

345 lines (256 loc) · 9.97 KB
Raw

Deploying small-llm-agent to Kind Cluster

This guide covers deploying the small-llm-agent Helm chart to your local Kind cluster with nginx ingress.

Prerequisites

  • kind, kubectl, and helm installed
  • envsubst available (part of gettext; on macOS: brew install gettext && brew link --force gettext)
  • Docker running locally

Once the steps below are complete, you'll have:

  • Kind cluster local-kind-cluster running (Kubernetes 1.33.1)
  • nginx-ingress controller deployed (v1.15.1), pinned to the control-plane node
  • IngressClass nginx available
  • Local storage mounted at /data on worker node

Setup

1. Create the Kind Cluster

local-cluster.yaml uses ${HOME} for the host data mount so the config stays portable across machines. Kind does not expand environment variables itself, so render it with envsubst before passing it in:

envsubst < infra/kind/local-cluster.yaml | kind create cluster --config -

Then deploy the ingress controller:

kubectl apply -f infra/kind/ingress-controller.yaml

2. Create StorageClass and Namespace

First, create the local storage class that uses your /data directory and the agent namespace:

# Apply storage class (uses /data mounted in Kind worker node)
kubectl apply -f infra/kind/storage-class.yaml

# Create namespace
kubectl create namespace small-llm-agent

Verify storage class:

kubectl get storageclass
kubectl get pv

3. Update /etc/hosts

For localhost access to the ingress:

echo "127.0.0.1 small-llm-agent.localhost" | sudo tee -a /etc/hosts

Verify:

ping small-llm-agent.localhost

4. Deploy small-llm-agent

# Deploy to small-llm-agent namespace
helm install small-llm-agent agents/small-llm-agent/helm \
  -f agents/small-llm-agent/helm/values-dev.yaml \
  --namespace small-llm-agent

Or with version control:

helm install small-llm-agent agents/small-llm-agent/helm \
  -f agents/small-llm-agent/helm/values-dev.yaml \
  --namespace default \
  --set image.tag=0.1.0

5. Monitor Deployment

# Watch pod startup (model pulling takes 1-2 minutes)
kubectl get pods -w -n small-llm-agent

# Check logs
kubectl logs -f deployment/small-llm-agent -n small-llm-agent

# Check ingress status
kubectl get ingress -n small-llm-agent

6. Test Access

Via Ingress (localhost):

curl http://small-llm-agent.localhost/api/tags

Via Port Forward:

kubectl port-forward -n small-llm-agent svc/small-llm-agent 11434:11434 &
curl http://localhost:11434/api/tags

Generate Text:

curl -X POST http://small-llm-agent.localhost/api/generate \
  -H "Content-Type: application/json" \
  -d '{
    "model": "tinyllama",
    "prompt": "What is Kubernetes?",
    "stream": false
  }'

Interacting with the Agent

The pod runs Ollama only (no interactive prompt) for reliable Kubernetes operation. You can interact via:

1. Ollama REST API (Recommended for K8s)

Already working via ingress or port-forward:

# List models
curl http://small-llm-agent.localhost/api/tags

# Generate response (streaming)
curl -X POST http://small-llm-agent.localhost/api/generate \
  -H "Content-Type: application/json" \
  -d '{
    "model": "tinyllama",
    "prompt": "Explain containers",
    "stream": true
  }' | jq '.response' -r

2. Local Docker (Interactive)

For interactive testing locally with the Python agent:

# Build and run with -it flags
docker build -t small-llm-agent agents/small-llm-agent/
docker run -it -p 11434:11434 small-llm-agent

3. Python Client (Port Forward)

For programmatic access:

# Terminal 1: Port forward
kubectl port-forward -n small-llm-agent svc/small-llm-agent 11434:11434 &

# Terminal 2: Use Python
python agents/small-llm-agent/app/main.py

Persistence

The deployment uses a PersistentVolumeClaim (small-llm-agent) backed by a shared local storage at /data/ on the Kind worker node. Each deployment gets its own subdirectory using PVC subPath.

How it works:

  1. StorageClass local-storage enables local provisioning
  2. PersistentVolume local-pv-data maps to /data (reusable by multiple deployments)
  3. PersistentVolumeClaim small-llm-agent binds to the PV with selector labels
  4. Pod mounts PVC at /root/.ollama with subPath: small-llm-agent
  5. This creates isolated directory: /data/small-llm-agent/ ↔ /root/.ollama

Benefits:

  • Single PV can be reused by multiple agents/deployments
  • Each deployment gets isolated subdirectory via subPath
  • Easy to add more agents without creating new PVs

Check PVC and PV:

kubectl get pvc -n small-llm-agent
kubectl get pv
kubectl describe pvc small-llm-agent -n small-llm-agent

Access data from host:

# Model files are stored on your host at:
ls -la ~/workspace/kind/small-llm-agent/

# Inside this folder, you'll find:
# - ollama/ directory with model cache

Model persists across:

  • Pod restarts ✅
  • Helm upgrades ✅
  • Kind cluster restarts ✅
  • Host filesystem (data saved to ~/workspace/kind/small-llm-agent/) ✅

Resource Usage

Dev Environment (as configured):

  • CPU Request: 500m (half a core)
  • CPU Limit: 1000m (one core)
  • Memory Request: 1Gi
  • Memory Limit: 2Gi

Monitor actual usage:

kubectl top pod -n small-llm-agent -l app.kubernetes.io/name=small-llm-agent

Troubleshooting

Pod stuck in Pending

kubectl describe pod -n small-llm-agent deployment/small-llm-agent
# Check for PVC binding issues, storage class, or insufficient resources
kubectl get pvc -n small-llm-agent  # Verify PVC is Bound
kubectl get pv  # Verify PV is Available

Model pulling stuck

# Check logs for download progress
kubectl logs -f deployment/small-llm-agent -n small-llm-agent
# Model is ~600MB, may take 1-2 minutes on first run

PVC not binding

# Check if local-storage storage class exists
kubectl get storageclass

# Check PV and PVC status
kubectl get pv local-pv-data
kubectl describe pvc small-llm-agent -n small-llm-agent

# Solution: Re-apply storage class
kubectl apply -f infra/kind/storage-class.yaml

Ingress not working

# Verify ingress controller is running
kubectl -n ingress-nginx get pods

# Check ingress status
kubectl describe ingress -n small-llm-agent small-llm-agent

# Test DNS (hosts entry)
ping small-llm-agent.localhost

# Check nginx logs
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx --tail=50

API not responding

# Check service endpoints
kubectl get endpoints -n small-llm-agent small-llm-agent

# Port forward and test directly
kubectl port-forward -n small-llm-agent svc/small-llm-agent 11434:11434 &
curl localhost:11434/api/tags

Cleanup

# Uninstall (keeps PVC and data for persistence)
helm uninstall small-llm-agent -n small-llm-agent

# Delete namespace (removes deployment and PVC)
kubectl delete namespace small-llm-agent

# Delete storage resources (optional, removes PV)
kubectl delete pv local-pv-data
kubectl delete storageclass local-storage

Note: Data in ~/workspace/kind/ will persist even after cleanup unless manually deleted.

Removing the cached image (save disk space)

The Kind worker node caches pulled images independently of your local Docker daemon, and imagePullPolicy: IfNotPresent means it won't be removed automatically. To free up space on your machine:

# Remove the image from the Kind worker node's containerd cache
docker exec local-kind-cluster-worker crictl rmi docker.io/natandias1/small-llm-agent:0.1.0

# Remove the image from your local Docker daemon too (if built/pulled locally)
docker rmi natandias1/small-llm-agent:0.1.0
docker rmi small-llm-agent

Publishing Charts to the Helm Repository

Charts are packaged and published automatically by .github/workflows/release-charts.yml whenever agents/**/helm/** changes on main. It publishes to the gh-pages branch, served via GitHub Pages at:

https://natan-dias.github.io/agentic-cicd/

One-time setup

  1. Push a change under agents/*/helm/ to main so the workflow runs once and creates the gh-pages branch.
  2. In GitHub: Settings → Pages → Source → Deploy from a branch → gh-pages / (root).
  3. Wait a minute for Pages to build, then verify: curl https://natan-dias.github.io/agentic-cicd/index.yaml

Bumping chart versions

helm repo index keys chart versions by the version field in Chart.yaml (independent of the app's version.txt/image tag). Bump it whenever the chart templates or default values change, so consumers can pin to a specific chart version:

# agents/small-llm-agent/helm/Chart.yaml
version: 0.1.1   # bump this
appVersion: 0.1.0 # matches the agent's version.txt

Consuming the Chart as a "Client"

This simulates a third party installing your chart with their own values, pulling from the published repo instead of a local path.

# Add the repo
helm repo add agentic-cicd https://natan-dias.github.io/agentic-cicd/
helm repo update

# See what's available
helm search repo agentic-cicd

# Install with a client-specific values file
helm install my-client-agent agentic-cicd/small-llm-agent \
  --namespace my-client-agent --create-namespace \
  -f agents/small-llm-agent/helm/values-client-example.yaml

values-client-example.yaml (in the chart directory) overrides just what a client would care about — different ingress host, smaller footprint, persistence disabled since it depends on this project's own Kind storage setup. A real client would copy this file and adjust it to their own cluster (e.g. point persistence.storageClassName at a storage class that exists for them).

This proves the chart works standalone from the repo, decoupled from this project's own values-dev.yaml/values-prod.yaml.

Next Steps

  1. Test the deployment in your Kind cluster
  2. Verify ingress access at http://small-llm-agent.localhost
  3. Once validated, update CLAUDE.md with deployment instructions
  4. Later: Set up ArgoCD for multi-environment deployments (dev/staging/prod)