Skip to content

SETUP.md

Latest commit

 

History

History
508 lines (376 loc) · 9.51 KB

SETUP.md

File metadata and controls

508 lines (376 loc) · 9.51 KB
Raw

DevOpsPilot AI — Complete Setup Guide

Production-grade DevOps platform. Autonomous deployment, monitoring, cost optimization, incident response.

Prerequisites

  • Node.js 18+
  • Python 3.11+
  • PostgreSQL 14+
  • Docker & Docker Compose
  • Git

Quick Start (5 minutes)

1. Clone & Setup

git clone <repo>
cd forge-autonomy-os

# Copy env template
cp .env.example .env

2. Configure .env

Edit .env:

# Database
DATABASE_URL=postgresql://devops:devops@localhost:5432/devops_pilot

# APIs
OPENAI_API_KEY=sk-your-key-here
ANTHROPIC_API_KEY=sk-ant-your-key-here

# GitHub
GITHUB_APP_ID=123456
GITHUB_APP_SECRET=your-secret
GITHUB_WEBHOOK_SECRET=your-webhook-secret

# Cloud (optional)
AWS_ACCESS_KEY_ID=your-key
AWS_SECRET_ACCESS_KEY=your-secret
GCP_PROJECT_ID=your-project
AZURE_SUBSCRIPTION_ID=your-subscription

# Backend
BACKEND_PORT=8000
LOG_LEVEL=info

# Frontend
VITE_API_URL=http://localhost:8000
VITE_ENV=development

3. Docker Compose (Easiest)

docker-compose up -d

Services start:

4. Verify

# Backend health
curl http://localhost:8000/health
# Should return: {"status": "healthy", "version": "1.0.0"}

# Frontend loads
open http://localhost:5173

Manual Setup (Production)

Backend Setup

cd backend

# Install dependencies
python -m pip install --upgrade pip
pip install -r requirements.txt

# Database migrations
alembic upgrade head

# Run server
python -m uvicorn app.main:app --host 0.0.0.0 --port 8000 --workers 4

Frontend Setup

# Install dependencies
npm install

# Dev server
npm run dev

# Production build
npm run build
npm run preview

Database Setup

# Create database
psql -U postgres -c "CREATE DATABASE devops_pilot"
psql -U postgres -c "CREATE USER devops WITH PASSWORD 'devops'; GRANT ALL PRIVILEGES ON DATABASE devops_pilot TO devops;"

# Verify connection
psql postgresql://devops:devops@localhost:5432/devops_pilot -c "SELECT version();"

API Endpoints Reference

Build Agent

POST   /api/v1/build-agent/analyze-repo
POST   /api/v1/build-agent/generate-dockerfile
GET    /api/v1/build-agent/recommendations
POST   /api/v1/build-agent/validate-dockerfile
POST   /api/v1/build-agent/estimate-build-time
GET    /api/v1/build-agent/sbom
POST   /api/v1/build-agent/test-build
GET    /api/v1/build-agent/build-history

Docker Agent

POST   /api/v1/docker-agent/scan-image
GET    /api/v1/docker-agent/layer-analysis
POST   /api/v1/docker-agent/optimize-image
GET    /api/v1/docker-agent/sbom
POST   /api/v1/docker-agent/set-registry-policy
GET    /api/v1/docker-agent/image-history
POST   /api/v1/docker-agent/cleanup-unused-images
GET    /api/v1/docker-agent/vulnerability-trends
POST   /api/v1/docker-agent/auto-tag

Kubernetes Agent

POST   /api/v1/k8s-agent/generate-manifests
POST   /api/v1/k8s-agent/validate-manifests
POST   /api/v1/k8s-agent/recommend-resources
POST   /api/v1/k8s-agent/generate-policies
POST   /api/v1/k8s-agent/deploy-strategy
GET    /api/v1/k8s-agent/cluster-health
POST   /api/v1/k8s-agent/generate-hpa
POST   /api/v1/k8s-agent/cost-analysis
POST   /api/v1/k8s-agent/multi-cluster-deploy
GET    /api/v1/k8s-agent/manifest-drift

Deployment Agent

POST   /api/v1/deployment-agent/deploy
GET    /api/v1/deployment-agent/status/:deployment_id
POST   /api/v1/deployment-agent/pause
POST   /api/v1/deployment-agent/rollback
GET    /api/v1/deployment-agent/health-metrics
POST   /api/v1/deployment-agent/smoke-tests
GET    /api/v1/deployment-agent/estimated-duration
POST   /api/v1/deployment-agent/validate-pre-deploy

Monitoring Agent

POST   /api/v1/monitoring-agent/generate-dashboards
POST   /api/v1/monitoring-agent/generate-alerts
POST   /api/v1/monitoring-agent/define-slos
GET    /api/v1/monitoring-agent/metric-recommendations
POST   /api/v1/monitoring-agent/validate-slos
GET    /api/v1/monitoring-agent/alert-effectiveness
POST   /api/v1/monitoring-agent/auto-scale-alerts
GET    /api/v1/monitoring-agent/dashboard-library
POST   /api/v1/monitoring-agent/export-config

Cost Agent

GET    /api/v1/cost-agent/analysis
POST   /api/v1/cost-agent/rightsizing
GET    /api/v1/cost-agent/savings-potential
POST   /api/v1/cost-agent/spot-candidates
POST   /api/v1/cost-agent/reserved-instances
GET    /api/v1/cost-agent/forecast
POST   /api/v1/cost-agent/budget-alerts
GET    /api/v1/cost-agent/committed-usage
POST   /api/v1/cost-agent/anomaly-spend
POST   /api/v1/cost-agent/generate-report

Natural Language CLI

POST   /api/v1/nlc/parse
POST   /api/v1/nlc/execute
GET    /api/v1/nlc/examples
GET    /api/v1/nlc/suggestions
POST   /api/v1/nlc/confirm
GET    /api/v1/nlc/history
GET    /api/v1/nlc/help

Kubernetes Deployment

Build & Push Images

# Backend image
docker build -t your-registry/devops-pilot-api:v1 backend/
docker push your-registry/devops-pilot-api:v1

# Frontend image
docker build -t your-registry/devops-pilot-ui:v1 .
docker push your-registry/devops-pilot-ui:v1

Deploy to K8s

# Create namespace
kubectl create namespace devops-pilot

# Create secrets
kubectl create secret generic devops-secrets \
  --from-literal=database-url=$DATABASE_URL \
  --from-literal=openai-key=$OPENAI_API_KEY \
  -n devops-pilot

# Deploy
kubectl apply -f deploy/kubernetes/cluster-config.yaml
kubectl apply -f deploy/kubernetes/deploy-multi-cluster.sh

# Port forward for testing
kubectl port-forward -n devops-pilot svc/api-gateway 8000:8000
kubectl port-forward -n devops-pilot svc/frontend 5173:5173

Common Commands

Backend

# Run tests
cd backend
pytest

# Check health
curl http://localhost:8000/health

# View API docs
open http://localhost:8000/docs

# Database reset (dev only)
alembic downgrade base
alembic upgrade head

Frontend

# Dev server
npm run dev

# Build for production
npm run build

# Preview production build
npm run preview

# Lint & format
npm run lint
npm run format

Database

# Connect to DB
psql $DATABASE_URL

# Run migrations
alembic upgrade head

# Rollback migrations
alembic downgrade -1

# Check migration history
alembic history

Troubleshooting

Backend Won't Start

# Check Python version
python --version  # Should be 3.11+

# Check dependencies
pip list | grep fastapi

# Check database connection
psql $DATABASE_URL -c "SELECT 1"

# Check logs
tail -f backend.log

Frontend Build Fails

# Clear cache
rm -rf node_modules .next
npm install

# Check Node version
node --version  # Should be 18+

# Rebuild
npm run build --verbose

Database Connection Issues

# Test connection
psql -h localhost -U devops -d devops_pilot -c "SELECT 1"

# Check running services
docker ps | grep postgres

# Restart database
docker-compose down postgres
docker-compose up -d postgres

Port Already in Use

# Kill process on port 8000
lsof -ti:8000 | xargs kill -9

# Kill process on port 5173
lsof -ti:5173 | xargs kill -9

Performance Tuning

Database

-- Create indexes for common queries
CREATE INDEX idx_deployments_app_env 
ON deployments(app_id, environment);

CREATE INDEX idx_cost_records_period 
ON cost_records(period_start, period_end);

-- Check slow queries
SELECT query, calls, mean_time FROM pg_stat_statements 
ORDER BY mean_time DESC LIMIT 10;

Backend

# In main.py, enable caching
from fastapi_cache2 import FastAPICache2
from fastapi_cache2.backends.redis import RedisBackend
from redis import asyncio as aioredis

redis = await aioredis.from_url("redis://localhost")
FastAPICache2.init(RedisBackend(redis))

Frontend

# Enable compression
npm install compression
# Add to vite.config.ts
import compression from 'vite-plugin-compression'

export default {
  plugins: [compression()]
}

Monitoring

Prometheus Metrics

# Access metrics
curl http://localhost:8000/metrics

# Common queries
rate(http_requests_total[5m])
histogram_quantile(0.99, http_request_duration_seconds_bucket)

Logs

# Backend logs
docker logs devops-pilot-api

# Frontend console
open http://localhost:5173 (F12 → Console)

# Database slow queries
tail -f /var/log/postgresql.log

Security

Before Production

  • Change default passwords
  • Enable HTTPS (SSL certificates)
  • Setup firewall rules
  • Enable database backups
  • Setup VPC/network isolation
  • Rotate API keys monthly
  • Enable audit logging
  • Setup rate limiting
  • Enable CORS properly (not *)
  • Setup secrets management (Vault, AWS Secrets)

Essential Commands

# Generate secure random password
openssl rand -base64 32

# Create self-signed cert for testing
openssl req -x509 -newkey rsa:4096 -nodes -out cert.pem -keyout key.pem -days 365

# Check for vulnerabilities
npm audit
pip install safety && safety check

Scaling

Horizontal

# Multiple backend instances
docker-compose up -d --scale api=3

# Load balance with nginx (deploy/nginx.conf)
docker-compose up -d nginx

Vertical

# Increase worker count
python -m uvicorn app.main:app --workers 8

# Increase database connections
# In .env: DATABASE_POOL_SIZE=20

Support

API Docs: http://localhost:8000/docs
GitHub Issues: https://github.com/your-org/devops-pilot/issues
Slack: #devops-pilot channel

Built with: FastAPI • React • PostgreSQL • Kubernetes • Prometheus • Terraform