Deploying Ampel to Fly.io

This guide provides comprehensive instructions for deploying Ampel to Fly.io using their global application platform with managed databases, private networking, and automated deployments.

Architecture Overview
Prerequisites
Quick Start
Infrastructure Setup
Application Deployment
Secrets Management
CI/CD with GitHub Actions
Custom Domain Setup
Monitoring and Operations
Scaling
Troubleshooting
Cost Estimation
References

Architecture Overview

Ampel deploys as three separate Fly.io applications connected via private networking:

┌─────────────────────────────────────────────────────────────────┐
│                      Fly.io Organization                        │
│                                                                 │
│  ┌────────────────┐  ┌────────────────┐  ┌──────────────────┐   │
│  │    Frontend    │  │   API Server   │  │     Worker       │   │
│  │    (nginx)     │  │  (Rust/Axum)   │  │  (Rust/Apalis)   │   │
│  │   Port: 8080   │  │   Port: 8080   │  │  No Public Port  │   │
│  │  Public HTTPS  │  │  Public HTTPS  │  │  Internal Only   │   │
│  └────────────────┘  └────────────────┘  └──────────────────┘   │
│          │                   │                    │             │
│          └───────────────────┼────────────────────┘             │
│                              │                                  │
│                   6PN Private Network (fdaa::/48)               │
│                              │                                  │
│         ┌────────────────────┼────────────────────┐             │
│         │                    │                    │             │
│  ┌──────┴───────┐    ┌───────┴───────┐     ┌──────┴──────┐      │
│  │   Managed    │    │    Upstash    │     │ Fly Metrics │      │
│  │  PostgreSQL  │    │     Redis     │     │  Dashboard  │      │
│  └──────────────┘    └───────────────┘     └─────────────┘      │
└─────────────────────────────────────────────────────────────────┘

Service Communication:

From	To	Network	URL Pattern
Frontend	API	Public HTTPS	`https://ampel-api.fly.dev/api`
API	PostgreSQL	Private 6PN	`postgres://ampel-db.flycast:5432`
API	Redis	Private 6PN	`redis://ampel-redis.flycast:6379`
Worker	PostgreSQL	Private 6PN	`postgres://ampel-db.flycast:5432`
Worker	Redis	Private 6PN	`redis://ampel-redis.flycast:6379`

Reference: Fly.io Private Networking

Prerequisites

Required Tools

Fly CLI (flyctl) - Install from fly.io/docs/flyctl/install:

# macOS/Linux
curl -L https://fly.io/install.sh | sh

# Windows (PowerShell)
pwsh -Command "iwr https://fly.io/install.ps1 -useb | iex"

# Homebrew
brew install flyctl

Docker - For local testing of deployment images
Fly.io Account - Sign up at fly.io

Authentication

# Login to Fly.io
fly auth login

# Verify authentication
fly auth whoami

Quick Start

For experienced users, here's the minimal deployment sequence:

# 1. Create organization and apps
fly orgs create ampel-org
fly apps create ampel-api --org ampel-org
fly apps create ampel-worker --org ampel-org
fly apps create ampel-frontend --org ampel-org

# 2. Create PostgreSQL database
fly postgres create --name ampel-db --org ampel-org --region iad

# 3. Create Redis
fly redis create --name ampel-redis --org ampel-org --region iad

# 4. Set secrets (see Secrets Management section for full list)
fly secrets set --app ampel-api \
  DATABASE_URL="postgres://..." \
  REDIS_URL="redis://..." \
  JWT_SECRET="$(openssl rand -hex 32)"

# 5. Deploy all services
fly deploy --config fly/fly.api.toml --remote-only
fly deploy --config fly/fly.worker.toml --remote-only
fly deploy --config fly/fly.frontend.toml --remote-only

Infrastructure Setup

1. Create Fly.io Organization

# Create organization
fly orgs create ampel-org

# List organizations
fly orgs list

# Set default organization
fly orgs select ampel-org

2. Provision Managed PostgreSQL

Fly.io Managed Postgres provides high-availability PostgreSQL with automatic backups.

fly postgres create \
  --name ampel-db \
  --org ampel-org \
  --region iad \
  --vm-size shared-cpu-1x \
  --volume-size 10

Configuration options:

Option	Recommended	Notes
`--region`	`iad`	Ashburn, VA - good for US users
`--vm-size`	`shared-cpu-1x`	Start small, scale as needed
`--volume-size`	`10`	10GB storage, expandable

After creation, save the connection credentials displayed. Connect and set up:

# Connect to database
fly postgres connect --app ampel-db

# Create database and extensions
CREATE DATABASE ampel;
\c ampel
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
CREATE EXTENSION IF NOT EXISTS "pg_trgm";
\q

Reference: Fly.io Managed Postgres

3. Provision Upstash Redis

Upstash Redis provides serverless Redis with per-request pricing.

fly redis create \
  --name ampel-redis \
  --org ampel-org \
  --region iad

Test the connection:

fly redis connect --app ampel-redis
PING
# Should return: PONG

Reference: Upstash for Redis on Fly.io

4. Create Application Entries

Create Fly.io apps without deploying:

# API Server
fly apps create ampel-api --org ampel-org

# Background Worker
fly apps create ampel-worker --org ampel-org

# Frontend
fly apps create ampel-frontend --org ampel-org

Application Deployment

Deployment Files

All deployment configuration files are in the fly/ directory:

fly/
├── fly.api.toml        # API server configuration
├── fly.worker.toml     # Worker configuration
└── fly.frontend.toml   # Frontend configuration

docker/
├── Dockerfile.api      # API multi-stage build
├── Dockerfile.worker   # Worker build
├── Dockerfile.frontend # Frontend build with nginx
├── docker-compose.yml  # Local development orchestration
└── nginx.conf          # Shared nginx config (used by both local and Fly.io)

Deploy API Server

# Deploy API
fly deploy --config fly/fly.api.toml --remote-only

# Verify deployment
fly status --app ampel-api
fly logs --app ampel-api

# Test health endpoint
curl https://ampel-api.fly.dev/health

Deploy Worker

# Deploy Worker
fly deploy --config fly/fly.worker.toml --remote-only

# Verify deployment
fly status --app ampel-worker
fly logs --app ampel-worker

Deploy Frontend

# Deploy Frontend (set API URL at build time)
fly deploy --config fly/fly.frontend.toml \
  --build-arg VITE_API_URL=https://ampel-api.fly.dev \
  --remote-only

# Verify deployment
fly status --app ampel-frontend
curl https://ampel-frontend.fly.dev

Run Database Migrations

# Run migrations via SSH
fly ssh console --app ampel-api -C "/app/ampel-api migrate run"

# Check migration status
fly ssh console --app ampel-api -C "/app/ampel-api migrate status"

Secrets Management

Fly.io stores secrets encrypted and injects them as environment variables at runtime.

Reference: Fly.io Secrets

Generate Secure Keys

# Generate 256-bit random keys
openssl rand -hex 32

API Server Secrets

fly secrets set --app ampel-api \
  DATABASE_URL="postgres://postgres:<PASSWORD>@ampel-db.flycast:5432/ampel" \
  REDIS_URL="redis://default:<PASSWORD>@ampel-redis.flycast:6379" \
  JWT_SECRET="<RANDOM_256_BIT_KEY>" \
  ENCRYPTION_KEY="<RANDOM_256_BIT_KEY>" \
  CORS_ORIGINS="https://ampel-frontend.fly.dev"

Note: Personal Access Tokens (PATs) are configured per-user through the UI after deployment. See PAT_SETUP.md for instructions.

Worker Secrets

fly secrets set --app ampel-worker \
  DATABASE_URL="postgres://postgres:<PASSWORD>@ampel-db.flycast:5432/ampel" \
  REDIS_URL="redis://default:<PASSWORD>@ampel-redis.flycast:6379" \
  ENCRYPTION_KEY="<SAME_AS_API>"

Verify Secrets

# List secret names (values are encrypted)
fly secrets list --app ampel-api

Bulk Import from File

Create .env.production (DO NOT COMMIT):

DATABASE_URL=postgres://postgres:xxx@ampel-db.flycast:5432/ampel
REDIS_URL=redis://default:xxx@ampel-redis.flycast:6379
JWT_SECRET=xxx
ENCRYPTION_KEY=xxx

Import:

fly secrets import --app ampel-api < .env.production

CI/CD with GitHub Actions

Setup Deploy Token

Generate a Fly.io deploy token:

fly tokens create deploy --name github-actions -x 999999h

Add to GitHub repository secrets:
- Go to: Repository → Settings → Secrets and variables → Actions
- Create secret: FLY_API_TOKEN
- Paste the token value (including FlyV1 prefix)

Deploy Workflow (`.github/workflows/deploy.yml`)

The deploy workflow is cost-conscious by design - it only triggers on pushes to the production branch (not main), preventing accidental deployments from development work.

Automatic Triggers:

Push to production branch with changes to:
- crates/**, frontend/**, fly/**
- docker/Dockerfile.*, Cargo.toml, Cargo.lock

Manual Dispatch Options:

Option	Description	Default
`environment`	Target environment (`production` or `staging`)	`production`
`deploy_api`	Deploy API server	`true`
`deploy_worker`	Deploy Worker service	`true`
`deploy_frontend`	Deploy Frontend	`true`
`run_migrations`	Run database migrations	`true`
`skip_tests`	Skip tests (use with caution)	`false`
`force_deploy`	Force deploy even without changes	`false`

Features:

Test jobs before deployment (backend + frontend)
Path-based triggers - only deploys changed components
Rolling deployment strategy for zero-downtime
Automatic database migrations after API deployment
Deployment verification with health checks
Multi-environment support - ampel-* (production) or ampel-staging-* (staging)

Required GitHub Secrets:

Secret	Description
`FLY_API_TOKEN`	Fly.io deploy token

Optional Secrets (for build-time injection):

Secret	Description
`VITE_API_URL`	API URL for frontend builds

Undeploy Workflow (`.github/workflows/undeploy.yml`)

The undeploy workflow allows you to destroy or scale down the Fly.io environment to save costs when not in use.

⚠️ This is a manual-only workflow - it requires explicit confirmation to prevent accidental destruction.

Options:

Option	Description	Default
`environment`	Target environment (`staging` or `production`)	`staging`
`destroy_api`	Destroy API server	`true`
`destroy_worker`	Destroy Worker service	`true`
`destroy_frontend`	Destroy Frontend	`true`
`destroy_database`	Destroy database (DANGEROUS - data loss!)	`false`
`confirm_destruction`	Must type "DESTROY" to proceed	Required
`scale_to_zero`	Scale to zero instead of destroying (cost-saving)	`false`

Scale to Zero Mode:

Instead of destroying apps, you can scale them to zero instances:

Stops all compute costs
Preserves app configuration and secrets
Can be restored by running the deploy workflow

# Equivalent manual commands
flyctl scale count 0 --app ampel-api --yes
flyctl scale count 0 --app ampel-worker --yes
flyctl scale count 0 --app ampel-frontend --yes

To restore after scale-to-zero:

Run the deploy workflow with force_deploy: true, or manually:

flyctl scale count 1 --app ampel-api
flyctl scale count 1 --app ampel-worker
flyctl scale count 1 --app ampel-frontend

Reference: Fly.io + GitHub Actions

Manual Deployment Script

Create scripts/deploy-fly.sh:

#!/bin/bash
set -e

echo "Deploying Ampel to Fly.io..."

echo "Deploying API..."
fly deploy --config fly/fly.api.toml --remote-only

echo "Running migrations..."
fly ssh console --app ampel-api -C "/app/ampel-api migrate run"

echo "Deploying Worker..."
fly deploy --config fly/fly.worker.toml --remote-only

echo "Deploying Frontend..."
fly deploy --config fly/fly.frontend.toml \
  --build-arg VITE_API_URL=https://ampel-api.fly.dev \
  --remote-only

echo "Deployment complete!"
echo "Frontend: https://ampel-frontend.fly.dev"
echo "API: https://ampel-api.fly.dev"

Custom Domain Setup

Add Custom Domains

# Frontend domain
fly certs add ampel.example.com --app ampel-frontend

# API domain
fly certs add api.ampel.example.com --app ampel-api

DNS Configuration

Add DNS records at your registrar:

Type	Name	Value
CNAME	`@` or `ampel`	`ampel-frontend.fly.dev`
CNAME	`api`	`ampel-api.fly.dev`

Update Configuration After Domain Setup

# Update CORS origins
fly secrets set --app ampel-api \
  CORS_ORIGINS="https://ampel.example.com,https://www.ampel.example.com"

Reference: Fly.io Custom Domains

Monitoring and Operations

View Logs

# Real-time logs
fly logs --app ampel-api -f

# Last 100 lines
fly logs --app ampel-api

# Filter by machine
fly logs --app ampel-api --machine <machine-id>

Check Application Status

# Status overview
fly status --app ampel-api

# Health checks
fly checks list --app ampel-api

# Machine details
fly machines list --app ampel-api

SSH Access

# Interactive shell
fly ssh console --app ampel-api

# Run single command
fly ssh console --app ampel-api -C "ls -la /app"

Database Operations

# Connect to PostgreSQL
fly postgres connect --app ampel-db

# Backup database
fly postgres db backup --app ampel-db

# List backups
fly postgres db list --app ampel-db

Restart Services

fly apps restart ampel-api
fly apps restart ampel-worker
fly apps restart ampel-frontend

Scaling

Horizontal Scaling (Instances)

# Scale API to 3 instances
fly scale count 3 --app ampel-api

# Scale Worker to 2 instances
fly scale count 2 --app ampel-worker

Vertical Scaling (Resources)

# Upgrade VM size
fly scale vm shared-cpu-2x --app ampel-api

# Increase memory
fly scale memory 1024 --app ampel-api

Regional Scaling

# Add instance in Frankfurt
fly scale count 1 --region fra --app ampel-api

# List available regions
fly platform regions

Auto-scaling Configuration

Auto-scaling is configured in fly.toml:

[http_service]
  auto_stop_machines = "stop"   # Stop idle machines
  auto_start_machines = true    # Start on demand
  min_machines_running = 1      # Minimum instances

Reference: Fly.io Scaling

Troubleshooting

App Won't Start

# Check logs for errors
fly logs --app ampel-api

# Verify secrets are set
fly secrets list --app ampel-api

# SSH and check binary
fly ssh console --app ampel-api -C "ls -la /app"

Database Connection Errors

# Verify DATABASE_URL uses .flycast address
fly secrets list --app ampel-api | grep DATABASE

# Test connection from app
fly ssh console --app ampel-api -C "psql \$DATABASE_URL -c 'SELECT 1;'"

# Check database status
fly status --app ampel-db

Health Checks Failing

# Test health endpoint
curl -v https://ampel-api.fly.dev/health

# Check from inside machine
fly ssh console --app ampel-api -C "curl localhost:8080/health"

# Review health check config in fly.toml

Rollback Deployment

# List releases
fly releases --app ampel-api

# Rollback to previous version
fly releases rollback --app ampel-api

# Rollback to specific version
fly releases rollback --app ampel-api --version 5

Cost Estimation

Monthly Costs (USD)

Service	Plan	Monthly Cost
Managed PostgreSQL	Shared-1x, 1GB	~$15-20
PostgreSQL Storage	10GB	~$2.50
Upstash Redis	Pay-per-request	~$0-10
API VM	shared-cpu-1x, 512MB	~$5-7
Worker VM	shared-cpu-1x, 512MB	~$5-7
Frontend VM	shared-cpu-1x, 256MB	~$3-5
Total		~$30-50/month

Notes:

Fly.io offers $5 free credit monthly for Hobby plan
VMs billed per-second when running
Auto-stop machines reduce costs significantly
Prices vary by region and actual usage

Reference: Fly.io Pricing

Security Checklist

Before going to production:

All secrets stored in Fly.io vault (not in code)
Database uses private .flycast address
HTTPS enforced (force_https = true)
CORS origins restricted to frontend domain
Non-root user in Dockerfiles
PAT encryption key securely generated and stored
Security headers configured in nginx
Database backups enabled
cargo audit run for vulnerabilities

References

Official Fly.io Documentation

Getting Started - Initial setup guide
Fly.io Configuration Reference - fly.toml specification
Managed Postgres - Database setup and management
Upstash Redis - Redis configuration
Private Networking - 6PN setup
Secrets Management - Secure configuration
Health Checks - Monitoring configuration
GitHub Actions Deployment - CI/CD setup
Rust on Fly.io - Rust deployment patterns
Static Sites - Frontend deployment

Community Resources

Deploying Rust Axum apps - Framework-specific guide
Fly.io Community Forum - Community support

File Reference

Deployment Configuration

File	Purpose
`fly/fly.api.toml`	API server Fly.io configuration
`fly/fly.worker.toml`	Worker Fly.io configuration
`fly/fly.frontend.toml`	Frontend Fly.io configuration
`docker/Dockerfile.api`	API multi-stage build with cargo-chef
`docker/Dockerfile.worker`	Worker multi-stage build
`docker/Dockerfile.frontend`	Frontend build with nginx
`docker/nginx.conf`	Shared nginx config (local dev + Fly.io production)
`docker/docker-compose.yml`	Local development orchestration

CI/CD

File	Purpose
`.github/workflows/deploy.yml`	Deploy to Fly.io (triggers on `production`)
`.github/workflows/undeploy.yml`	Destroy or scale-to-zero Fly.io environment

Supplementary Documentation

File	Purpose
`docs/deployment/RUNBOOK.md`	Detailed operations runbook
`docs/deployment/SECRETS_TEMPLATE.md`	Secrets configuration templates

Local Development (`docker/`)

File	Purpose
`docker/docker-compose.yml`	Local development environment
`docker/Dockerfile.api`	Local API build (with BuildKit cache)
`docker/Dockerfile.worker`	Local worker build
`docker/Dockerfile.frontend`	Local frontend build

FilesExpand file tree

DEPLOY.md

Latest commit

History