ADR: Docker-Based Local Development with LocalStack

[jimmoffet]

Context

The Notify.gov API currently relies on Cloud Foundry (cloud.gov) for deployment and requires terraform/development/run.sh to provision local AWS credentials for development. This setup has several limitations:

1. Environment Parity Gap: Local development does not mirror the full production stack - developers run Flask and Celery directly via make run-procfile without containerization
2. Cloud Dependency: Developers need real AWS credentials provisioned via Terraform for S3, SES, and SNS even during local development
3. Onboarding Friction: New team members must navigate complex setup steps (make bootstrap, Terraform provisioning) before sending their first test notification
4. Testing Gaps: Integration tests against AWS services are limited because they require real cloud credentials
5. Migration Preparation: As part of the broader infrastructure migration from Cloud Foundry to AWS (App Runner/ECS), we need Docker-based local development as the foundation for containerized deployments

The AWS clients (app/clients/sms/aws_sns.py and app/clients/email/aws_ses.py) already use boto3 for AWS communication. The SNS client already supports LOCALSTACK_ENDPOINT_URL for local testing, but the SES client does not have this capability.

Decision

We will implement Docker-based local development using LocalStack to simulate AWS services. This involves:

1. Multi-stage Dockerfile
We will create a multi-stage Dockerfile with distinct build targets:

• base: Common Python dependencies and application code
• web: Flask/Gunicorn web server (port 6011)
• worker: Celery worker with gevent pool
• scheduler: Celery beat scheduler

2. Docker Compose Service Composition
A docker-compose.yml will orchestrate the full stack:

• db: PostgreSQL 15 for notification data
• redis: Redis 7.0 for Celery task queue and caching
• localstack: LocalStack container with S3, SES, and SNS services
• api: Flask web server (build target: web)
• worker: Celery worker (build target: worker)
• scheduler: Celery beat (build target: scheduler)

3. LocalStack Initialization
An initialization script (docker/localstack/init-aws.sh) will provision:

• S3 bucket for CSV uploads (notify-csv-uploads)
• SNS topic for SMS delivery receipts
• SES verified email identity for local testing

4. boto3 endpoint_url Pattern
We will modify app/clients/email/aws_ses.py to support LocalStack by checking for LOCALSTACK_ENDPOINT_URL environment variable and passing it as endpoint_url to the boto3 client, matching the existing pattern in aws_sns.py.

5. Makefile Targets
New targets for Docker operations:

• docker-up: Start full Docker stack
• docker-down: Stop Docker stack
• docker-build: Build Docker images
• docker-logs: View Docker logs
• docker-migrate: Run database migrations

Why Docker + LocalStack over current approach:

• Environment parity: Containers match production deployment model
• Zero cloud dependency: No AWS credentials needed for local development
• Fast onboarding: Single make docker-up starts entire stack
• Comprehensive testing: Full AWS service simulation enables integration testing
• Migration path: Docker images become the deployment artifact for AWS App Runner/ECS

Alternatives not chosen:

• moto library only: Requires code changes for each test, doesn't provide running services
• Terraform localstack provider: Adds complexity without Docker orchestration benefits
• AWS SAM Local: Designed for Lambda, not suitable for long-running Flask/Celery services

Consequences

Positive:

• Developers can run the full notification stack with make docker-up and no cloud credentials
• New team members can be productive within minutes of cloning the repository
• Integration tests can run against LocalStack services in CI/CD pipelines
• Docker images created for local development become the production deployment artifacts
• Environment consistency reduces "works on my machine" issues
• LocalStack provides realistic AWS service behavior including error conditions and throttling simulation

Negative:

• Docker and Docker Compose become development prerequisites (most developers already have these)
• LocalStack has some behavioral differences from real AWS services (e.g., no actual email delivery)
• Additional disk space required for Docker images (estimated ~2GB for full stack)
• Developers must learn Docker Compose commands alongside existing Makefile targets

Neutral:

• The existing make bootstrap and native Python development workflow will remain available for developers who prefer it
• This change does not modify production deployment - it only affects local development

Author

@jim

Stakeholders

@ccostino @stvnrlly @kenkehl

Next Steps

Once accepted, implementation will proceed in these phases:

1. Create Docker infrastructure (Phase 1 of AWS migration plan)

   • Create docker/Dockerfile with multi-stage builds
   • Create docker/docker-compose.yml with all services
   • Create docker/localstack/init-aws.sh initialization script
   • Create .env.docker.example template file
   • Add docker-* targets to Makefile

2. Modify application code for LocalStack support

   • Update app/clients/email/aws_ses.py to support LOCALSTACK_ENDPOINT_URL
   • Verify S3 client supports LocalStack endpoint (if needed)

3. Documentation updates

   • Update README with Docker development instructions
   • Update CLAUDE.md with Docker workflow guidance