Skip to content

DEMO.md

Latest commit

 

History

History
845 lines (652 loc) · 22.1 KB

DEMO.md

File metadata and controls

845 lines (652 loc) · 22.1 KB

🚀 Terragrunt vs Vanilla Terraform: A Practical Demo

Executive Summary

This document demonstrates why Terragrunt is superior to vanilla Terraform for managing multi-environment infrastructure. Using real examples from this repository, we'll show how Terragrunt reduces configuration by 70%, eliminates duplication, and makes infrastructure management significantly easier.

The Problem: Vanilla Terraform Pain Points

1. Massive Code Duplication

With vanilla Terraform, managing 3 environments (dev, staging, prod) requires:

terraform/
├── dev/
│   ├── backend.tf          # Duplicated 3x
│   ├── provider.tf         # Duplicated 3x
│   ├── terraform.tfvars    # 50+ lines each
│   ├── vpc/
│   │   ├── backend.tf      # Duplicated 9x total
│   │   ├── provider.tf     # Duplicated 9x total
│   │   └── main.tf
│   ├── ec2/
│   │   ├── backend.tf
│   │   ├── provider.tf
│   │   └── main.tf
│   └── s3/
│       ├── backend.tf
│       ├── provider.tf
│       └── main.tf
├── staging/
│   └── [same structure, all duplicated]
└── prod/
    └── [same structure, all duplicated]

Result: ~500+ lines of duplicated configuration across 27+ files

2. Manual Backend Setup

Before running Terraform, you must:

  1. Manually create S3 buckets for each environment
  2. Manually create DynamoDB tables for state locking
  3. Configure encryption, versioning, and tags
  4. Update backend.tf files with bucket names
  5. Repeat for every environment

3. Dependency Management Nightmare

# In ec2/main.tf - manually reference remote state
data "terraform_remote_state" "vpc" {
  backend = "s3"
  config = {
    bucket = "my-terraform-state-dev"  # Hardcoded!
    key    = "vpc/terraform.tfstate"
    region = "us-east-1"
  }
}

# Use the output
vpc_id = data.terraform_remote_state.vpc.outputs.vpc_id

Problems:

  • Hardcoded bucket names
  • Manual state file path management
  • No automatic dependency ordering
  • Easy to break with typos

4. Environment Configuration Hell

# dev/terraform.tfvars
environment        = "dev"
vpc_cidr           = "10.10.0.0/16"
availability_zones = ["us-east-1a"]
instance_type      = "t2.micro"
instance_count     = 1
enable_versioning  = false

# staging/terraform.tfvars
environment        = "staging"
vpc_cidr           = "10.20.0.0/16"
availability_zones = ["us-east-1a", "us-east-1b"]
instance_type      = "t2.micro"
instance_count     = 2
enable_versioning  = true

# prod/terraform.tfvars
environment        = "prod"
vpc_cidr           = "10.30.0.0/16"
availability_zones = ["us-east-1a", "us-east-1b"]
instance_type      = "t2.micro"
instance_count     = 2
enable_versioning  = true

Problems:

  • Duplicated across 3+ files
  • Easy to have inconsistencies
  • Changing a value requires editing multiple files
  • No single source of truth

The Solution: Terragrunt

1. Zero Duplication with Configuration Maps

One file to rule them all: root.hcl

# Define ALL environment configurations in ONE place
locals {
  # Auto-detect environment from directory path
  parsed_path = regex(".*/environments/(?P<env>[^/]+)/.*", get_terragrunt_dir())
  environment = local.parsed_path.env
  
  # Configuration maps - the magic sauce!
  vpc_cidr_map = {
    dev     = "10.10.0.0/16"
    staging = "10.20.0.0/16"
    prod    = "10.30.0.0/16"
  }
  
  instance_count_map = {
    dev     = 1
    staging = 2
    prod    = 2
  }
  
  # Automatically select values based on environment
  vpc_cidr       = local.vpc_cidr_map[local.environment]
  instance_count = local.instance_count_map[local.environment]
}

Benefits:

  • Change dev VPC CIDR? Edit ONE line in ONE file
  • Add new environment? Add ONE entry to each map
  • No duplication, no inconsistencies
  • Single source of truth

2. Automatic Backend Creation

# In root.hcl - Terragrunt creates everything automatically
remote_state {
  backend = "s3"
  
  config = {
    bucket         = "terragrunt-demo-state-${local.environment}-${get_aws_account_id()}"
    key            = "${path_relative_to_include()}/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terragrunt-demo-locks-${local.environment}"
  }
}

What Terragrunt does automatically:

  • Creates S3 bucket if it doesn't exist
  • Creates DynamoDB table if it doesn't exist
  • Enables encryption
  • Applies proper tags
  • Configures versioning
  • Sets up state locking

You do: Nothing. Just run terragrunt apply

3. Minimal Environment Files

Vanilla Terraform: 50+ lines per environment file

Terragrunt: 8 lines per environment file

# environments/dev/vpc/terragrunt.hcl - That's it!
include "root" {
  path = find_in_parent_folders("root.hcl")
}

terraform {
  source = "../../../modules/vpc"
}

# Everything else (backend, provider, inputs) inherited from root.hcl!

4. Elegant Dependency Management

# environments/dev/ec2/terragrunt.hcl
include "root" {
  path = find_in_parent_folders("root.hcl")
}

terraform {
  source = "../../../modules/ec2"
}

# Declare dependency - Terragrunt handles the rest
dependency "vpc" {
  config_path = "../vpc"
}

inputs = {
  vpc_id    = dependency.vpc.outputs.vpc_id
  subnet_id = dependency.vpc.outputs.public_subnet_ids[0]
}

Benefits:

  • No hardcoded bucket names
  • No manual state file paths
  • Automatic dependency ordering
  • Type-safe output references
  • Terragrunt ensures VPC is created before EC2

5. Auto-Generated Provider Configuration

# In root.hcl - generate provider for ALL modules
generate "provider" {
  path      = "provider.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
provider "aws" {
  region = "${local.aws_region}"
  
  default_tags {
    tags = ${jsonencode(local.common_tags)}
  }
}
EOF
}

Result: Every module automatically gets:

  • AWS provider configuration
  • Common tags applied to all resources
  • No duplication needed

Side-by-Side Comparison

Scenario: Change Dev Environment Instance Count

Vanilla Terraform:

  1. Open dev/terraform.tfvars
  2. Change instance_count = 1 to instance_count = 3
  3. Open dev/ec2/terraform.tfvars (if separate)
  4. Change instance_count = 1 to instance_count = 3
  5. Verify consistency across files
  6. Run terraform apply

Terragrunt:

  1. Open root.hcl
  2. Change dev = 1 to dev = 3 in instance_count_map
  3. Run terragrunt apply

Scenario: Add New Environment (QA)

Vanilla Terraform:

  1. Copy entire dev/ directory to qa/
  2. Update backend.tf in 3+ places with new bucket name
  3. Update provider.tf in 3+ places
  4. Create new terraform.tfvars with QA values
  5. Manually create S3 bucket for QA state
  6. Manually create DynamoDB table for QA locks
  7. Update all hardcoded references
  8. ~30 minutes of work, high error risk

Terragrunt:

  1. Copy environments/dev/ to environments/qa/
  2. Add QA entries to maps in root.hcl:
vpc_cidr_map = {
  dev     = "10.10.0.0/16"
  staging = "10.20.0.0/16"
  prod    = "10.30.0.0/16"
  qa      = "10.40.0.0/16"  # Add this line
}
  1. Run terragrunt apply
  2. ~5 minutes of work, zero errors

Scenario: Apply Common Tags to All Resources

Vanilla Terraform:

  1. Update provider configuration in 9+ files
  2. Or create complex locals in each module
  3. Ensure consistency across all files
  4. High risk of missing files

Terragrunt:

  1. Update common_tags in root.hcl
  2. Done. All modules inherit automatically

Real Numbers from This Repository

Metric Vanilla Terraform Terragrunt Savings
Total config files 27+ files 10 files 63% fewer
Lines of config ~500 lines ~150 lines 70% reduction
Backend configs 9 files 1 block 89% reduction
Provider configs 9 files 1 block 89% reduction
Environment vars 3 files × 50 lines 1 file with maps 85% reduction
Time to add environment 30 minutes 5 minutes 83% faster
Manual AWS setup Required Automatic 100% eliminated

Powerful Terragrunt Features in Action

1. Built-in Functions

# Get AWS account ID dynamically
bucket = "state-${get_aws_account_id()}"

# Parse directory path
environment = regex(".*/environments/(?P<env>[^/]+)/.*", get_terragrunt_dir()).env

# Find parent configuration
path = find_in_parent_folders("root.hcl")

# Get relative path for state key
key = "${path_relative_to_include()}/terraform.tfstate"

2. Run-all Commands

# Apply all modules in correct dependency order
terragrunt run-all apply

# Plan all modules
terragrunt run-all plan

# Destroy all modules in reverse dependency order
terragrunt run-all destroy

# Initialize all modules
terragrunt run-all init

Vanilla Terraform equivalent:

cd dev/vpc && terraform apply && cd ../ec2 && terraform apply && cd ../s3 && terraform apply
cd staging/vpc && terraform apply && cd ../ec2 && terraform apply && cd ../s3 && terraform apply
cd prod/vpc && terraform apply && cd ../ec2 && terraform apply && cd ../s3 && terraform apply

3. Dependency Graph

# Visualize dependencies
terragrunt graph-dependencies

Output:

vpc
├── ec2 (depends on vpc)
└── s3

Terragrunt automatically applies in correct order!

Common Objections Addressed

"Terragrunt adds complexity"

Reality: Terragrunt reduces complexity by eliminating duplication. Initial learning curve is small compared to long-term maintainability gains.

  • Learning Terragrunt basics: 1-2 hours
  • Time saved per month: 5-10 hours
  • ROI: Positive within first week

"We already have Terraform working"

Reality: Migration is straightforward and can be done incrementally.

  1. Create root.hcl with common configuration
  2. Migrate one environment at a time
  3. Keep existing Terraform modules unchanged
  4. Typical migration time: 1-2 days for small projects

"What if Terragrunt is abandoned?"

Reality: Terragrunt is:

  • Maintained by Gruntwork (established company)
  • Used by thousands of companies
  • Open source with active community
  • Worst case: It's a thin wrapper, easy to migrate back

"Our team doesn't know Terragrunt"

Reality: If your team knows Terraform, they can learn Terragrunt in hours.

  • Same HCL syntax
  • Same Terraform modules
  • Just adds DRY principles
  • Better practices from day one

Live Demo: See It In Action

Step 1: Deploy Dev Environment

cd environments/dev
terragrunt run-all init
terragrunt run-all apply

What happens:

  1. Terragrunt creates S3 bucket: terragrunt-demo-state-dev-123456789
  2. Terragrunt creates DynamoDB table: terragrunt-demo-locks-dev
  3. Terragrunt generates provider.tf in each module
  4. Terragrunt generates backend.tf in each module
  5. Terragrunt applies VPC first (no dependencies)
  6. Terragrunt applies EC2 second (depends on VPC)
  7. Terragrunt applies S3 third (no dependencies)

Total time: ~3 minutes Manual steps: 0

Step 2: Deploy Staging with Different Config

cd environments/staging
terragrunt run-all apply

What's different automatically:

  • VPC CIDR: 10.20.0.0/16 (vs dev's 10.10.0.0/16)
  • Instance count: 2 (vs dev's 1)
  • Availability zones: 2 (vs dev's 1)
  • S3 versioning: enabled (vs dev's disabled)
  • Separate state bucket and lock table

Configuration changes needed: 0 (all in maps!)

Step 3: Make a Change Across All Environments

# Edit root.hcl - change instance_type for all environments
instance_type_map = {
  dev     = "t2.small"  # Changed from t2.micro
  staging = "t2.small"  # Changed from t2.micro
  prod    = "t2.small"  # Changed from t2.micro
}

# Apply to all environments
cd environments/dev && terragrunt apply
cd ../staging && terragrunt apply
cd ../prod && terragrunt apply

Files edited: 1 Consistency: Guaranteed

Cost-Benefit Analysis

Time Investment

Activity Vanilla Terraform Terragrunt Savings
Initial setup 2 hours 3 hours -1 hour
Add environment 30 min 5 min 25 min
Change config 15 min 2 min 13 min
Debug state issues 1 hour 10 min 50 min
Monthly maintenance 4 hours 1 hour 3 hours

Break-even point: After 2-3 environments or 1 month of use

Risk Reduction

  • 70% less code = 70% fewer places for bugs
  • Automatic backend setup = no manual errors
  • Dependency management = no ordering mistakes
  • Single source of truth = no inconsistencies

Getting Started with This Demo

Prerequisites

# Install Terragrunt
# Windows (using Chocolatey)
choco install terragrunt

# Or download from: https://github.com/gruntwork-io/terragrunt/releases

# Verify installation
terragrunt --version
terraform --version

Quick Start

# 1. Clone this repository
git clone <your-repo>
cd <your-repo>

# 2. Configure AWS credentials
aws configure

# 3. Deploy dev environment
cd environments/dev
terragrunt run-all init
terragrunt run-all plan
terragrunt run-all apply

# 4. See the magic!
# - Check AWS console for auto-created S3 bucket
# - Check DynamoDB for auto-created lock table
# - Check EC2 for instances with proper tags

Explore the Configuration

# View dependency graph
terragrunt graph-dependencies

# See what Terragrunt will do
terragrunt run-all plan

# Apply specific module
cd vpc
terragrunt apply

# Destroy everything
cd environments/dev
terragrunt run-all destroy

Advanced: Multi-Sovereign-Cloud Deployments

The Challenge: Same Code, Different Compliance Requirements

Imagine deploying the same infrastructure across:

  • Commercial AWS (standard compliance)
  • US GovCloud (FedRAMP requirements)
  • Canada GovCloud (PBMM requirements)

Each sovereign cloud has different:

  • AWS regions
  • Compliance tags
  • Encryption requirements
  • Audit logging retention
  • Access control policies

Vanilla Terraform approach: Duplicate entire codebase 3 times, maintain separately

Terragrunt approach: One codebase, runtime injection of sovereign cloud configs

The Injection Pattern

This repository demonstrates a powerful pattern for multi-sovereign-cloud deployments:

  1. Base repo contains minimal, cloud-agnostic configuration
  2. GitLab CI in each sovereign cloud injects compliance-specific configs
  3. Terragrunt merges base + injection at runtime
  4. Modules remain completely unchanged

How It Works

Step 1: Base Configuration (root.hcl)

locals {
  # Load injection config if it exists (created by GitLab CI)
  injection_config_path = "${get_parent_terragrunt_dir()}/injection.hcl"
  injection = fileexists(local.injection_config_path) ? read_terragrunt_config(local.injection_config_path) : null
  
  # Base configuration works everywhere
  vpc_cidr_map = {
    dev     = "10.10.0.0/16"
    staging = "10.20.0.0/16"
    prod    = "10.30.0.0/16"
  }
  
  # Merge with injection
  aws_region = try(local.injection.locals.aws_region, "us-east-1")
  
  common_tags = merge(
    local.base_tags,
    try(local.injection.locals.additional_tags, {})
  )
}

Step 2: Sovereign Cloud Injections (injections/)

Commercial AWS:

# injections/commercial.hcl.example
locals {
  aws_region = "us-east-1"
  additional_tags = {
    CostCenter = "Engineering"
  }
}

US GovCloud (FedRAMP):

# injections/us-govcloud.hcl.example
locals {
  aws_region = "us-gov-west-1"
  
  additional_tags = {
    Compliance         = "FedRAMP-Moderate"
    DataClassification = "CUI"
    ITAR               = "true"
  }
  
  provider_config = <<-EOT
    assume_role {
      role_arn = "arn:aws-us-gov:iam::123456789012:role/TerraformRole"
    }
  EOT
  
  additional_inputs = {
    encryption_algorithm  = "aws:kms"
    enable_cloudtrail     = true
    log_retention_days    = 2555  # 7 years
    block_public_access   = true
  }
}

Canada GovCloud (PBMM):

# injections/canada-govcloud.hcl.example
locals {
  aws_region = "ca-central-1"
  
  additional_tags = {
    Compliance         = "PBMM"
    DataClassification = "Protected-B"
    DataResidency      = "Canada"
  }
  
  additional_inputs = {
    encryption_algorithm  = "aws:kms"
    data_residency        = "ca-central-1"
    log_retention_days    = 2555
  }
}

Step 3: GitLab CI Injection

Each sovereign cloud's GitLab instance runs:

# Commercial GitLab
before_script:
  - cp injections/commercial.hcl.example injection.hcl

# US GovCloud GitLab
before_script:
  - cp injections/us-govcloud.hcl.example injection.hcl

# Canada GovCloud GitLab
before_script:
  - cp injections/canada-govcloud.hcl.example injection.hcl

Real-World Scenario: Deploy to All Clouds

Vanilla Terraform:

  1. Maintain 3 separate repositories
  2. Update each repo independently
  3. Ensure consistency manually
  4. 3x the maintenance burden

Terragrunt with Injection:

  1. Update base repo once
  2. Push to all GitLab instances
  3. Each CI pipeline injects appropriate config
  4. Deploy with confidence

What Gets Injected

The injection pattern supports:

  1. AWS Region Override

    aws_region = "us-gov-west-1"

  2. Compliance Tags

    additional_tags = {
  Compliance = "FedRAMP-Moderate"
  ITAR       = "true"
}

  3. Provider Configuration

    provider_config = <<-EOT
  assume_role {
    role_arn = "arn:aws-us-gov:iam::123:role/TerraformRole"
  }
EOT

  4. Backend Overrides

    backend_overrides = {
  kms_key_id = "arn:aws-us-gov:kms:us-gov-west-1:123:key/fedramp-key"
}

  5. Compliance Inputs

    additional_inputs = {
  enable_cloudtrail    = true
  log_retention_days   = 2555
  encryption_algorithm = "aws:kms"
}

Benefits of the Injection Pattern

  1. Single Source of Truth: One base repo for all clouds
  2. Zero Duplication: Modules unchanged across deployments
  3. Compliance Separation: Sovereign cloud configs isolated
  4. Easy Auditing: Each cloud's requirements clearly visible
  5. Secure: injection.hcl is gitignored, created at runtime
  6. Scalable: Add new sovereign clouds without touching base repo

Comparison: Multi-Cloud Deployment

Approach Repositories Maintenance Consistency Compliance Isolation
Vanilla Terraform 3 separate repos 3x effort Manual Mixed with code
Terragrunt (no injection) 1 repo with if/else Complex conditionals Guaranteed Mixed with code
Terragrunt + Injection 1 base repo 1x effort Guaranteed Clean separation

Adding a New Sovereign Cloud

Vanilla Terraform: Fork entire repo, modify everywhere

Terragrunt + Injection:

  1. Create injections/new-cloud.hcl.example
  2. Define region and compliance requirements
  3. Update GitLab CI in that cloud
  4. Done. Base repo unchanged.

Time: 10 minutes vs 2+ hours

Security Considerations

  1. injection.hcl is gitignored
  2. Never commit secrets to injection files
  3. Use CI/CD variables for sensitive values
  4. ARNs and account IDs templated in examples
  5. Each sovereign cloud controls its own injection

Try It Yourself

# Simulate commercial deployment
cp injections/commercial.hcl.example injection.hcl
cd environments/dev
terragrunt run-all plan

# Simulate US GovCloud deployment
cp injections/us-govcloud.hcl.example injection.hcl
cd environments/dev
terragrunt run-all plan
# Notice: Different region, tags, and compliance settings!

# Simulate Canada deployment
cp injections/canada-govcloud.hcl.example injection.hcl
cd environments/dev
terragrunt run-all plan

Real Numbers: Multi-Sovereign-Cloud

Metric Vanilla Terraform Terragrunt + Injection
Repositories 3 1
Code duplication 300% 0%
Compliance updates 3 files each 1 injection file
Time to add cloud 2+ hours 10 minutes
Consistency risk High Zero
Audit complexity High Low

Conclusion: Why Terragrunt Wins

Quantifiable Benefits

  1. 70% less configuration code
  2. 83% faster environment provisioning
  3. 100% elimination of manual backend setup
  4. Zero configuration duplication
  5. Automatic dependency management
  6. Multi-sovereign-cloud support with zero code duplication

Qualitative Benefits

  1. Easier to onboard new team members
  2. Reduced cognitive load
  3. Fewer bugs and inconsistencies
  4. Better sleep at night
  5. More time for actual infrastructure work
  6. Clean separation of compliance requirements

The Bottom Line

Terragrunt doesn't replace Terraform—it makes Terraform better. It's the DRY principle applied to infrastructure as code. If you're managing multiple environments or modules, Terragrunt pays for itself immediately.

For multi-sovereign-cloud deployments, Terragrunt's injection pattern is a game-changer. Deploy the same infrastructure to commercial AWS, US GovCloud, and Canada GovCloud from a single codebase, with each cloud's compliance requirements cleanly isolated.

Next Steps

  1. Try this demo: Deploy the dev environment and see how easy it is
  2. Read the code: Check out root.hcl to see the configuration maps
  3. Experiment: Add a new environment or change a configuration value
  4. Try multi-cloud: Test the injection pattern with different sovereign clouds
  5. Migrate: Start migrating your existing Terraform to Terragrunt
  6. Evangelize: Show your team how much time they'll save

Resources

Ready to convince your team? Show them this demo. The numbers speak for themselves.

Questions? The configuration in this repository is production-ready and follows best practices. Use it as a template for your own infrastructure.

Still skeptical? Run terragrunt run-all apply in the dev environment and watch Terragrunt create everything automatically. You'll be convinced in 5 minutes.