Bootstrap Infrastructure and CI

This guide walks you through the one-time setup required to deploy FilmDrop infrastructure. Follow these steps once per AWS account to create the foundational infrastructure.

CI/CD Platform Note: This guide is optimized for GitHub Actions. If you're using a different CI/CD platform (GitLab CI, CircleCI, Jenkins, etc.), you can still use this guide - simply skip or adapt the GitHub-specific sections as needed.

Prerequisites
AWS Infrastructure (Required for all platforms)
- Terraform State Bucket
- Deployment IAM Role
Service-Linked Roles
GitHub Actions Configuration
- Repository Secrets
Common CI/CD Platform Examples

Prerequisites

AWS CLI installed and configured
Appropriate AWS credentials with permissions to create CloudFormation stacks, S3 buckets, IAM roles, and service linked roles
Important: Each environment (e.g., dev, staging, prod) should be deployed to different AWS accounts

AWS Infrastructure (Required for all platforms)

This section creates the foundational AWS resources needed for Terraform deployments. We use CloudFormation templates to automate this setup:

cloudformation/terraform-state-bucket.yaml - Creates the S3 bucket for Terraform state storage
cloudformation/github-oidc-role.yaml - Creates the IAM role for GitHub Actions OIDC authentication

Terraform State Bucket

Overview

FilmDrop resources are deployed as Terraform modules with S3-backed state management. This S3 bucket stores the Terraform state files and must be created once per AWS account before deploying any FilmDrop infrastructure.

Deploy State Bucket Stack

Update the values in the commands below and execute them in sequence.

# Configure AWS credentials for the target account
# aws configure --profile dev
# export AWS_PROFILE=dev

# IMPORTANT - Set your values
AWS_REGION=""    # Change this to the region where the bucket should live
PROJECT_NAME=""  # Change this to your project name
ENVIRONMENT=""   # Change to: dev, staging, prod, etc.

STACK_NAME="filmdrop-${PROJECT_NAME}-${ENVIRONMENT}-terraform-state-${AWS_REGION}"
BUCKET_NAME="filmdrop-${PROJECT_NAME}-${ENVIRONMENT}-terraform-state-${AWS_REGION}"

# Deploy the CloudFormation stack
aws cloudformation create-stack \
  --stack-name $STACK_NAME \
  --template-body file://bootstrap/cloudformation/terraform-state-bucket.yaml \
  --parameters \
    ParameterKey=BucketName,ParameterValue=$BUCKET_NAME \
    ParameterKey=ProjectName,ParameterValue=$PROJECT_NAME \
  --region $AWS_REGION

# Wait for stack creation to complete
aws cloudformation wait stack-create-complete \
  --stack-name $STACK_NAME \
  --region $AWS_REGION

# Get the bucket name from stack outputs
echo "State bucket created:"
aws cloudformation describe-stacks \
  --stack-name $STACK_NAME \
  --region $AWS_REGION \
  --query 'Stacks[0].Outputs[?OutputKey==`BucketName`].OutputValue' \
  --output text

Terraform Backend Configuration

After creating the state bucket, create an environment-specific backend configuration file to reference it.

Create Backend Configuration File:

Create a backend config file (e.g., backends/dev.s3.backend.hcl):

bucket         = ""  # Update
key            = ""  # Update
region         = ""  # Update
use_lockfile   = true
encrypt        = true

Set bucket to the bucket name from the CloudFormation stack output
Set key to match your environment name, e.g., dev.tfstate
Set region to match your state bucket region

Note: To deploy for additional environments, such as staging or prod, change the ENVIRONMENT variable to staging or prod, configure the appropriate AWS credentials, and rerun the commands above. Create corresponding backend files (e.g., backends/staging.s3.backend.hcl, backends/prod.s3.backend.hcl).

Update State Bucket Stack (Optional)

To update an existing state bucket stack after making any changes to the template:

# Configure AWS credentials for the target account
# aws configure --profile dev
# export AWS_PROFILE=dev

# Set your values
AWS_REGION=""    # Change this to your region; it does not need to match where FilmDrop will be deployed but should for best practices
PROJECT_NAME=""  # Change this to your project name
ENVIRONMENT=""   # Change to: dev, staging, prod, etc.

STACK_NAME="filmdrop-${PROJECT_NAME}-${ENVIRONMENT}-terraform-state-${AWS_REGION}"
BUCKET_NAME="filmdrop-${PROJECT_NAME}-${ENVIRONMENT}-terraform-state-${AWS_REGION}"

aws cloudformation update-stack \
  --stack-name $STACK_NAME \
  --template-body file://bootstrap/cloudformation/terraform-state-bucket.yaml \
  --parameters \
    ParameterKey=BucketName,ParameterValue=$BUCKET_NAME \
    ParameterKey=ProjectName,ParameterValue=$PROJECT_NAME \
  --region $AWS_REGION

Note: To update for staging or prod environments, change the ENVIRONMENT variable, configure the appropriate AWS credentials, and rerun the command above.

Deployment IAM Role

Overview

The deployment IAM role is assumed by your CI/CD platform to deploy FilmDrop infrastructure.

GitHub Actions Configuration: This guide creates an OIDC-based IAM role configured for GitHub Actions workflows. The role can be assumed by workflows running on the main branch and pull requests, providing a secure, keyless authentication mechanism. This is naturally not the only option.

Alternative Configurations:

Different CI/CD platform: If you're using a different CI/CD platform, you'll need to modify the IAM role trust policy in the cloudformation/github-oidc-role.yaml CloudFormation template to align with your platform's OIDC provider or authentication method.
Using an existing deployment role: If your organization already has a deployment role configured and managed outside of this guide, you can skip creating this role. Ensure the existing role has the necessary permissions by comparing its policy to the one in the cloudformation/github-oidc-role.yaml CloudFormation template.
Restricting to specific branches: The default trust policy allows workflows from the main branch and pull requests. To restrict to specific branches or tags, modify the StringLike condition in the trust policy.

Note on Permissions: The IAM role has broad permissions across many AWS services to support the full range of FilmDrop components without being overly specific. Review these permissions for your security requirements.

IAM Role Permissions Review

IMPORTANT: Before deploying to production or making this repository public, you must review the IAM permissions in cloudformation/github-oidc-role.yaml.

The default role grants wildcard (*) permissions for the following services:

ACM, API Gateway, AOSS, AutoScaling, Application AutoScaling, Batch
CloudFront, CloudWatch, CodeBuild, DynamoDB, EC2, ECR, ECS, EKS
EFS, ELB, ElasticSearch, EventBridge, Kinesis Firehose, KMS
Lambda, CloudWatch Logs, Route53, S3, Secrets Manager
SNS, SQS, Step Functions, TimeStream, WAFv2

Plus explicit IAM and SSM management permissions.

Security Review Checklist:

Identify Required Services: Review your FilmDrop configuration to determine which AWS services you actually use
Remove Unused Services: Delete permissions for services you don't need

Add Resource Restrictions: Where possible, restrict permissions to specific resources:

# Instead of:
- Effect: Allow
  Action: "s3:*"
  Resource: "*"

# Consider:
- Effect: Allow
  Action: "s3:*"
  Resource:
    - "arn:aws:s3:::my-filmdrop-bucket"
    - "arn:aws:s3:::my-filmdrop-bucket/*"

Implement Permission Boundaries: Consider using IAM permission boundaries for additional protection
Use Separate Roles: Use different roles with different permissions for dev/staging/prod environments
Document Changes: Keep a record of why permissions were added or removed
Regular Audits: Review permissions quarterly or when adding new FilmDrop components

Alternative Approach: If your organization has established least-privilege policies, consider:

Creating custom IAM policies aligned with your security requirements
Using AWS IAM Access Analyzer to identify unused permissions
Implementing Service Control Policies (SCPs) in AWS Organizations

Prerequisites

Before deploying the IAM role, ensure you have:

GitHub OIDC Provider configured in your AWS account (one-time setup per account)
- Follow GitHub's OIDC documentation for AWS
- To retrieve an existing provider ARN:
```
aws iam list-open-id-connect-providers --query "OpenIDConnectProviderList[?contains(Arn, 'token.actions.githubusercontent.com')].Arn" --output text
```
GitHub repository information: Organization name and repository name
State bucket created (see previous section)

Deploy IAM Role Stack

Update the values in the commands below and execute them in sequence.

# Configure AWS credentials for the target account
# aws configure --profile dev
# export AWS_PROFILE=dev

# Set your values
AWS_REGION=""    # Change this to your state bucket region
PROJECT_NAME=""  # Change this to your project name (max 8 characters)
ENVIRONMENT=""   # Change to: dev, staging, prod, etc.

STACK_NAME="filmdrop-${PROJECT_NAME}-github-role-${ENVIRONMENT}-${AWS_REGION}"

GITHUB_OIDC_PROVIDER_ARN=""  # Change to your OIDC provider ARN
GITHUB_ORG=""                # Change this to your GitHub organization name
GITHUB_REPO=""               # Change this to your repository name

# Deploy the CloudFormation stack
aws cloudformation create-stack \
  --stack-name $STACK_NAME \
  --template-body file://bootstrap/cloudformation/github-oidc-role.yaml \
  --parameters \
    ParameterKey=ProjectName,ParameterValue=$PROJECT_NAME \
    ParameterKey=Environment,ParameterValue=$ENVIRONMENT \
    ParameterKey=GitHubOIDCProviderArn,ParameterValue=$GITHUB_OIDC_PROVIDER_ARN \
    ParameterKey=GitHubOrg,ParameterValue=$GITHUB_ORG \
    ParameterKey=GitHubRepo,ParameterValue=$GITHUB_REPO \
  --capabilities CAPABILITY_NAMED_IAM \
  --region $AWS_REGION

# Wait for stack creation to complete
aws cloudformation wait stack-create-complete \
  --stack-name $STACK_NAME \
  --region $AWS_REGION

# Get the role ARN from stack outputs
echo "GitHub role created:"
aws cloudformation describe-stacks \
  --stack-name $STACK_NAME \
  --region $AWS_REGION \
  --query 'Stacks[0].Outputs[?OutputKey==`RoleArn`].OutputValue' \
  --output text

Record the role ARN: you'll need to reference it in your deployment configuration later.

Note: To deploy for additional environments, such as staging or prod, change the ENVIRONMENT variable and (for environments in different accounts) GITHUB_OIDC_PROVIDER_ARN variable appropriately, configure the appropriate AWS credentials, and rerun the commands above.

Update IAM Role Stack (Optional)

To update the GitHub OIDC role stack with changes to the template, such as adding new permissions:

# Configure AWS credentials for the target account (if using separate accounts)
# export AWS_PROFILE=dev

# Set your values
AWS_REGION=""    # Change this to your state bucket region
PROJECT_NAME=""  # Change this to your project name (max 8 characters)
ENVIRONMENT=""   # Change to: dev, staging, prod, etc.

STACK_NAME="filmdrop-${PROJECT_NAME}-github-role-${ENVIRONMENT}-${AWS_REGION}"

GITHUB_OIDC_PROVIDER_ARN=""  # Change to your OIDC provider ARN
GITHUB_ORG=""                # Change this to your GitHub organization name
GITHUB_REPO=""               # Change this to your repository name

aws cloudformation update-stack \
  --stack-name $STACK_NAME \
  --template-body file://bootstrap/cloudformation/github-oidc-role.yaml \
  --parameters \
    ParameterKey=ProjectName,ParameterValue=$PROJECT_NAME \
    ParameterKey=Environment,ParameterValue=$ENVIRONMENT \
    ParameterKey=GitHubOIDCProviderArn,ParameterValue=$GITHUB_OIDC_PROVIDER_ARN \
    ParameterKey=GitHubOrg,ParameterValue=$GITHUB_ORG \
    ParameterKey=GitHubRepo,ParameterValue=$GITHUB_REPO \
  --capabilities CAPABILITY_NAMED_IAM \
  --region $AWS_REGION

Note: To update for staging or prod environments, change the ENVIRONMENT variable and (for environments in different accounts) GITHUB_OIDC_PROVIDER_ARN variable, configure the appropriate AWS credentials, and rerun the command above.

Service-Linked Roles

Overview

AWS service-linked roles are IAM roles that are linked directly to AWS services. These roles provide predefined permissions that services require to call other AWS services on your behalf. These roles are shared by all resources of their respective service types within an AWS account, so they may already exist if other resources in the account use these services.

Note: These roles are often automatically created when you create certain resources through the AWS Console, but some must be explicitly created when using Terraform or other infrastructure-as-code tools.

Deploy All Service-Linked Roles

The commands below check if each role already exists and creates it if needed. No state management is required for these operations. While some may not be used depending on which FilmDrop components you enable, there is no harm or cost in deploying them either way.

# Configure AWS credentials for the target account
# aws configure --profile dev
# export AWS_PROFILE=dev

# OpenSearch Service (for stac-server)
aws iam get-role --role-name AWSServiceRoleForAmazonOpenSearchService &> /dev/null && echo "AWSServiceRoleForAmazonOpenSearchService already exists" || aws iam create-service-linked-role --aws-service-name opensearchservice.amazonaws.com

# ECS (for cirrus batch compute)
aws iam get-role --role-name AWSServiceRoleForECS &> /dev/null && echo "AWSServiceRoleForECS already exists" || aws iam create-service-linked-role --aws-service-name ecs.amazonaws.com

# EC2 Spot (for cirrus batch compute with spot-type instances)
aws iam get-role --role-name AWSServiceRoleForEC2Spot &> /dev/null && echo "AWSServiceRoleForEC2Spot already exists" || aws iam create-service-linked-role --aws-service-name spot.amazonaws.com

# EC2 Spot Fleet (for cirrus batch compute with spot-type instances)
aws iam get-role --role-name AWSServiceRoleForEC2SpotFleet &> /dev/null && echo "AWSServiceRoleForEC2SpotFleet already exists" || aws iam create-service-linked-role --aws-service-name spotfleet.amazonaws.com

echo ""
echo "All required service-linked roles are configured."

Note: These are account-level resources. Role names are fixed by AWS and cannot be customized.

GitHub Actions Configuration

This section is specific to GitHub Actions. If you're using a different CI/CD platform, you will need to:

Configure secrets/variables in your platform's native secret/variable management system
Store the deployment role ARN and backend configuration as secrets/variables
Configure any platform-specific authentication and deployment settings

If you're using GitHub Actions, complete this section to configure the automation.

Important: The S3 state bucket and service-linked roles are required regardless of your CI/CD platform. Only this section is GitHub Actions-specific.

Repository Secrets

Add the following secrets at the repository level to enable automated deployments:

Secret Name	Value	Description
`DEV_AWS_ROLE_ARN`	Output from Deployment IAM Role Stack	IAM role ARN for dev environment (e.g., `arn:aws:iam::123456789012:role/filmdrop-example-dev-github-role`)
`DEV_TF_BACKEND_REGION`	AWS region for state bucket	Region where the Terraform state bucket is located (e.g., `us-west-2`)
`DEV_TF_STATE_BUCKET_NAME`	State bucket name	Name of the S3 bucket for Terraform state (e.g., `filmdrop-example-dev-terraform-state-us-west-2`)

Add Repository Secrets

Navigate to Settings -> Secrets and variables -> Actions in your GitHub repository
Click New repository secret
Add each secret with its corresponding value from your CloudFormation stack outputs

Note: If you're deploying to multiple environments (e.g., staging, prod), create corresponding secrets like STAGING_AWS_ROLE_ARN, STAGING_TF_BACKEND_REGION, etc. with the appropriate values for each environment.

Optional: Branch Protection Rules

For production-grade deployments, consider configuring branch protection rules:

Navigate to Settings -> Branches in your GitHub repository
Add a branch protection rule for main
Enable:
- Require a pull request before merging
- Require approvals (1 or more reviewers)
- Require status checks to pass (select your workflow checks)
- Require branches to be up to date

This ensures all infrastructure changes are reviewed before deployment.

AWS Credentials Management

Local Development Best Practices

When working with AWS locally, follow these security best practices:

Use Named Profiles (Not Default)

# Configure a named profile
aws configure --profile filmdrop-dev

# Use the profile
export AWS_PROFILE=filmdrop-dev

# Or use inline
aws s3 ls --profile filmdrop-dev

Use Temporary Credentials

# Assume a role for temporary credentials
aws sts assume-role \
  --role-arn arn:aws:iam::123456789012:role/filmdrop-dev-role \
  --role-session-name local-dev-session \
  --duration-seconds 3600

# Set the temporary credentials in your environment
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=...

Credential File Location

Store credentials in ~/.aws/credentials (never in your project directory)
Store configuration in ~/.aws/config
These files are excluded from git by default (not in project .gitignore)

Security Tips:

Enable MFA for all AWS users
Rotate access keys every 90 days (or use temporary credentials exclusively)
Never commit AWS credentials to git
Use different IAM users/roles for different projects
Set up AWS IAM Identity Center (AWS SSO) for centralized credential management

CI/CD Authentication (OIDC)

This template uses OpenID Connect (OIDC) for GitHub Actions authentication, which provides several security benefits:

Benefits of OIDC:

✅ No long-lived credentials stored in GitHub Secrets
✅ Temporary credentials that automatically expire (2-hour sessions)
✅ Granular trust policies (restrict by branch, tag, or pull request)
✅ Audit trail through CloudTrail for all assumed role actions
✅ Easy credential rotation (no secrets to update)

How OIDC Works:

GitHub Actions workflow requests an OIDC token from GitHub
Token is presented to AWS STS (Security Token Service)
AWS validates the token against the IAM role trust policy
Temporary credentials are issued (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN)
Credentials expire after 2 hours (MaxSessionDuration)

Trust Policy Security: The IAM role trust policy restricts which workflows can assume the role:

StringLike:
  'token.actions.githubusercontent.com:sub':
    - 'repo:org/repo:ref:refs/heads/main'      # Only main branch
    - 'repo:org/repo:pull_request'              # Only pull requests

Alternative Authentication Methods (Not Recommended):

❌ Long-lived AWS access keys stored in GitHub Secrets (security risk)
❌ IAM user credentials (requires manual rotation)
⚠️ Self-hosted runners with instance profiles (infrastructure overhead)

Credential Rotation Before Public Release

CRITICAL: If you plan to make this repository public, you must rotate any credentials that may have been committed to git history, even if they were later removed.

Rotation Checklist:

Scan Git History:

# Use gitleaks to scan entire history
gitleaks detect --source . --verbose --log-opts "--all"

# Search for common patterns
git log -p --all | grep -i "AKIA\|aws_secret\|password\|token"

Rotate AWS Credentials:
- Delete any AWS access keys that may have been exposed
- Create new access keys if needed (or transition to OIDC)
- Update any systems using the old credentials
Rotate GitHub Tokens:
- Revoke any GitHub Personal Access Tokens that were committed
- Generate new tokens with minimal required scopes
Rotate Other Secrets:
- API keys for external services
- Database passwords
- TLS/SSL certificates and private keys
Update GitHub Secrets:
- Ensure repository secrets contain only non-sensitive references (ARNs, bucket names)
- Verify no actual credentials are stored in plaintext
Consider BFG Repo-Cleaner: If secrets were committed, consider using BFG Repo-Cleaner to remove them from git history:
```
# WARNING: This rewrites git history
bfg --delete-files credentials
bfg --replace-text passwords.txt
```

After Rotation:

Monitor AWS CloudTrail for any unexpected API calls using old credentials
Set up AWS IAM Access Analyzer to identify any permission issues
Review GitHub's security alerts and dependency scan results

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bootstrap Infrastructure and CI

Table of Contents

Prerequisites

AWS Infrastructure (Required for all platforms)

Terraform State Bucket

Overview

Deploy State Bucket Stack

Terraform Backend Configuration

Update State Bucket Stack (Optional)

Deployment IAM Role

Overview

IAM Role Permissions Review

Prerequisites

Deploy IAM Role Stack

Update IAM Role Stack (Optional)

Service-Linked Roles

Overview

Deploy All Service-Linked Roles

GitHub Actions Configuration

Repository Secrets

Add Repository Secrets

Optional: Branch Protection Rules

AWS Credentials Management

Local Development Best Practices

CI/CD Authentication (OIDC)

Credential Rotation Before Public Release

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

Bootstrap Infrastructure and CI

Table of Contents

Prerequisites

AWS Infrastructure (Required for all platforms)

Terraform State Bucket

Overview

Deploy State Bucket Stack

Terraform Backend Configuration

Update State Bucket Stack (Optional)

Deployment IAM Role

Overview

IAM Role Permissions Review

Prerequisites

Deploy IAM Role Stack

Update IAM Role Stack (Optional)

Service-Linked Roles

Overview

Deploy All Service-Linked Roles

GitHub Actions Configuration

Repository Secrets

Add Repository Secrets

Optional: Branch Protection Rules

AWS Credentials Management

Local Development Best Practices

CI/CD Authentication (OIDC)

Credential Rotation Before Public Release