|
| 1 | +# ECR Lambda Mirror with Terraform |
| 2 | + |
| 3 | +This project provisions a Lambda function with Terraform that mirrors images from **cgr.dev** into **AWS ECR**. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +* lists all repos + tags in your Chainguard group |
| 8 | +* ensures a matching ECR repo exists (same path, optional prefix), |
| 9 | +* creates the ECR repository if it does not exist |
| 10 | +* Pulls from cgr.dev/<namespace>/<repo>:<tag> and mirrors into ECR. |
| 11 | +* Uses your pull token for cgr.dev. |
| 12 | +* Auths into ECR via the AWS SDK default credentials chain. |
| 13 | +* Pre-checks if the image already exists in ECR (by tag+digest) before copying |
| 14 | +* If it exists, it skips and logs skip exists without downloading layers |
| 15 | +* schedule.tf runs the lamba function every 4 hours by default |
| 16 | +* Each repository copy is invoked in a single lambda function |
| 17 | +* repo-tags var allows to specify which tags to mirror (if not all or latest) |
| 18 | + |
| 19 | +## Architecture |
| 20 | + |
| 21 | + |
| 22 | + |
| 23 | +### Environment variables the Lambda expects |
| 24 | + |
| 25 | +Set these in your Terraform aws_lambda_function environment {}: |
| 26 | + |
| 27 | +``` |
| 28 | + SRC_REGISTRY = var.src_registry |
| 29 | + GROUP_NAME = var.group_name |
| 30 | + DST_PREFIX = var.dst_prefix |
| 31 | +
|
| 32 | + # Pull-token credentials for cgr.dev (username=identity id, password=JWT) |
| 33 | + CGR_USERNAME = var.cgr_username |
| 34 | + CGR_PASSWORD = var.cgr_password |
| 35 | +
|
| 36 | + # Optional knobs for your chaining main.go |
| 37 | + REPO_LIST_JSON = jsonencode(var.repo_list) |
| 38 | + COPY_ALL_TAGS = tostring(var.copy_all_tags) |
| 39 | + REPO_TAGS_JSON = jsonencode(var.repo_tags) |
| 40 | +
|
| 41 | + # Booleans must be strings in Lambda env |
| 42 | + MIRROR_DRY_RUN = tostring(var.mirror_dry_run) |
| 43 | +``` |
| 44 | + |
| 45 | +### Mirroring & Skips |
| 46 | + |
| 47 | +main.go uses the same underlying library (go-containerregistry), but through its Go APIs: |
| 48 | + |
| 49 | +* remote.List(repoRef, …) → lists tags from cgr.dev |
| 50 | +* remote.Get(srcRef, …) → pulls an image/index manifest |
| 51 | +* remote.Write(dstRef, img, …) / remote.WriteIndex(dstRef, idx, …) → pushes into ECR |
| 52 | + |
| 53 | +### Destination Repo Settings |
| 54 | + |
| 55 | +terraform.tfvars |
| 56 | +``` |
| 57 | +aws_region = "us-east-2" |
| 58 | +aws_profile = "cg-dev" |
| 59 | +
|
| 60 | +group_name = "bannon.dev" |
| 61 | +name_prefix = "chainguar-image-mirror" |
| 62 | +
|
| 63 | +# optional: dst_prefix = "mirrors" |
| 64 | +
|
| 65 | +# identity id (username) for your pull token |
| 66 | +cgr_username = "b3afeb8ee1de8a24fe87ccb26faee88b5ba3cac0/7d8f1d77937ae3d2" |
| 67 | +
|
| 68 | +mirror_dry_run = false |
| 69 | +
|
| 70 | +repo_list = [ |
| 71 | + "cgr.dev/bannon.dev/datadog-agent", |
| 72 | + "cgr.dev/bannon.dev/node", |
| 73 | + "cgr.dev/bannon.dev/python", |
| 74 | + "cgr.dev/bannon.dev/jdk", |
| 75 | + "cgr.dev/bannon.dev/jre", |
| 76 | + "cgr.dev/bannon.dev/envoy", |
| 77 | +] |
| 78 | +
|
| 79 | +copy_all_tags = true |
| 80 | +
|
| 81 | +repo_tags = { |
| 82 | + "cgr.dev/bannon.dev/node" = ["22] |
| 83 | + "cgr.dev/bannon.dev/datadog-agent" = ["7.69", "7.69-dev"] |
| 84 | +} |
| 85 | +``` |
| 86 | + |
| 87 | +# Usage |
| 88 | + |
| 89 | +## Go Mod Sanity Check |
| 90 | + |
| 91 | +``` |
| 92 | +go mod tidy |
| 93 | +``` |
| 94 | +## Create the image-copy-all repository to execute Lambda mirror from |
| 95 | + |
| 96 | +Note: requires pull token password during init. Your pull token username is defined in terraform.tfvars and configured to use this variable. |
| 97 | + |
| 98 | +``` |
| 99 | +cd iac/ |
| 100 | +export AWS_PROFILE=cg-dev |
| 101 | +export AWS_REGION=us-east-2 |
| 102 | +
|
| 103 | +terraform init -upgrade |
| 104 | +terraform plan |
| 105 | +terraform apply -auto-approve \ |
| 106 | + -var='cgr_password=<PULL_TOKEN_PASS>' |
| 107 | +``` |
| 108 | + |
| 109 | +## Invoke the Lambda Function |
| 110 | + |
| 111 | +``` |
| 112 | + aws lambda invoke \ |
| 113 | + --function-name image-copy-all \ |
| 114 | + --region us-east-2 \ |
| 115 | + --log-type Tail \ |
| 116 | + --payload '{}' \ |
| 117 | + response.json |
| 118 | +``` |
| 119 | + |
| 120 | +## Follow the logs for progress |
| 121 | + |
| 122 | +``` |
| 123 | +aws logs tail /aws/lambda/image-copy-all --region us-east-2 --follow |
| 124 | +``` |
| 125 | +For a specific image (.e Datadog) |
| 126 | +``` |
| 127 | +aws logs tail /aws/lambda/image-copy-all --region us-east-2 --follow | grep datadog-agent |
| 128 | +``` |
0 commit comments