Skip to content

Commit 08d823a

Browse files
authored
adding new ecr mirroring example to platform-examples (#246)
Signed-off-by: Eric Bannon <eric.bannon4@gmail.com>
1 parent 0b43c43 commit 08d823a

File tree

13 files changed

+1086
-0
lines changed

13 files changed

+1086
-0
lines changed
Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# ECR Lambda Mirror with Terraform
2+
3+
This project provisions a Lambda function with Terraform that mirrors images from **cgr.dev** into **AWS ECR**.
4+
5+
## Overview
6+
7+
* lists all repos + tags in your Chainguard group
8+
* ensures a matching ECR repo exists (same path, optional prefix),
9+
* creates the ECR repository if it does not exist
10+
* Pulls from cgr.dev/<namespace>/<repo>:<tag> and mirrors into ECR.
11+
* Uses your pull token for cgr.dev.
12+
* Auths into ECR via the AWS SDK default credentials chain.
13+
* Pre-checks if the image already exists in ECR (by tag+digest) before copying
14+
* If it exists, it skips and logs skip exists without downloading layers
15+
* schedule.tf runs the lamba function every 4 hours by default
16+
* Each repository copy is invoked in a single lambda function
17+
* repo-tags var allows to specify which tags to mirror (if not all or latest)
18+
19+
## Architecture
20+
21+
![Architecture Diagram](assets/arch.png)
22+
23+
### Environment variables the Lambda expects
24+
25+
Set these in your Terraform aws_lambda_function environment {}:
26+
27+
```
28+
SRC_REGISTRY = var.src_registry
29+
GROUP_NAME = var.group_name
30+
DST_PREFIX = var.dst_prefix
31+
32+
# Pull-token credentials for cgr.dev (username=identity id, password=JWT)
33+
CGR_USERNAME = var.cgr_username
34+
CGR_PASSWORD = var.cgr_password
35+
36+
# Optional knobs for your chaining main.go
37+
REPO_LIST_JSON = jsonencode(var.repo_list)
38+
COPY_ALL_TAGS = tostring(var.copy_all_tags)
39+
REPO_TAGS_JSON = jsonencode(var.repo_tags)
40+
41+
# Booleans must be strings in Lambda env
42+
MIRROR_DRY_RUN = tostring(var.mirror_dry_run)
43+
```
44+
45+
### Mirroring & Skips
46+
47+
main.go uses the same underlying library (go-containerregistry), but through its Go APIs:
48+
49+
* remote.List(repoRef, …) → lists tags from cgr.dev
50+
* remote.Get(srcRef, …) → pulls an image/index manifest
51+
* remote.Write(dstRef, img, …) / remote.WriteIndex(dstRef, idx, …) → pushes into ECR
52+
53+
### Destination Repo Settings
54+
55+
terraform.tfvars
56+
```
57+
aws_region = "us-east-2"
58+
aws_profile = "cg-dev"
59+
60+
group_name = "bannon.dev"
61+
name_prefix = "chainguar-image-mirror"
62+
63+
# optional: dst_prefix = "mirrors"
64+
65+
# identity id (username) for your pull token
66+
cgr_username = "b3afeb8ee1de8a24fe87ccb26faee88b5ba3cac0/7d8f1d77937ae3d2"
67+
68+
mirror_dry_run = false
69+
70+
repo_list = [
71+
"cgr.dev/bannon.dev/datadog-agent",
72+
"cgr.dev/bannon.dev/node",
73+
"cgr.dev/bannon.dev/python",
74+
"cgr.dev/bannon.dev/jdk",
75+
"cgr.dev/bannon.dev/jre",
76+
"cgr.dev/bannon.dev/envoy",
77+
]
78+
79+
copy_all_tags = true
80+
81+
repo_tags = {
82+
"cgr.dev/bannon.dev/node" = ["22]
83+
"cgr.dev/bannon.dev/datadog-agent" = ["7.69", "7.69-dev"]
84+
}
85+
```
86+
87+
# Usage
88+
89+
## Go Mod Sanity Check
90+
91+
```
92+
go mod tidy
93+
```
94+
## Create the image-copy-all repository to execute Lambda mirror from
95+
96+
Note: requires pull token password during init. Your pull token username is defined in terraform.tfvars and configured to use this variable.
97+
98+
```
99+
cd iac/
100+
export AWS_PROFILE=cg-dev
101+
export AWS_REGION=us-east-2
102+
103+
terraform init -upgrade
104+
terraform plan
105+
terraform apply -auto-approve \
106+
-var='cgr_password=<PULL_TOKEN_PASS>'
107+
```
108+
109+
## Invoke the Lambda Function
110+
111+
```
112+
aws lambda invoke \
113+
--function-name image-copy-all \
114+
--region us-east-2 \
115+
--log-type Tail \
116+
--payload '{}' \
117+
response.json
118+
```
119+
120+
## Follow the logs for progress
121+
122+
```
123+
aws logs tail /aws/lambda/image-copy-all --region us-east-2 --follow
124+
```
125+
For a specific image (.e Datadog)
126+
```
127+
aws logs tail /aws/lambda/image-copy-all --region us-east-2 --follow | grep datadog-agent
128+
```
116 KB
Loading

0 commit comments

Comments
 (0)