Skip to content

Commit c69b5a4

Browse files
committed
library-copy-aws-code-artifact: add npm mirror to AWS CodeArtifact example
Add an example that mirrors npm packages from Chainguard Libraries to AWS CodeArtifact. Reads an npm package-lock.json or pnpm-lock.yaml, downloads each package from Chainguard (with public npm fallback), and publishes to a private CodeArtifact repository with multi-pass retry for on-demand ingestion. Includes the mirror script, a CodeArtifact setup script and guide, and a readme documenting prerequisites, configuration, and usage.
1 parent 61ca955 commit c69b5a4

4 files changed

Lines changed: 1310 additions & 0 deletions

File tree

Lines changed: 217 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,217 @@
1+
# AWS CodeArtifact Setup Guide
2+
3+
This guide will help you set up AWS CodeArtifact to test the npm mirroring script.
4+
5+
## Prerequisites
6+
7+
1. **AWS CLI installed**
8+
```bash
9+
# Check if installed
10+
aws --version
11+
12+
# If not installed, visit: https://aws.amazon.com/cli/
13+
```
14+
15+
2. **AWS credentials configured**
16+
```bash
17+
# Configure AWS CLI with your credentials
18+
aws configure
19+
20+
# Enter:
21+
# - AWS Access Key ID
22+
# - AWS Secret Access Key
23+
# - Default region (e.g., us-east-1)
24+
# - Default output format (json)
25+
```
26+
27+
3. **Chainguard credentials**
28+
- You need a Chainguard account with access to Chainguard Libraries
29+
- Get your authentication token from Chainguard
30+
31+
## Quick Setup
32+
33+
### Option 1: Automated Setup (Recommended)
34+
35+
Use the provided setup script:
36+
37+
```bash
38+
# Make it executable
39+
chmod +x setup-codeartifact.sh
40+
41+
# Run the setup
42+
./setup-codeartifact.sh
43+
```
44+
45+
The script will:
46+
- Create a CodeArtifact domain named `npm-mirror-test`
47+
- Create a repository named `cg-npm-packages`
48+
- Display the environment variables you need to export
49+
50+
### Option 2: Manual Setup
51+
52+
If you prefer to set it up manually:
53+
54+
```bash
55+
# Set your region
56+
export AWS_REGION="us-east-1"
57+
58+
# Create a CodeArtifact domain
59+
aws codeartifact create-domain \
60+
--domain npm-mirror-test \
61+
--region $AWS_REGION
62+
63+
# Create a repository
64+
aws codeartifact create-repository \
65+
--domain npm-mirror-test \
66+
--repository cg-npm-packages \
67+
--description "npm packages mirror from Chainguard" \
68+
--region $AWS_REGION
69+
```
70+
71+
## Configure Environment Variables
72+
73+
After setup, export these environment variables:
74+
75+
```bash
76+
# AWS CodeArtifact settings
77+
export AWS_REGION="us-east-1"
78+
export CODEARTIFACT_DOMAIN="npm-mirror-test"
79+
export CODEARTIFACT_REPOSITORY="cg-npm-packages"
80+
# CODEARTIFACT_DOMAIN_OWNER is optional - will auto-detect your AWS account ID
81+
82+
# Chainguard credentials
83+
export CGR_USER="your-chainguard-identity"
84+
export CGR_TOKEN="your-chainguard-token"
85+
```
86+
87+
## Test the Mirror Script
88+
89+
### 1. Create a test package-lock.json
90+
91+
Create a simple test project:
92+
93+
```bash
94+
# Create test directory
95+
mkdir test-mirror
96+
cd test-mirror
97+
98+
# Initialize npm project
99+
npm init -y
100+
101+
# Install a small package to generate package-lock.json
102+
npm install lodash@4.17.21
103+
```
104+
105+
### 2. Run the mirror script
106+
107+
```bash
108+
# Run the mirror script from the code-artifact directory
109+
cd ../code-artifact
110+
./npm-codeartifact-mirror.sh ../test-mirror/package-lock.json
111+
```
112+
113+
### 3. Verify packages in CodeArtifact
114+
115+
```bash
116+
# List packages in your repository
117+
aws codeartifact list-packages \
118+
--domain npm-mirror-test \
119+
--repository cg-npm-packages \
120+
--format npm \
121+
--region $AWS_REGION
122+
123+
# List versions of a specific package
124+
aws codeartifact list-package-versions \
125+
--domain npm-mirror-test \
126+
--repository cg-npm-packages \
127+
--format npm \
128+
--package lodash \
129+
--region $AWS_REGION
130+
131+
# For scoped packages, use the --namespace parameter
132+
aws codeartifact list-package-versions \
133+
--domain npm-mirror-test \
134+
--repository cg-npm-packages \
135+
--format npm \
136+
--package fs-minipass \
137+
--namespace isaacs \
138+
--region $AWS_REGION
139+
```
140+
141+
## Features
142+
143+
The mirroring script supports:
144+
- **Regular npm packages** (e.g., `lodash`, `express`)
145+
- **Scoped packages** (e.g., `@types/node`, `@babel/core`, `@isaacs/fs-minipass`)
146+
- **Duplicate detection** - skips packages already in CodeArtifact
147+
- **Attestation preservation** - npm package attestations are preserved during mirroring
148+
149+
## Test npm install from CodeArtifact
150+
151+
Configure npm to use your CodeArtifact repository:
152+
153+
```bash
154+
# Get auth token
155+
export CODEARTIFACT_AUTH_TOKEN=$(aws codeartifact get-authorization-token \
156+
--domain npm-mirror-test \
157+
--query authorizationToken \
158+
--output text \
159+
--region $AWS_REGION)
160+
161+
# Get repository endpoint
162+
export CODEARTIFACT_REGISTRY=$(aws codeartifact get-repository-endpoint \
163+
--domain npm-mirror-test \
164+
--repository cg-npm-packages \
165+
--format npm \
166+
--query repositoryEndpoint \
167+
--output text \
168+
--region $AWS_REGION)
169+
170+
# Configure npm to use CodeArtifact
171+
npm config set registry $CODEARTIFACT_REGISTRY
172+
npm config set //$(echo $CODEARTIFACT_REGISTRY | sed 's|https://||')/:_authToken $CODEARTIFACT_AUTH_TOKEN
173+
174+
# Test install
175+
npm install lodash
176+
```
177+
178+
## Cleanup (Optional)
179+
180+
When you're done testing:
181+
182+
```bash
183+
# Delete the repository
184+
aws codeartifact delete-repository \
185+
--domain npm-mirror-test \
186+
--repository cg-npm-packages \
187+
--region $AWS_REGION
188+
189+
# Delete the domain
190+
aws codeartifact delete-domain \
191+
--domain npm-mirror-test \
192+
--region $AWS_REGION
193+
```
194+
195+
## Troubleshooting
196+
197+
### "Access Denied" errors
198+
- Ensure your AWS IAM user/role has CodeArtifact permissions
199+
- Required permissions: `codeartifact:*`, `sts:GetServiceBearerToken`
200+
201+
### "Package not found" in Chainguard
202+
- Not all npm packages are available in Chainguard Libraries
203+
- Chainguard curates packages for security
204+
- Check the Chainguard catalog for available packages
205+
206+
### Authentication failures
207+
- CodeArtifact tokens expire after 12 hours
208+
- Re-run the auth token command if needed
209+
- Ensure Chainguard credentials are correct
210+
211+
## Cost Considerations
212+
213+
AWS CodeArtifact pricing (as of 2024):
214+
- **Storage**: $0.05 per GB per month
215+
- **Requests**: $0.05 per 10,000 requests
216+
217+
For testing with a few packages, costs should be minimal (typically < $1/month).

0 commit comments

Comments
 (0)