Skip to content

Commit f79f5af

Browse files
committed
feat(migrate): Sync migrate/ from GitFarm
Source: Startups-Migrate@918af4f
1 parent bab769a commit f79f5af

71 files changed

Lines changed: 14243 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"name": "startups-migrate",
3+
"owner": {
4+
"name": "AWS Startups - Migrate"
5+
},
6+
"description": "Migration tools and plugins for moving workloads to AWS",
7+
"metadata": {
8+
"pluginRoot": "./plugins"
9+
},
10+
"plugins": [
11+
{
12+
"name": "migration-to-aws",
13+
"source": "./migration-to-aws",
14+
"version": "1.1.0",
15+
"description": "Migrate GCP/Azure infrastructure and AI workloads to AWS with resource discovery, architecture mapping, cost analysis, and execution planning"
16+
}
17+
]
18+
}

migrate/OWNERS.yaml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
team: Startups-Migrate
2+
resolver_group: STARTUPS-MIGRATE
3+
cti:
4+
category: AWS
5+
type: Startups
6+
item: Migrate
7+
contacts:
8+
primary: carthick
9+
oncall: startups-migrate-oncall
10+
github_team: "@awslabs/startups-migrate"

migrate/README.md

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Migrate
2+
3+
AI agent plugins and tools for migrating workloads to AWS.
4+
5+
## Plugins
6+
7+
| Plugin | Description | Status |
8+
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | --------- |
9+
| **migration-to-aws** | Migrate GCP/Azure infrastructure and AI workloads to AWS with resource discovery, architecture mapping, cost analysis, and execution planning | Available |
10+
11+
## Installation
12+
13+
### Claude Code
14+
15+
```bash
16+
/plugin marketplace add awslabs/startups --sparse migrate
17+
/plugin install migration-to-aws@startups-migrate
18+
```
19+
20+
### Kiro
21+
22+
Powers are available under `powers/` for direct use with Kiro.
23+
24+
## Structure
25+
26+
```
27+
migrate/
28+
├── plugins/ # Claude Code and Cursor plugins
29+
├── powers/ # Kiro powers (generated from plugin source)
30+
├── docs/ # Documentation
31+
├── tests/ # Evaluation fixtures and invariants
32+
└── tools/ # Evaluation tooling and generators
33+
```
34+
35+
## Ownership
36+
37+
This folder is maintained by the **Startups-Migrate** team. See `OWNERS.yaml` for routing details.

migrate/docs/evaluation-guide.md

Lines changed: 305 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,305 @@
1+
# Evaluation Guide
2+
3+
How to validate the migration skill after making changes to prompt files.
4+
5+
---
6+
7+
## Quick Reference
8+
9+
| What | Command |
10+
| ------------------------------------- | ------------------------------------------------------------------- |
11+
| Structural check (instant, no Claude) | `mise run eval:check` |
12+
| Full evaluation (after migration run) | `python tools/eval_check.py --migration-dir <DIR> --fixture <NAME>` |
13+
| Guided evaluation (Claude assists) | Tell Claude: `Read tools/EVAL_SKILL.md and follow the instructions` |
14+
15+
---
16+
17+
## When to Evaluate
18+
19+
Run the evaluation harness whenever you:
20+
21+
- Edit any prompt file under `features/migration-to-aws/skills/gcp-to-aws/`
22+
- Modify schema definitions, classification rules, or mapping tables
23+
- Change the fast-path table, rubric, or pricing cache
24+
- Add or remove questions from the Clarify phase
25+
26+
---
27+
28+
## Step-by-Step Workflow
29+
30+
### 1. Make your prompt changes
31+
32+
Edit the relevant files under `features/migration-to-aws/skills/gcp-to-aws/`.
33+
34+
### 2. Run the structural check
35+
36+
```bash
37+
mise run eval:check
38+
```
39+
40+
This runs instantly with no Claude API calls. It verifies that critical directives
41+
still exist in the prompt files:
42+
43+
- FORBIDDEN blocks (scope boundaries for each phase)
44+
- BigQuery specialist gate directive
45+
- Security rules (no hardcoded credentials, no wildcard IAM)
46+
- Dry-run default for generated scripts
47+
48+
If this fails, a directive was accidentally deleted. The output tells you
49+
exactly which file and line to check.
50+
51+
### 3. Pick a fixture
52+
53+
Choose the fixture that best covers your change:
54+
55+
| Fixture | Use when you changed... |
56+
| -------------------------- | ------------------------------------------------------------------------- |
57+
| `minimal-cloud-run-sql` | General prompt changes, state machine, phase ordering, generate phase |
58+
| `bigquery-specialist-gate` | BigQuery handling, specialist gate, Clarify advisory, analytics exclusion |
59+
| `ai-workload-openai` | AI detection, model mapping, lifecycle rules, Category F questions |
60+
| `user-preferences` | Clarify question flow, preference schema, Design preference consumption |
61+
| `negative-services` | Classification rules, auth exclusion, forbidden service mappings |
62+
63+
For broad changes, run `minimal-cloud-run-sql` first (it has the most invariants),
64+
then any fixture specific to your change area.
65+
66+
### 4. Run the migration skill against the fixture
67+
68+
```bash
69+
cd tests/fixtures/minimal-cloud-run-sql
70+
```
71+
72+
Open Claude Code and trigger the migration:
73+
74+
```
75+
migrate from GCP to AWS
76+
```
77+
78+
The skill runs all 6 phases using the fixture's Terraform files and pre-seeded
79+
preferences. Wait for it to complete (you'll see "Migration planning complete"
80+
or similar).
81+
82+
### 5. Run the evaluation
83+
84+
**Option A — Direct (recommended for CI and scripting):**
85+
86+
```bash
87+
python tools/eval_check.py \
88+
--migration-dir tests/fixtures/minimal-cloud-run-sql/.migration/<RUN_ID> \
89+
--fixture minimal-cloud-run-sql
90+
```
91+
92+
Replace `<RUN_ID>` with the directory name (e.g., `0419-1200`). The checker
93+
outputs JSON with pass/fail for every invariant.
94+
95+
**Option B — Claude-assisted (recommended for understanding failures):**
96+
97+
In Claude Code, type:
98+
99+
```
100+
Read tools/EVAL_SKILL.md and follow the instructions to evaluate the migration results
101+
```
102+
103+
Claude runs the checker, explains any failures in plain language, computes
104+
hashes, and writes `.eval-results.json`.
105+
106+
### 6. Review results
107+
108+
The checker output looks like:
109+
110+
```json
111+
{
112+
"summary": {
113+
"hard_total": 26,
114+
"hard_passed": 26,
115+
"hard_failed": 0,
116+
"hard_skipped": 0,
117+
"status": "pass"
118+
}
119+
}
120+
```
121+
122+
- **All pass** — your change is safe. Proceed to commit.
123+
- **Failures** — read the `details` field for each failed invariant. It tells
124+
you what was found, what was expected, and which prompt file to check.
125+
- **Skips** — a custom handler isn't implemented yet. Not a blocker.
126+
127+
### 7. Commit the results
128+
129+
```bash
130+
git add .eval-results.json
131+
git commit -m "eval: results for minimal-cloud-run-sql"
132+
```
133+
134+
The `.eval-results.json` file is SHA-bound to your commit and prompt file
135+
hashes. CI can verify it wasn't forged or stale.
136+
137+
---
138+
139+
## Test Fixtures
140+
141+
### minimal-cloud-run-sql
142+
143+
The default fixture. 2 PRIMARY resources (Cloud Run + Cloud SQL), 3 SECONDARY
144+
resources. Tests the full 6-phase pipeline with all defaults.
145+
146+
**26 hard invariants** covering: phase ordering, inventory schema, cluster
147+
integrity, preferences schema, design mappings, confidence values, cost
148+
estimation, security rules, dry-run defaults.
149+
150+
**Location:** `tests/fixtures/minimal-cloud-run-sql/`
151+
152+
### bigquery-specialist-gate
153+
154+
Tests the BigQuery specialist gate — the rule that `google_bigquery_*` resources
155+
must map to "Deferred — specialist engagement" and be excluded from cost totals.
156+
157+
**9 hard invariants** focused on: deferred mapping, no Redshift/Athena/Glue/EMR,
158+
non-BigQuery resources still get normal mappings.
159+
160+
**Location:** `tests/fixtures/bigquery-specialist-gate/`
161+
162+
### ai-workload-openai
163+
164+
Tests AI workload detection for apps using the OpenAI SDK on GCP. Includes
165+
both Terraform and Python source code (`src/chatbot.py` with `openai` imports).
166+
167+
**11 hard invariants** focused on: AI profile creation, model detection,
168+
confidence threshold, excluded/legacy model rules, Category F questions.
169+
170+
**Location:** `tests/fixtures/ai-workload-openai/`
171+
172+
### user-preferences
173+
174+
Same infrastructure as minimal, but with pre-seeded user answers
175+
(target_region=us-west-2, availability=single-az, cutover_strategy=blue-green).
176+
Tests that Design output respects user preferences.
177+
178+
**10 hard invariants** focused on: chosen_by="user" preserved, Design region
179+
matches preference, single-AZ honored.
180+
181+
**Location:** `tests/fixtures/user-preferences/`
182+
183+
### negative-services
184+
185+
Tests that forbidden AWS services never appear in Design output. Contains
186+
auth resources (should be excluded from inventory), BigQuery (should be
187+
deferred), and Cloud Run (should map to Fargate, not Lightsail/Beanstalk).
188+
189+
**8 hard invariants** focused on: auth exclusion, no Cognito, no Lightsail,
190+
no Elastic Beanstalk, no analytics services for BigQuery.
191+
192+
**Location:** `tests/fixtures/negative-services/`
193+
194+
---
195+
196+
## Understanding Invariants
197+
198+
Invariants are defined in YAML files:
199+
200+
- **Shared:** `tests/invariants.yml` — used by `minimal-cloud-run-sql`
201+
- **Per-fixture:** `tests/fixtures/<name>/invariants.yml` — used by all other fixtures
202+
203+
Each invariant has:
204+
205+
| Field | Purpose |
206+
| ------------- | ------------------------------------------------------------ |
207+
| `id` | Unique identifier (e.g., H25, BQ3, NEG1) |
208+
| `description` | What's being checked |
209+
| `source` | Which prompt file contains the rule (file:line) |
210+
| `check.type` | How it's checked (file_exists, content_absent, custom, etc.) |
211+
212+
### Check types
213+
214+
| Type | What it does | Needs Python handler? |
215+
| ----------------- | ----------------------------------------------------- | --------------------- |
216+
| `file_exists` | Assert a file exists in migration output | No |
217+
| `file_absent` | Assert a file does NOT exist | No |
218+
| `content_present` | Assert patterns exist in file(s) | No |
219+
| `content_absent` | Assert patterns do NOT exist in file(s) | No |
220+
| `json_path_value` | Assert a JSON field has a specific value | No |
221+
| `json_every` | Assert every item in a JSON array has required fields | No |
222+
| `uniqueness` | Assert all values at a path are unique | No |
223+
| `cross_file_join` | Assert values in one file match values in another | No |
224+
| `custom` | Delegate to a Python script in `tools/invariants/` | Yes |
225+
226+
### Custom handlers
227+
228+
Python scripts in `tools/invariants/` handle logic too complex for YAML.
229+
Each script:
230+
231+
1. Takes the migration directory as `sys.argv[1]`
232+
2. Reads the relevant JSON file(s)
233+
3. Prints `{"status": "pass"}` or `{"status": "fail", "details": "..."}`
234+
4. Includes documentation with: invariant description, skill file reference,
235+
and pass/fail examples
236+
237+
---
238+
239+
## Adding a New Invariant
240+
241+
### For declarative checks (no Python):
242+
243+
Add an entry to the fixture's `invariants.yml`:
244+
245+
```yaml
246+
- id: H50
247+
description: "New check description"
248+
source: "prompt-file.md:line"
249+
check:
250+
type: content_absent
251+
file: gcp-resource-inventory.json
252+
patterns:
253+
- "forbidden pattern"
254+
```
255+
256+
### For complex checks (Python handler):
257+
258+
1. Add the YAML entry pointing to a handler:
259+
260+
```yaml
261+
- id: H50
262+
description: "New check description"
263+
source: "prompt-file.md:line"
264+
check:
265+
type: custom
266+
handler: "tools/invariants/h50_new_check.py"
267+
```
268+
269+
1. Create `tools/invariants/h50_new_check.py` following the pattern of
270+
existing handlers. Include the docstring with invariant description,
271+
skill file reference, and examples.
272+
273+
1. Run the evaluation to verify your new invariant passes on existing output.
274+
275+
---
276+
277+
## Troubleshooting
278+
279+
### `pip install pyyaml` error
280+
281+
The checker requires PyYAML. Install it:
282+
283+
```bash
284+
pip install pyyaml
285+
```
286+
287+
### "No invariants found for fixture"
288+
289+
The fixture name doesn't match any `invariants.yml`. Check:
290+
291+
- `tests/fixtures/<name>/invariants.yml` exists, OR
292+
- `tests/invariants.yml` has `fixture: <name>` at the top
293+
294+
### Handler returns "skip"
295+
296+
The custom Python handler file doesn't exist yet. Either implement it or
297+
accept the skip — skipped invariants don't affect pass/fail.
298+
299+
### False positive on H41 (hardcoded credentials)
300+
301+
The pattern `password = "` catches Terraform attribute assignments with string
302+
values. If your generated Terraform has a legitimate use (e.g.,
303+
`manage_master_user_password = true`), the check should already pass since we
304+
only match `password = "` (with a quote). If you encounter a new false
305+
positive, tighten the pattern in `tests/invariants.yml`.
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{
2+
"name": "migration-to-aws",
3+
"version": "1.1.0",
4+
"description": "This no-cost tool assesses your current cloud provider's usage, geography, and billing data to estimate and compare AWS services and pricing, and recommends migration or continued use of your current provider. AWS pricing is based on current published pricing and may vary over time. The tool may generate a .migration folder containing comparison and migration execution data, which you may delete upon completion or use to migrate to AWS.",
5+
"author": {
6+
"name": "Amazon Web Services"
7+
},
8+
"license": "MIT-0",
9+
"keywords": [
10+
"aws",
11+
"gcp",
12+
"google-cloud",
13+
"migration",
14+
"cloud-migration",
15+
"terraform",
16+
"fargate",
17+
"rds",
18+
"eks"
19+
]
20+
}

0 commit comments

Comments
 (0)