feat: add AWS API tracking and validation workflows

[robmorgan]

Summary

• Port three GitHub Actions workflows from infraspec-api repository
• Add automated AWS SDK tracking and AI-powered implementation generation
• Add weekly API parity checks with PR coverage comments
• Add scheduled feature validation using the embedded emulator

Workflows Added

Workflow	Schedule	Purpose
aws-api-autotrack.yml	Daily	Track AWS SDK releases, analyze API changes, generate implementations with Claude Code
cloudmirror.yml	Weekly + PRs	AWS API parity analysis, coverage comments on PRs touching internal/emulator/services/
continuous-validation.yml	Every 6 hours	Run Gherkin features and unit tests with embedded emulator

Path Adaptations

Adjusted paths from infraspec-api to match this repo's structure:

infraspec-api	infraspec
internal/services/	internal/emulator/services/
cmd/emulator/	cmd/infraspec/
terraform/tests/	features/ (Gherkin tests)

Required Secrets

• CLAUDE_CODE_OAUTH_TOKEN - For AI code generation in aws-api-autotrack.yml (optional - can skip generation)

Test plan

• Manually trigger cloudmirror.yml workflow and verify it runs
• Manually trigger continuous-validation.yml workflow and verify feature tests execute
• Verify PR comments appear when internal/emulator/services/ files change

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Review	Updated (UTC)
infraspec	Ready	Preview, Comment	Jan 8, 2026 1:46am

[claude]

PR Review: AWS API Tracking and Validation Workflows

Great work porting these workflows from infraspec-API! This is a comprehensive automation system that will significantly improve API parity tracking and validation. Here is my detailed review:

---

✅ Strengths

Architecture & Design

• Well-structured workflow separation: Each workflow has a clear, single responsibility
• Smart artifact passing: Good use of GitHub Actions artifacts to pass data between jobs
• Proper error handling: Failure notification job in aws-api-autotrack.yml ensures issues don't go unnoticed
• Path adaptations: Correctly mapped paths from infraspec-API to this repo's structure

Code Quality

• Follows conventional commits: PR description and auto-generated commits follow the project's commit standards
• Good use of conditionals: Jobs run only when needed
• Comprehensive scheduling: Daily SDK tracking + weekly parity checks + 6-hour validation provides good coverage

---

⚠️ Issues & Concerns

1. Security: Secrets Management (HIGH SEVERITY)

Line 207 in aws-api-autotrack.yml uses CLAUDE_CODE_OAUTH_TOKEN but the workflow will fail if it's not set, despite being marked as optional in the PR description.

Fix: Add proper conditional checks to skip AI generation when secret is missing:

generate-implementations:
  if: |
    needs.analyze-changes.outputs.has_changes == 'true' &&
    inputs.skip_generation != true &&
    secrets.CLAUDE_CODE_OAUTH_TOKEN != ''

2. Bug: Incorrect Sed Usage for Variable Fixing (MEDIUM SEVERITY)

Lines 275-285 and 387-397 use fragile sed patterns to auto-fix unused variables. This could:

• Break valid code where variables are used later
• Rename variables in comments or strings
• Create hard-to-debug issues

Recommendation: Remove auto-fix logic. Generated code should be correct from the start, and failures should be manually reviewed.

3. Missing Error Handling: Git Clone Failures (MEDIUM SEVERITY)

If the SDK version tag doesn't exist, git clone will fail without a clear error message.

Fix: Add error handling:

if ! git clone --depth 1 --branch "$NEW_VERSION" \
  https://github.com/aws/aws-sdk-go-v2.git /tmp/sdk-new; then
  echo "::error::Failed to clone SDK version $NEW_VERSION"
  exit 1
fi

4. Resource Inefficiency: Large SDK Clones (LOW SEVERITY)

Cloning the full AWS SDK multiple times per workflow run is wasteful.

Optimization: Use GitHub Actions cache:

- name: Cache AWS SDK
  uses: actions/cache@v4
  with:
    path: /tmp/aws-sdk-go-v2
    key: aws-sdk-v2-${{ needs.check-sdk-release.outputs.new_version }}

5. Incomplete Baseline Initialization (MEDIUM SEVERITY)

The .cloudmirror-baseline.json file doesn't exist in the repo yet. First run will skip diff detection.

Fix: Initialize baseline in this PR or in the workflow:

if [ ! -f .cloudmirror-baseline.json ]; then
  echo "[]" > .cloudmirror-baseline.json
  echo "Initialized baseline"
fi

6. Potential Race Condition (LOW SEVERITY)

Multiple concurrent workflow runs could create duplicate PRs or branches.

Fix: Add concurrency control:

concurrency:
  group: aws-sdk-update
  cancel-in-progress: false

---

🔧 Suggestions for Improvement

1. Add Workflow Status Badges

Include in README.md to show workflow health at a glance

2. Pin Claude Code Action Version

Use specific version (e.g., v1.2.3) instead of v1 for reproducibility

3. Add Workflow Timeouts

Prevent runaway jobs by adding timeout-minutes to long-running jobs (suggest 30 minutes for AI generation)

4. Improve PR Body Template

Include direct links to PARITY_REPORT.md artifact and specific list of new operations

5. Add Failure Context to continuous-validation.yml

Upload failure logs and test results as artifacts for easier debugging

---

🧪 Test Coverage Assessment

The workflows themselves are not tested before merge. Recommendations:

1. Test each workflow manually before merging (as noted in PR description)
2. Consider adding a dry-run mode for aws-api-autotrack.yml
3. Verify cloudmirror binary builds successfully in CI

---

📊 Performance Considerations

Estimated Resource Usage (per run):

• aws-api-autotrack.yml: 15-30 minutes (with AI generation)
• cloudmirror.yml: 2-5 minutes
• continuous-validation.yml: 10-20 minutes

Recommendation: Monitor GitHub Actions minutes usage. Consider:

• Running SDK tracking weekly instead of daily (AWS SDK releases are typically monthly)
• Running validation every 12 hours instead of 6

---

🔒 Security Review

Good:

• Proper permissions scoping
• No hardcoded secrets
• id-token: write only where needed

Concerns:

• Auto-commits to main branch in cloudmirror.yml (line 187) should require PR approval
• AI-generated code merged automatically could introduce vulnerabilities

Fix: Change baseline update to PR-based workflow instead of direct commit

---

📝 Documentation Gaps

Missing documentation for:

1. How to set up CLAUDE_CODE_OAUTH_TOKEN secret
2. Expected behavior when secret is not set
3. Workflow trigger strategy rationale
4. How to interpret parity reports

Recommendation: Add .github/workflows/README.md explaining workflow purposes, required secrets, manual triggers, and output interpretation

---

✅ Final Recommendation

Conditional APPROVE with required changes

Must Fix Before Merge:

1. Add secret existence check (Issue Docker Support #1)
2. Initialize .cloudmirror-baseline.json (Issue Add initial RDS support #5)
3. Change baseline updates to PR-based (Security concern)
4. Add concurrency control (Issue chore: bump go to 1.24.3 #6)

Should Fix Soon After:

1. Remove auto-fix sed logic (Issue Setup CI #2)
2. Add error handling for git clones (Issue Verbosity Flag and Output UX Improvements #3)
3. Add SDK caching (Issue license for the tool #4)
4. Add workflow documentation

Nice to Have:

• Workflow status badges
• Timeout settings
• Failure log artifacts

---

🎯 Testing Checklist

Before merging, verify:

• Manual trigger of cloudmirror.yml succeeds
• Manual trigger of continuous-validation.yml succeeds
• PR comment appears when touching internal/emulator/services/
• Workflow fails gracefully when CLAUDE_CODE_OAUTH_TOKEN is missing
• Baseline file is created on first run

---

Overall: This is excellent infrastructure automation work! The workflows are well-designed and will provide significant value. Address the security and initialization issues, and this will be ready to merge. 🚀

Generated with Claude Code