awslabs
diff --git a/‎.github/CI.md‎
Lines changed: 3 additions & 0 deletions b/‎.github/CI.md‎
Lines changed: 3 additions & 0 deletions
diff --git a/‎.github/scripts/dependency-scan.sh‎
Lines changed: 85 additions & 4 deletions b/‎.github/scripts/dependency-scan.sh‎
Lines changed: 85 additions & 4 deletions
diff --git a/‎.github/scripts/lib_dependency_scan.sh‎
Lines changed: 118 additions & 0 deletions b/‎.github/scripts/lib_dependency_scan.sh‎
Lines changed: 118 additions & 0 deletions
diff --git a/‎.github/workflows/deps-scan.yml‎
Lines changed: 6 additions & 2 deletions b/‎.github/workflows/deps-scan.yml‎
Lines changed: 6 additions & 2 deletions
diff --git a/‎.github/workflows/security.yml‎
Lines changed: 9 additions & 1 deletion b/‎.github/workflows/security.yml‎
Lines changed: 9 additions & 1 deletion
diff --git a/‎.pre-commit-config.yaml‎
Lines changed: 2 additions & 2 deletions b/‎.pre-commit-config.yaml‎
Lines changed: 2 additions & 2 deletions
@@ -162,8 +162,10 @@ Ecosystems tracked:
 | Helm charts | `lambda/helm-installer/charts.yaml` | Uses `helm show chart` for OCI charts and `helm search repo` for traditional repos |
 | EKS add-ons | `addon_name`/`addon_version` pairs extracted from `gco/stacks/constants.py` | Requires AWS credentials (via OIDC). The script pre-flights `sts get-caller-identity`; without valid creds the add-on section is explicitly **skipped** and the report notes why — everything else still runs |
 | Aurora PostgreSQL engine | `AURORA_POSTGRES_VERSION_DISPLAY` from `gco/stacks/constants.py` | Requires AWS credentials (via OIDC). Queries `rds describe-db-engine-versions` for the latest minor release within the same major line |
+| Pre-commit hooks | `repo:` / `rev:` blocks in `.pre-commit-config.yaml` | Calls `GET /repos/{owner}/{repo}/tags` on GitHub for each hook and reports drift when our pinned `rev:` is older than the highest semver-shaped tag. Unauthenticated; SHA pins and non-GitHub repos are skipped silently |
 | CDK enum constants | `LAMBDA_PYTHON_RUNTIME` and `AURORA_POSTGRES_VERSION` from `gco/stacks/constants.py` | Introspects the installed `aws-cdk-lib` (the `deps-scan` workflow installs the latest) for `aws_lambda.Runtime.PYTHON_X_Y` and `aws_rds.AuroraPostgresEngineVersion.VER_X_Y` and reports drift when our pinned enum is older than the highest member exposed by the library. Skipped with a note when `aws-cdk-lib` isn't importable |
 | Python release | `LAMBDA_PYTHON_RUNTIME` (the major Python version we standardise on across Lambdas) | Queries `https://endoflife.date/api/python.json` for the highest currently-supported stable cycle and reports drift compared to the `LAMBDA_PYTHON_RUNTIME` constant. Public endpoint, no AWS creds |
+| Pre-commit hooks | `repo:` / `rev:` pairs in `.pre-commit-config.yaml` | Queries `api.github.com/repos/<owner>/<repo>/tags` for each hook and compares against the pinned `rev:`. Uses `GITHUB_TOKEN` for the authenticated rate limit when CI runs the workflow. SHA-pinned hooks and non-GitHub repos are silently skipped — no false drift |
 
 Images matching `gco/*` are skipped (we build those). Non-semver tags (`latest`, branch names, SHAs) are ignored.
 
@@ -203,6 +205,7 @@ The console output shows each surface's drift inline. To trigger the exact workf
 - **New Helm chart** — nothing to change; the script walks every entry in `lambda/helm-installer/charts.yaml`.
 - **New EKS add-on** — add the constant in `gco/stacks/constants.py` and reference it in `regional_stack.py`. The scanner imports from the constants module.
 - **New Aurora engine version** — update `AURORA_POSTGRES_VERSION` and `AURORA_POSTGRES_VERSION_DISPLAY` in `gco/stacks/constants.py`.
+- **New pre-commit hook** — nothing to change; `extract_precommit_hooks` walks every `repo:` block in `.pre-commit-config.yaml` and the GitHub-tags lookup picks up the hook automatically (as long as the upstream lives on GitHub and tags semver-shaped releases).
 - **New CDK enum constant** — add the constant in `gco/stacks/constants.py`, then add a comparison block in `dependency-scan.sh`'s "Checking CDK enum constants" section that calls a new `get_latest_<name>` helper from `lib_dependency_scan.sh`. Pattern-match the existing `LAMBDA_PYTHON_RUNTIME` and `AURORA_POSTGRES_VERSION` blocks.
 
 #### Failure modes & debugging
 
@@ -15,8 +15,10 @@
 #   - EKS Kubernetes minor from cdk.json (AWS creds)
 #   - Aurora PostgreSQL engine versions (AWS creds)
 #   - EMR Serverless release labels (AWS creds)
-#   - Dockerfile.dev ARG pins (Node LTS major, CDK CLI, kubectl, AWS CLI v2,
-#     Docker CLI) — public endpoints, no AWS creds needed
+#   - Dockerfile.dev ARG pins (Node LTS major, npm, CDK CLI, kubectl,
+#     AWS CLI v2, Docker CLI) — public endpoints, no AWS creds needed
+#   - Pre-commit hook revisions in .pre-commit-config.yaml compared
+#     against the latest tag published upstream (GitHub API)
 #   - CDK enum constants from gco/stacks/constants.py compared against the
 #     installed aws-cdk-lib (LAMBDA_PYTHON_RUNTIME, AURORA_POSTGRES_VERSION)
 #   - Latest stable Python release from endoflife.date — public endpoint
@@ -489,6 +491,7 @@ EMR_COUNT="$(wc -l < "$EMR_RESULTS" 2>/dev/null | tr -d ' ')"
 # Each pin has its own upstream:
 #
 #   NODE_MAJOR     github://nodejs/Release → schedule.json (LTS majors)
+#   NPM_VERSION    registry.npmjs.org/npm/latest
 #   CDK_VERSION    registry.npmjs.org/aws-cdk/latest
 #   KUBECTL_VERSION https://dl.k8s.io/release/stable-<minor>.txt
 #                  (minor from cdk.json::kubernetes_version)
@@ -539,6 +542,17 @@ if candidates:
         "https://registry.npmjs.org/aws-cdk/latest" 2>/dev/null \
         | jq -r '.version // empty' 2>/dev/null)" || true
       ;;
+    NPM_VERSION)
+      # ``npm`` is part of the dev container's pinned tooling — the
+      # version that ships bundled inside nodesource's nodejs apt
+      # package isn't pinned to a patch, so the Dockerfile installs a
+      # specific ``npm@X.Y.Z`` to keep rebuilds reproducible (same
+      # rationale as CDK_VERSION above). The canonical "latest" is the
+      # ``latest`` dist-tag on npmjs.org, same source CDK uses.
+      latest="$(curl -fsSL --max-time 15 \
+        "https://registry.npmjs.org/npm/latest" 2>/dev/null \
+        | jq -r '.version // empty' 2>/dev/null)" || true
+      ;;
     KUBECTL_VERSION)
       # Match the minor line already committed to cdk.json so the pin
       # and the EKS cluster stay within the ±1 minor skew policy.
@@ -604,6 +618,55 @@ fi
 DOCKERFILE_COUNT="$(wc -l < "$DOCKERFILE_RESULTS" 2>/dev/null | tr -d ' ')"
 [ -z "$DOCKERFILE_COUNT" ] && DOCKERFILE_COUNT=0
 
+# ---------------------------------------------------------------------------
+# Pre-commit hook revisions
+#
+# Compares the ``rev:`` pinned for each ``repo:`` block in
+# ``.pre-commit-config.yaml`` against the latest semver-shaped tag
+# published by the upstream Git host. This catches drift Dependabot
+# can't see — pre-commit pins live in YAML, not in the package
+# ecosystems Dependabot monitors — and matters in practice because
+# stale hook pins quietly miss new lint rules and bug fixes.
+#
+# Each hook's repo URL is resolved to a tag list via the GitHub API
+# (the only host we use today). The helper returns empty for
+# non-GitHub repos or SHA-pinned ``rev:`` values, in which case that
+# hook is silently skipped — same pattern used by the AWS-creds-gated
+# checks above. Calls are unauthenticated; we make one request per
+# hook, which is well below the 60 req/h public limit.
+# ---------------------------------------------------------------------------
+echo ""
+echo "=== Checking pre-commit hook revisions ==="
+
+PRECOMMIT_RESULTS="$(mktemp)"
+PRECOMMIT_CONFIG=".pre-commit-config.yaml"
+
+if [ -f "$PRECOMMIT_CONFIG" ]; then
+  extract_precommit_hooks "$PRECOMMIT_CONFIG" | while IFS='|' read -r repo current_rev; do
+    [ -z "$repo" ] && continue
+    [ -z "$current_rev" ] && continue
+
+    latest_rev="$(get_latest_precommit_hook_release "$repo")"
+    [ -z "$latest_rev" ] && continue
+
+    # Strip ``v`` so compare_semver ranks ``v0.22.1`` vs ``v0.22.2``
+    # (and the rare unprefixed ``1.38.0`` from yamllint historically)
+    # consistently. We keep the original ``current_rev`` / ``latest_rev``
+    # strings in the report so the operator copy-pastes the exact
+    # value pre-commit expects.
+    if [ "$current_rev" != "$latest_rev" ] \
+       && [ "$(compare_semver "$current_rev" "$latest_rev")" = "newer" ]; then
+      echo "  - ${repo}: ${current_rev} -> ${latest_rev}"
+      echo "${repo}|${current_rev}|${latest_rev}" >> "$PRECOMMIT_RESULTS"
+    fi
+  done
+else
+  echo "  $PRECOMMIT_CONFIG not found, skipping."
+fi
+
+PRECOMMIT_COUNT="$(wc -l < "$PRECOMMIT_RESULTS" 2>/dev/null | tr -d ' ')"
+[ -z "$PRECOMMIT_COUNT" ] && PRECOMMIT_COUNT=0
+
 # ---------------------------------------------------------------------------
 # CDK enum constants
 #
@@ -733,6 +796,7 @@ else
   echo "EMR Serverless release:   $EMR_COUNT"
 fi
 echo "Dockerfile.dev pins:      $DOCKERFILE_COUNT"
+echo "Pre-commit hooks:         $PRECOMMIT_COUNT"
 if [ -n "$CDK_ENUM_SKIP_REASON" ]; then
   echo "CDK enum constants:       (skipped)"
 else
@@ -749,6 +813,7 @@ if [ "$PYTHON_COUNT" -eq 0 ] && [ "$DOCKER_COUNT" -eq 0 ] \
    && [ "$EKS_K8S_COUNT" -eq 0 ] \
    && [ "$AURORA_COUNT" -eq 0 ] && [ "$EMR_COUNT" -eq 0 ] \
    && [ "$DOCKERFILE_COUNT" -eq 0 ] \
+   && [ "$PRECOMMIT_COUNT" -eq 0 ] \
    && [ "$CDK_ENUM_COUNT" -eq 0 ] \
    && [ "$PYTHON_RELEASE_COUNT" -eq 0 ]; then
   echo ""
@@ -781,7 +846,7 @@ if [ "$PYTHON_COUNT" -eq 0 ] && [ "$DOCKER_COUNT" -eq 0 ] \
   else
     echo "All dependencies are up to date."
   fi
-  rm -f "$DOCKER_RESULTS" "$HELM_RESULTS" "$ADDON_RESULTS" "$EKS_K8S_RESULTS" "$AURORA_RESULTS" "$EMR_RESULTS" "$DOCKERFILE_RESULTS" "$CDK_ENUM_RESULTS" "$PYTHON_RELEASE_RESULTS"
+  rm -f "$DOCKER_RESULTS" "$HELM_RESULTS" "$ADDON_RESULTS" "$EKS_K8S_RESULTS" "$AURORA_RESULTS" "$EMR_RESULTS" "$DOCKERFILE_RESULTS" "$PRECOMMIT_RESULTS" "$CDK_ENUM_RESULTS" "$PYTHON_RELEASE_RESULTS"
   if [ -n "${GITHUB_OUTPUT:-}" ]; then
     echo "has_drift=false" >> "$GITHUB_OUTPUT"
   fi
@@ -918,6 +983,22 @@ fi
     echo ""
   fi
 
+  if [ "$PRECOMMIT_COUNT" -gt 0 ]; then
+    echo "## Pre-commit Hooks"
+    echo ""
+    echo "Hook \`rev:\` pins in \`.pre-commit-config.yaml\` are behind the"
+    echo "latest tag published by their upstream repos. Bump in"
+    echo "\`.pre-commit-config.yaml\`, then run \`pre-commit autoupdate\`"
+    echo "locally (or edit by hand) and verify the hooks still pass."
+    echo ""
+    echo "| Repo | Current | Latest |"
+    echo "|------|---------|--------|"
+    while IFS='|' read -r repo cur lat; do
+      echo "| $repo | \`$cur\` | \`$lat\` |"
+    done < "$PRECOMMIT_RESULTS"
+    echo ""
+  fi
+
   if [ "$CDK_ENUM_COUNT" -gt 0 ]; then
     echo "## CDK Enum Constants"
     echo ""
@@ -977,7 +1058,7 @@ fi
   echo "_Automatically created by the \`deps-scan\` workflow._"
 } > "$REPORT_FILE"
 
-rm -f "$DOCKER_RESULTS" "$HELM_RESULTS" "$ADDON_RESULTS" "$EKS_K8S_RESULTS" "$AURORA_RESULTS" "$EMR_RESULTS" "$DOCKERFILE_RESULTS" "$CDK_ENUM_RESULTS" "$PYTHON_RELEASE_RESULTS"
+rm -f "$DOCKER_RESULTS" "$HELM_RESULTS" "$ADDON_RESULTS" "$EKS_K8S_RESULTS" "$AURORA_RESULTS" "$EMR_RESULTS" "$DOCKERFILE_RESULTS" "$PRECOMMIT_RESULTS" "$CDK_ENUM_RESULTS" "$PYTHON_RELEASE_RESULTS"
 
 if [ -n "${GITHUB_OUTPUT:-}" ]; then
   echo "has_drift=true"            >> "$GITHUB_OUTPUT"
 
@@ -242,6 +242,7 @@ extract_k8s_version() {
 # Example output for Dockerfile.dev:
 #
 #     NODE_MAJOR|24
+#     NPM_VERSION|11.14.1
 #     CDK_VERSION|2.1120.0
 #     KUBECTL_VERSION|v1.35.4
 #     AWSCLI_VERSION|2.34.42
@@ -253,6 +254,7 @@ extract_dockerfile_pins() {
 import re, sys
 allowlist = {
     'NODE_MAJOR',
+    'NPM_VERSION',
     'CDK_VERSION',
     'KUBECTL_VERSION',
     'AWSCLI_VERSION',
@@ -271,6 +273,43 @@ with open(sys.argv[1]) as f:
 " "$file" 2>/dev/null
 }
 
+# extract_precommit_hooks [config_path]
+#
+# Parses ``.pre-commit-config.yaml`` and emits one ``repo|rev`` pair per
+# hook ``repo:`` block. The repo URL is left intact (it's needed to
+# resolve the upstream releases endpoint), and ``rev`` is the literal
+# string committed to the config — usually a tag like ``v0.15.7`` or
+# ``v1.19.1`` but pre-commit also tolerates plain semver and full SHAs.
+# Local hook stanzas (``repo: local``) and the pre-commit hook
+# meta-stanza (``repo: meta``) are skipped: there's no upstream release
+# to compare against.
+#
+# Falls back silently to an empty list if the file is missing or the
+# YAML can't be parsed — the caller treats that as "skip" rather than
+# "no drift", same pattern as the other extractors in this file.
+extract_precommit_hooks() {
+  local file="${1:-.pre-commit-config.yaml}"
+  [ -f "$file" ] || return 0
+  python3 -c "
+import sys, yaml
+try:
+    with open(sys.argv[1]) as f:
+        data = yaml.safe_load(f)
+except Exception:
+    sys.exit(0)
+for entry in (data or {}).get('repos', []) or []:
+    repo = (entry or {}).get('repo', '') or ''
+    rev = (entry or {}).get('rev', '') or ''
+    # ``local`` and ``meta`` are pre-commit conventions for hooks
+    # that aren't backed by an upstream repo; skip them.
+    if not repo or repo in ('local', 'meta'):
+        continue
+    if not rev:
+        continue
+    print(f'{repo}|{rev}')
+" "$file" 2>/dev/null
+}
+
 # extract_emr_versions <file>
 #
 # Extracts the pinned EMR Serverless release label from the constants module.
@@ -425,3 +464,82 @@ if candidates:
     print(max(candidates)[1])
 " 2>/dev/null
 }
+# get_latest_precommit_hook_release <repo_url>
+#
+# Given the ``repo:`` URL committed to ``.pre-commit-config.yaml``,
+# prints the latest semver-shaped tag from the upstream Git host so
+# the dep-scan can compare it against the pinned ``rev:``. Empty
+# output on network failure, an unsupported host, or when no tag
+# matches — callers treat that as "skip" rather than as drift.
+#
+# Today only GitHub repos are supported. Every hook in the project's
+# ``.pre-commit-config.yaml`` is hosted there, and the pre-commit
+# ecosystem is overwhelmingly GitHub-based. If a future hook lives
+# elsewhere (GitLab, Codeberg) the helper will return empty and the
+# scan logs a one-line skip note for that hook — no false drift.
+#
+# We use ``GET /repos/{owner}/{repo}/tags`` rather than
+# ``releases/latest`` because pre-commit pins ``rev:`` to a Git tag,
+# not a GitHub Release — and several hooks (yamllint, mirrors-mypy,
+# markdownlint-cli2) tag without ever cutting a Release. The tags
+# endpoint returns newest-first, so we filter to ``vX.Y.Z`` /
+# ``X.Y.Z`` / ``X.Y`` shapes, drop pre-release suffixes (``-rc1``,
+# ``-beta``), and take the highest by semver.
+#
+# Unauthenticated. The monthly scan calls this once per hook (four
+# times against today's config) — the unauthenticated GitHub API
+# limit is 60 req/h per IP, so a per-PAT/GITHUB_TOKEN bump to the
+# 5000 req/h authenticated bucket isn't worth the extra coupling.
+get_latest_precommit_hook_release() {
+  local repo_url="$1"
+  [ -n "$repo_url" ] || return 0
+
+  # Only GitHub is supported today. Strip any trailing ``.git`` or
+  # ``/`` so the owner/repo extraction works for both forms commonly
+  # seen in pre-commit configs.
+  local cleaned="${repo_url%.git}"
+  cleaned="${cleaned%/}"
+  case "$cleaned" in
+    https://github.com/*) ;;
+    *) return 0 ;;
+  esac
+
+  local owner_repo="${cleaned#https://github.com/}"
+  # Reject anything that isn't ``owner/repo`` (no extra path segments).
+  case "$owner_repo" in
+    */*/*) return 0 ;;
+    */*) ;;
+    *) return 0 ;;
+  esac
+
+  curl -fsSL --max-time 15 \
+    -H "Accept: application/vnd.github+json" \
+    -H "X-GitHub-Api-Version: 2022-11-28" \
+    "https://api.github.com/repos/${owner_repo}/tags?per_page=100" 2>/dev/null \
+    | python3 -c "
+import json, re, sys
+try:
+    data = json.load(sys.stdin)
+except Exception:
+    sys.exit(0)
+# pre-commit's ``rev:`` accepts ``vX.Y[.Z]``, ``X.Y[.Z]``, or full
+# SHAs. We compare on the semver-shaped ones; SHA-pinned hooks are
+# left alone (the helper returns empty and the caller skips them).
+pat = re.compile(r'^v?\d+\.\d+(?:\.\d+)?$')
+candidates = []
+for entry in data or []:
+    name = (entry or {}).get('name', '')
+    if not pat.match(name):
+        continue
+    stripped = name.lstrip('v')
+    parts = stripped.split('.')
+    try:
+        nums = tuple(int(p) for p in parts)
+    except ValueError:
+        continue
+    candidates.append((nums, name))
+if candidates:
+    print(max(candidates)[1])
+" 2>/dev/null
+}
+
@@ -14,8 +14,12 @@
 #     via OIDC; falls back gracefully when no creds are configured)
 #   - Aurora PostgreSQL engine versions (AWS creds via OIDC)
 #   - EMR Serverless release labels (AWS creds via OIDC)
-#   - Dockerfile.dev ARG pins (NODE_MAJOR, CDK_VERSION, KUBECTL_VERSION,
-#     AWSCLI_VERSION, DOCKER_VERSION) — public endpoints, no AWS creds needed
+#   - Dockerfile.dev ARG pins (NODE_MAJOR, NPM_VERSION, CDK_VERSION,
+#     KUBECTL_VERSION, AWSCLI_VERSION, DOCKER_VERSION) — public endpoints,
+#     no AWS creds needed
+#   - Pre-commit hook revisions in .pre-commit-config.yaml compared against
+#     the latest tag from the upstream Git host (unauthenticated; one call
+#     per hook well below the 60 req/h limit)
 #   - CDK enum constants in gco/stacks/constants.py compared against the
 #     latest aws-cdk-lib (LAMBDA_PYTHON_RUNTIME, AURORA_POSTGRES_VERSION)
 #   - Latest stable Python release from endoflife.date
 
@@ -16,7 +16,9 @@
 #   - security:pip-audit:deps            — pip-audit on requirements-lock.txt
 #   - security:semgrep:sast              — semgrep scan --config auto
 #   - security:trivy:filesystem          — trivy fs on the source tree
-#   - security:trivy:container-scan      — trivy image on each built service image
+#   - security:trivy:container-scan      — trivy image on each built image
+#                                          (4 service images + helm-installer
+#                                          + Dockerfile.dev)
 #   - security:trufflehog:secrets        — trufflehog filesystem scan
 #
 # All scans are strict: if a scan finds issues the job fails. Fix the code,
@@ -170,6 +172,12 @@ jobs:
           - image: helm-installer
             dockerfile: lambda/helm-installer/Dockerfile
             context: lambda/helm-installer
+          # Dev container — never deployed, but contributors run host-side
+          # gco/cdk/kubectl commands inside it. Scanning here means a CVE
+          # in the dev image surfaces on the same PR that introduced the
+          # bump (rather than waiting for the next weekly cve-scan run).
+          - image: dev
+            dockerfile: Dockerfile.dev
     steps:
       - uses: actions/checkout@v6
       - uses: docker/setup-buildx-action@v4
 
@@ -1,6 +1,6 @@
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.15.7
+    rev: v0.15.13
     hooks:
       # Formatter. Run before ``ruff`` so the check sees formatter output.
       - id: ruff-format
@@ -11,7 +11,7 @@ repos:
         args: [--fix, --exclude, lambda/kubectl-applier-simple-build, --exclude, lambda/helm-installer-build]
 
   - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v1.19.1
+    rev: v2.1.0
     hooks:
       - id: mypy
         additional_dependencies: [types-requests, types-PyYAML]