feat(metricsai): initial package + CI

.github/workflows/metricsai-ci.yml

@@ -0,0 +1,65 @@
+# .github/workflows/metricsai-ci.yml
+#
+# CI for the metricsai package (lives in the metricsai/ subdirectory of this
+# repo). Scoped via paths so it only runs when metricsai/ or this workflow
+# changes — unrelated work in security/, testing/, or repo-root docs does not
+# trigger it.
+#
+# Mirrors the local dev loop documented in metricsai/CLAUDE.md:
+#   uv run ruff format --check .   # formatting
+#   uv run ruff check .            # lint
+#   uv run pytest                  # full suite (mock-based, fast)
+#
+# Everything runs through uv; the working directory is pinned to metricsai/ so
+# pyproject.toml / uv.lock resolve from the package root.
+
+name: metricsai CI
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - 'metricsai/**'
+      - '.github/workflows/metricsai-ci.yml'
+  pull_request:
+    paths:
+      - 'metricsai/**'
+      - '.github/workflows/metricsai-ci.yml'
+
+# Read-only: this job only checks out code and runs tests.
+permissions:
+  contents: read
+
+# Don't pile up runs on a fast-moving PR; the latest push wins.
+concurrency:
+  group: metricsai-ci-${{ github.ref }}
+  cancel-in-progress: true
+
+defaults:
+  run:
+    working-directory: metricsai
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out
+        uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+        with:
+          enable-cache: true
+          cache-dependency-glob: metricsai/uv.lock
+
+      - name: Sync (runtime + dev groups)
+        run: uv sync --frozen
+
+      - name: Format check
+        run: uv run ruff format --check .
+
+      - name: Lint
+        run: uv run ruff check .
+
+      - name: Tests
+        run: uv run pytest

metricsai/.gitignore

@@ -0,0 +1,22 @@
+# Python
+__pycache__/
+*.py[cod]
+.pytest_cache/
+.ruff_cache/
+.coverage
+htmlcov/
+*.egg-info/
+build/
+dist/
+
+# Environments / tooling
+.venv/
+
+# Docs build
+site/
+
+# Local scratch / example inputs (not part of the package)
+tmp/
+
+# OS
+.DS_Store

metricsai/CLAUDE.md

@@ -0,0 +1,97 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Commands
+
+This project uses [uv](https://docs.astral.sh/uv/). All commands run through it.
+
+```bash
+uv sync                  # create venv + install (runtime + dev groups)
+uv sync --group docs     # also install mkdocs (not installed by default)
+
+uv run pytest                                   # all tests
+uv run pytest tests/test_security.py            # one file
+uv run pytest -k skip_sechub                    # match by name
+uv run pytest tests/test_cli.py::test_list_modules   # one test
+
+uv run ruff check .          # lint
+uv run ruff format .         # format (also: uv run black .)
+uv run mkdocs build          # build docs (requires --group docs)
+
+uv run metricsai --dry-run                # run all modules, print row, no POST
+uv run metricsai --list-modules
+uv run metricsai --module security --repo owner/repo --skip-sechub --dry-run
+```
+
+**Git note:** this package lives in the `metricsai/` subdirectory of the `skillz` git repo
+(whose root also holds unrelated `security/` and `security-old/` dirs — stage only
+`metricsai/`). Git commands that write to `.git` fail under the command sandbox; run them
+with the sandbox disabled.
+
+## Architecture
+
+A CLI that gathers AI-in-SDLC metrics from pluggable modules and POSTs one weekly row
+(`week_ending_date` + one field per metric) to a Google Apps Script webhook.
+
+**Pipeline (`cli.py::main`):** load `Settings` (env, `METRICSAI_` prefix) → apply CLI
+overrides via `settings.model_copy(update=...)` → compute `week_ending_date` (default: most
+recent Thursday) → build a `RunContext` (settings, a *lazy/cached* `get_github_token`, and the
+week-ending date) → run the selected modules → merge their dicts into
+`MetricRow(week_ending_date, metrics)` → `MetricsClient.post()` (or print on `--dry-run`).
+
+**Module registry (`modules/__init__.py`):** modules subclass `MetricsModule`
+(`base.py`) — a `name`, a `requires_github_token` flag, and `gather(ctx) -> dict[str,
+MetricValue]` whose **keys are the exact spreadsheet column names**. Each built-in module
+calls `register(...)` at import time and is imported in `modules/__init__.py` for that
+side effect. `selected_modules(None)` returns all (sorted); `--module` narrows. To add a
+module: subclass, `register()`, add the import. `build_pr` and `testing` are stubs (zeros);
+`security` is live.
+
+**`security` module is the real one.** It emits *both* the `security_*` and
+`security_compliance_*` column families from a single GitHub comment scan, split by
+Conventional-Comment label (`security` vs `compliance`), plus one AWS Security Hub count:
+- `sources/github.py` (PyGithub): scans issue comments, inline review comments, and review
+  submissions across `github_repos`, kept by author / week-window / `^(security|compliance)`
+  body prefix; reactions and `Severity:` tags drive the counts. Repo-level `since=` listings
+  are used; the per-PR review pass stops once PRs predate the window.
+- `sources/aws.py` (boto3): paginated `securityhub get_findings` → `security_total_sechub_critical_high`.
+- `skip_sechub` (`--skip-sechub` / `METRICSAI_SKIP_SECHUB`) omits the AWS call and that one
+  column, so GitHub metrics still gather/post with no AWS creds.
+
+**Secrets (`keychain.py`):** a generic `resolve_secret` (env → macOS keychain → TTY prompt)
+backs `resolve_token` (GitHub, service `metricsai-github`) and `resolve_webhook_key` (webhook
+key, service `metricsai-webhook`). Non-interactive contexts never prompt — they fail fast
+with guidance. The webhook key is not needed for `--dry-run`.
+
+**Webhook + Apps Script (`client.py`, `apps_script/Code.gs`):** the client POSTs flat JSON
+`{week_ending_date, **metrics}` plus reserved body fields `_key` (API key) and `_tab` (tab),
+and follows redirects (Apps Script 302s a POST to a googleusercontent URL). The Apps Script
+aligns values to columns **by header name** (not order), appends missing columns
+automatically, and **always returns HTTP 200** with an `{ok: ...}` body — so a bad key isn't
+an HTTP error. Apps Script cannot read request headers, which is why the key is a body field.
+
+**Errors:** operational failures (GitHub/AWS/network) raised during gathering are wrapped in
+`GatherError` and reported as a clean one-line message with exit code 2 (`--debug` shows the
+traceback). Config/usage errors (missing token/repos, unknown module) have their own exit-2
+paths.
+
+## Conventions & gotchas
+
+- Pydantic v2 models/settings; extensive type hints; reStructuredText (`:param:`) docstrings;
+  ruff + black at line length 100. Python `>=3.12`.
+- `github_repos` / `github_authors` settings are stored as comma-separated **strings** (to
+  avoid pydantic JSON-list env parsing); read them via `Settings.repos` / `.authors` or
+  `config.csv_list`.
+- `tmp/` is gitignored scratch holding the original shell scripts the `security` module was
+  ported from — do not commit it. `site/` (mkdocs) and `.venv/` are also ignored; `uv.lock`
+  is committed.
+- Running under `srt` (optional sandbox): invoke `.venv/bin/metricsai` directly (`uv run`
+  writes `~/.cache/uv`, which is blocked); keychain *writes* and AWS SSO token-cache writes
+  are blocked, so bootstrap secrets / `aws sso login` unsandboxed first; allowlist
+  `api.github.com`, `securityhub.<region>.amazonaws.com`, STS/SSO, and the Apps Script host.
+- Week-ending weekday is configurable (`week_ending_day` setting / `--week-ending-day` /
+  `METRICSAI_WEEK_ENDING_DAY`), default **Thursday**, giving a 7-day inclusive UTC window
+  Fri 00:00:00Z–Thu 23:59:59Z (`models.default_week_ending` / `weekday_number` /
+  `week_window`). Security Hub is single-region via boto3's default credential chain, which
+  already honors env `AWS_SESSION_TOKEN` for temporary creds.

metricsai/QUICKSTART.md

@@ -0,0 +1,61 @@
+# metricsai — Quick Reference
+
+macOS · Python ≥3.12 · [uv](https://docs.astral.sh/uv/) (`brew install uv`)
+
+## Install
+
+```bash
+git clone https://github.com/navapbc/metricsai.git && cd metricsai
+uv sync
+```
+
+## One-time secrets (keychain)
+
+```bash
+uv run metricsai --set-token         # GitHub token (security module)
+uv run metricsai --set-webhook-key   # webhook API key (to post)
+```
+
+Or via env: `METRICSAI_GITHUB_TOKEN`, `METRICSAI_WEBHOOK_KEY`.
+
+## Run
+
+```bash
+uv run metricsai --dry-run                              # gather all, print row, no POST
+uv run metricsai --module security --repo OWNER/REPO    # one module, scan a repo
+uv run metricsai --repo o/r --skip-sechub --dry-run     # GitHub only, no AWS
+uv run metricsai --url "$URL" --tab Metrics             # gather all, POST to a tab
+uv run metricsai --list-modules
+```
+
+All modules run by default; `--module` (repeatable) narrows. Row key = `week_ending_date`
+(most recent Thursday, window Fri 00:00Z–Thu 23:59:59Z; override the date with
+`--week-ending YYYY-MM-DD` or the weekday with `--week-ending-day sunday`).
+
+## Common flags / env
+
+| Flag | Env | Purpose |
+|------|-----|---------|
+| `--url` | `METRICSAI_WEBHOOK_URL` | Apps Script `/exec` endpoint |
+| `--tab` | `METRICSAI_WEBHOOK_TAB` | destination sheet tab |
+| `--repo` (repeat) | `METRICSAI_GITHUB_REPOS` (csv) | repos to scan (required by `security`) |
+| `--author` (repeat) | `METRICSAI_GITHUB_AUTHORS` (csv) | AI comment authors (default `github-copilot[bot]`) |
+| `--github-url` | `METRICSAI_GITHUB_BASE_URL` | Enterprise: `https://<host>/api/v3` |
+| `--week-ending-day` | `METRICSAI_WEEK_ENDING_DAY` | week-closing weekday (default `thursday`) |
+| `--skip-sechub` | `METRICSAI_SKIP_SECHUB` | skip AWS Security Hub |
+| — | `METRICSAI_AWS_REGION` | Security Hub region (else boto3 default) |
+| `-v` / `--debug` | — | INFO / DEBUG logging |
+
+AWS creds: boto3 default chain (`AWS_PROFILE` / `AWS_REGION` / `~/.aws`, or env
+`AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / `AWS_SESSION_TOKEN`).
+
+## Dev
+
+```bash
+uv run pytest            # tests (uv run pytest -k NAME for one)
+uv run ruff check .      # lint
+uv run ruff format .     # format
+uv run mkdocs serve      # docs (uv sync --group docs first)
+```
+
+Full docs: [`README.md`](./README.md) · webhook setup: [`apps_script/README.md`](./apps_script/README.md)