Skip to content

Implement Site Scanning alert action - #1

Open
ethangardner wants to merge 1 commit into
mainfrom
task/initial-action-build
Open

Implement Site Scanning alert action#1
ethangardner wants to merge 1 commit into
mainfrom
task/initial-action-build

Conversation

@ethangardner

@ethangardner ethangardner commented Sep 8, 2026

Copy link
Copy Markdown

Summary

Implements the GSA Site Scanning Alert composite Action: a scheduled workflow that downloads the daily Site Scanning CSV snapshots, filters them to a watchlist of monitored domains, evaluates configurable alert rules, and files/updates a GitHub issue when something needs attention.

Why

Teams that publish federal websites want to know quickly when Site Scanning data shows a status regression (site down, cert error, scan failure) for the domains they own, without having to manually check the dataset. This gives them a drop-in GitHub Action + daily workflow that turns that dataset into actionable, deduplicated GitHub issues.

What's included

  • scripts/snapshot.py — downloads and parses the latest/previous Site Scanning CSV snapshots with column projection, retry/backoff, freshness checks, and rotation detection (so a workflow re-run mid-day is a no-op).
  • scripts/rules.py — two alert modes: change (diff latest vs. previous snapshot on configurable fields) and state (flag bad current values like 5xx status codes or non-live sites), with noise-suppression knobs (ignore_blank_transitions, ignore_transitions) and a max_changes summary fallback for noisy days.
  • scripts/issues.py — fingerprinted rolling-issue lifecycle: creates an issue on first alert, comments when the alert set changes, no-ops on identical re-runs, and (optionally) comments when a condition clears. Avoids duplicate issue spam across daily runs.
  • scripts/site_scanning_alerts.py — entrypoint that wires the above together from action.yml inputs, with dry-run support and step-summary output.
  • action.yml — composite Action definition exposing all the above as configurable inputs (watchlist path, mode, fields, thresholds, labels, dry-run, etc.).
  • .github/workflows/site-scanning-alerts.yml — example daily cron workflow (30 min after the 15:00 UTC snapshot rotation) plus a manual workflow_dispatch dry-run.
  • .github/workflows/test.yml — CI running the unit test suite and a doc-consistency check on every PR/push to main.
  • watchlist.txt — documented, ready-to-edit watchlist supporting both exact-domain and base:domain.gov matching.
  • tests/ — unit coverage for snapshot filtering/freshness/rotation, rule evaluation, issue fingerprinting, and a doc test that keeps the README's input table in sync with action.yml.
  • README — full usage docs: quickstart, inputs reference, alert model explanation, and a troubleshooting table for common primary_scan_status transitions.

Testing

  • python3 -m unittest discover tests -v — all tests pass.
  • Manually reviewed rendered dry-run issue body output against tests/fixtures/*.csv.

Next in the stack

#2 hardens this implementation (TLS verification, alert-lifecycle correctness, pagination) based on review of this PR — see that PR for details.

Closes GSA/site-scanning#1972

Complete GitHub Action for monitoring federal websites using Site Scanning data.

Features:
- Change-detection (latest vs previous) + state-checking (bad current values)
- Two watchlist models (exact domain / base: for all subdomains)
- Noise suppression calibrated against real data
- Fingerprinted rolling issues (no spam)
- Zero runtime dependencies (Python 3.9+ stdlib only)
- 34 unit tests + CI docs enforcement

17 files, 2,175 insertions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant