signups, modal, style, AGENTS, update README

Author: jackharrhy

AGENTS.md

@@ -0,0 +1,53 @@
+# AGENTS.md
+
+## Project overview
+
+Where the Plow -- real-time and historical snowplow tracker for St. John's, NL.
+Python FastAPI backend serving a vanilla JS/HTML/CSS frontend. DuckDB for
+storage. No build step, no frontend framework.
+
+Production: https://plow.jackharrhy.dev
+
+## Architecture
+
+- **Backend:** `src/where_the_plow/` -- FastAPI app (`main.py`), routes
+  (`routes.py`), Pydantic models (`models.py`), DuckDB wrapper (`db.py`),
+  AVL API client (`client.py`), background collector (`collector.py`),
+  realtime snapshot builder (`snapshot.py`), file-based coverage cache
+  (`cache.py`), config (`config.py`).
+- **Frontend:** `src/where_the_plow/static/` -- three files only:
+  `index.html`, `app.js`, `style.css`. MapLibre GL JS for the map,
+  noUiSlider for time range controls. All loaded from CDN, no bundler.
+- **Styling:** Plain CSS with custom properties defined in `:root`
+  (`--color-*`, `--border-*`, `--font-sans`). Dark theme. Use the
+  existing tokens -- don't hardcode colors.
+
+## Database & migrations
+
+DuckDB with the spatial extension. Schema is defined in `db.py`
+`Database.init()` using `CREATE TABLE IF NOT EXISTS`.
+
+**Important:** `CREATE TABLE IF NOT EXISTS` does NOT alter existing tables.
+If you add columns to a table definition, existing production databases will
+silently keep the old schema and inserts referencing new columns will fail.
+
+When adding columns to existing tables, you MUST add an explicit migration
+in `Database.init()` after the table creation block. The pattern:
+
+1. Query `information_schema.columns` to check if the column exists.
+2. Run `ALTER TABLE ... ADD COLUMN` if it doesn't.
+
+See the existing migrations in `db.init()` for examples (e.g. `geom` on
+`positions`, `ip`/`user_agent` on `viewports` and `signups`).
+
+## Key conventions
+
+- No frontend framework -- all DOM manipulation is vanilla JS with
+  `getElementById` / `addEventListener`.
+- Two main JS classes: `PlowMap` (map layer management) and `PlowApp`
+  (application state and logic). Event wiring is at the bottom of `app.js`.
+- Analytics/write endpoints (`/track`, `/signup`) use the `RateLimiter`
+  class in `routes.py` for in-memory per-IP rate limiting, and store
+  `ip` + `user_agent` for fingerprinting.
+- `_client_ip(request)` helper in `routes.py` extracts IP from
+  `X-Forwarded-For` with fallback to `request.client.host`.

README.md

@@ -1,6 +1,6 @@
-# Where the Plow
+# where the plow
 
-Real-time tracking of City of St. John's snowplow vehicles.
+Real-time and historical tracking of City of St. John's snowplow vehicles.
 
 Polls the city's public AVL (Automatic Vehicle Location) API every 6 seconds, stores historical position data in DuckDB, serves it as GeoJSON, and visualizes it on a live map.
 
@@ -12,6 +12,12 @@ Polls the city's public AVL (Automatic Vehicle Location) API every 6 seconds, st
 
 Requires [uv](https://docs.astral.sh/uv/):
 
+```
+uv run cli.py dev
+```
+
+This starts the app with auto-reload and sets `DB_PATH=./data/plow.db`. You can also run uvicorn directly:
+
 ```
 uv run uvicorn where_the_plow.main:app --host 0.0.0.0 --port 8000
 ```
@@ -39,12 +45,16 @@ All geo endpoints return GeoJSON. Full OpenAPI docs at [`/docs`](https://plow.ja
 
 | Endpoint | Description |
 |---|---|
-| `GET /vehicles` | Latest position for every vehicle |
+| `GET /vehicles` | Latest position for every vehicle (with mini-trails) |
 | `GET /vehicles/nearby?lat=&lng=&radius=` | Vehicles within radius (meters) |
 | `GET /vehicles/{id}/history?since=&until=` | Position history for one vehicle |
 | `GET /coverage?since=&until=` | Per-vehicle LineString trails with timestamps |
 | `GET /stats` | Collection statistics |
 | `GET /health` | Health check |
+| `POST /track` | Record anonymous viewport focus event |
+| `POST /signup` | Email signup for notifications |
+
+All GET list endpoints support cursor-based pagination via `limit` and `after` query parameters. Write endpoints (`/track`, `/signup`) are rate-limited per IP.
 
 ## Database schema
 
@@ -76,6 +86,8 @@ CREATE TABLE positions (
 
 Deduplication is by `(vehicle_id, timestamp)` composite key -- if the API returns the same `LocationDateTime` for a vehicle, the row is skipped.
 
+There are also `viewports` (analytics) and `signups` (email signups) tables -- see `db.py` for their full schemas.
+
 ## Stack
 
-Python 3.12, FastAPI, DuckDB (spatial), httpx, MapLibre GL JS, Docker.
+Python 3.12, FastAPI, DuckDB (spatial), httpx, MapLibre GL JS, noUiSlider, Docker.

docs/plans/2026-02-19-api-and-map-impl.md

@@ -844,7 +844,7 @@ Update `main.py` to include the router and add OpenAPI metadata:
 from where_the_plow.routes import router
 
 app = FastAPI(
-    title="Where the Plow",
+    title="where the plow",
     description="Real-time and historical plow tracker for the City of St. John's. "
                 "All geo endpoints return GeoJSON FeatureCollections with cursor-based pagination.",
     version="0.1.0",

docs/plans/2026-02-19-data-collection-impl.md

@@ -799,7 +799,7 @@ async def lifespan(app: FastAPI):
     logger.info("Shutdown complete")
 
 
-app = FastAPI(title="Where the Plow", lifespan=lifespan)
+app = FastAPI(title="where the plow", lifespan=lifespan)
 
 
 @app.get("/health")

docs/poll_rate.py

@@ -18,8 +18,6 @@
 
 import argparse
 import time
-import json
-import sys
 from datetime import datetime, timezone
 
 import httpx
@@ -101,17 +99,26 @@ def diff_snapshots(prev: dict[str, dict], curr: dict[str, dict]) -> dict:
 
 
 def main():
-    parser = argparse.ArgumentParser(description="Poll AVL endpoint to measure update rate")
-    parser.add_argument("--duration", type=int, default=120, help="How long to poll in seconds (default: 120)")
-    parser.add_argument("--interval", type=int, default=3, help="Poll interval in seconds (default: 3)")
+    parser = argparse.ArgumentParser(
+        description="Poll AVL endpoint to measure update rate"
+    )
+    parser.add_argument(
+        "--duration",
+        type=int,
+        default=120,
+        help="How long to poll in seconds (default: 120)",
+    )
+    parser.add_argument(
+        "--interval", type=int, default=3, help="Poll interval in seconds (default: 3)"
+    )
     args = parser.parse_args()
 
     duration = args.duration
     interval = args.interval
     total_ticks = duration // interval
 
     print(f"Polling every {interval}s for {duration}s ({total_ticks} ticks)")
-    print(f"Filtering: isDriving = 'maybe' (active vehicles only)")
+    print("Filtering: isDriving = 'maybe' (active vehicles only)")
     print("-" * 70)
 
     client = httpx.Client()
@@ -146,7 +153,9 @@ def main():
                 ticks_with_changes += 1
                 updated = [v for v in changes.values() if v["type"] == "updated"]
                 appeared = [v for v in changes.values() if v["type"] == "appeared"]
-                disappeared = [v for v in changes.values() if v["type"] == "disappeared"]
+                disappeared = [
+                    v for v in changes.values() if v["type"] == "disappeared"
+                ]
 
                 parts = []
                 if updated:
@@ -156,12 +165,16 @@ def main():
                 if disappeared:
                     parts.append(f"{len(disappeared)} disappeared")
 
-                print(f"[{now}] Tick {tick:3d}: {len(curr)} vehicles | {', '.join(parts)}")
+                print(
+                    f"[{now}] Tick {tick:3d}: {len(curr)} vehicles | {', '.join(parts)}"
+                )
 
                 # Track per-vehicle updates
                 for vid, change in changes.items():
                     if change["type"] == "updated":
-                        vehicle_update_counts[vid] = vehicle_update_counts.get(vid, 0) + 1
+                        vehicle_update_counts[vid] = (
+                            vehicle_update_counts.get(vid, 0) + 1
+                        )
                         vehicle_descriptions[vid] = change["description"]
                     # Track descriptions/types from current data
                     if vid in curr:
@@ -188,9 +201,11 @@ def main():
     print("RESULTS")
     print("=" * 70)
     print(f"Total ticks polled:        {actual_ticks}")
-    print(f"Ticks with changes:        {ticks_with_changes} ({ticks_with_changes/actual_ticks*100:.1f}%)")
+    print(
+        f"Ticks with changes:        {ticks_with_changes} ({ticks_with_changes / actual_ticks * 100:.1f}%)"
+    )
     print(f"Ticks with no changes:     {actual_ticks - ticks_with_changes}")
-    print(f"Avg changes per tick:      {sum(tick_change_counts)/actual_ticks:.1f}")
+    print(f"Avg changes per tick:      {sum(tick_change_counts) / actual_ticks:.1f}")
     print(f"Max changes in one tick:   {max(tick_change_counts)}")
     print()
 
@@ -199,7 +214,7 @@ def main():
         print()
         print("Per-vehicle update frequency (sorted by most updates):")
         print(f"  {'Vehicle':<30} {'Type':<16} {'Updates':>8} {'Rate':>10}")
-        print(f"  {'-'*30} {'-'*16} {'-'*8} {'-'*10}")
+        print(f"  {'-' * 30} {'-' * 16} {'-' * 8} {'-' * 10}")
         for vid, count in sorted(vehicle_update_counts.items(), key=lambda x: -x[1]):
             desc = vehicle_descriptions.get(vid, vid)
             vtype = vehicle_types.get(vid, "?")
@@ -209,13 +224,23 @@ def main():
     print()
     print("Interpretation:")
     if ticks_with_changes / actual_ticks > 0.8:
-        print(f"  Data updates very frequently. Polling every {interval}s is reasonable.")
+        print(
+            f"  Data updates very frequently. Polling every {interval}s is reasonable."
+        )
     elif ticks_with_changes / actual_ticks > 0.4:
-        effective = actual_ticks * interval // ticks_with_changes if ticks_with_changes else 0
-        print(f"  Data updates moderately. Consider polling every ~{effective}s instead.")
+        effective = (
+            actual_ticks * interval // ticks_with_changes if ticks_with_changes else 0
+        )
+        print(
+            f"  Data updates moderately. Consider polling every ~{effective}s instead."
+        )
     else:
-        effective = actual_ticks * interval // ticks_with_changes if ticks_with_changes else 0
-        print(f"  Data updates infrequently. Polling every ~{effective}s would be sufficient.")
+        effective = (
+            actual_ticks * interval // ticks_with_changes if ticks_with_changes else 0
+        )
+        print(
+            f"  Data updates infrequently. Polling every ~{effective}s would be sufficient."
+        )
 
 
 if __name__ == "__main__":

src/where_the_plow/db.py

@@ -59,6 +59,8 @@ def init(self):
             CREATE TABLE IF NOT EXISTS viewports (
                 id          BIGINT DEFAULT nextval('viewports_seq') PRIMARY KEY,
                 timestamp   TIMESTAMPTZ NOT NULL DEFAULT now(),
+                ip          VARCHAR,
+                user_agent  VARCHAR,
                 zoom        DOUBLE NOT NULL,
                 center_lng  DOUBLE NOT NULL,
                 center_lat  DOUBLE NOT NULL,
@@ -68,6 +70,22 @@ def init(self):
                 ne_lat      DOUBLE NOT NULL
             )
         """)
+        cur.execute("""
+            CREATE SEQUENCE IF NOT EXISTS signups_seq
+        """)
+        cur.execute("""
+            CREATE TABLE IF NOT EXISTS signups (
+                id              BIGINT DEFAULT nextval('signups_seq') PRIMARY KEY,
+                timestamp       TIMESTAMPTZ NOT NULL DEFAULT now(),
+                email           VARCHAR NOT NULL,
+                ip              VARCHAR,
+                user_agent      VARCHAR,
+                notify_plow     BOOLEAN NOT NULL DEFAULT FALSE,
+                notify_projects BOOLEAN NOT NULL DEFAULT FALSE,
+                notify_siliconharbour BOOLEAN NOT NULL DEFAULT FALSE,
+                note            VARCHAR
+            )
+        """)
 
         # Migration: add geom column to existing tables that lack it
         cols = cur.execute(
@@ -82,6 +100,33 @@ def init(self):
             "UPDATE positions SET geom = ST_Point(longitude, latitude) WHERE geom IS NULL"
         )
 
+        # Migration: add ip/user_agent columns to viewports
+        vp_cols = {
+            r[0]
+            for r in cur.execute(
+                "SELECT column_name FROM information_schema.columns "
+                "WHERE table_name='viewports'"
+            ).fetchall()
+        }
+        if "ip" not in vp_cols:
+            cur.execute("ALTER TABLE viewports ADD COLUMN ip VARCHAR")
+        if "user_agent" not in vp_cols:
+            cur.execute("ALTER TABLE viewports ADD COLUMN user_agent VARCHAR")
+
+        # Migration: add ip/user_agent columns to signups
+        su_cols = {
+            r[0]
+            for r in cur.execute(
+                "SELECT column_name FROM information_schema.columns "
+                "WHERE table_name='signups'"
+            ).fetchall()
+        }
+        if su_cols:  # table exists from a prior run
+            if "ip" not in su_cols:
+                cur.execute("ALTER TABLE signups ADD COLUMN ip VARCHAR")
+            if "user_agent" not in su_cols:
+                cur.execute("ALTER TABLE signups ADD COLUMN user_agent VARCHAR")
+
     def upsert_vehicles(self, vehicles: list[dict], now: datetime):
         cur = self._cursor()
         for v in vehicles:
@@ -408,15 +453,69 @@ def insert_viewport(
         sw_lat: float,
         ne_lng: float,
         ne_lat: float,
+        ip: str | None = None,
+        user_agent: str | None = None,
     ):
         """Record a user viewport focus event."""
         self._cursor().execute(
             """
-            INSERT INTO viewports (zoom, center_lng, center_lat, sw_lng, sw_lat, ne_lng, ne_lat)
+            INSERT INTO viewports (ip, user_agent, zoom, center_lng, center_lat, sw_lng, sw_lat, ne_lng, ne_lat)
+            VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
+            """,
+            [
+                ip,
+                user_agent,
+                zoom,
+                center_lng,
+                center_lat,
+                sw_lng,
+                sw_lat,
+                ne_lng,
+                ne_lat,
+            ],
+        )
+
+    def insert_signup(
+        self,
+        email: str,
+        ip: str | None = None,
+        user_agent: str | None = None,
+        notify_plow: bool = False,
+        notify_projects: bool = False,
+        notify_siliconharbour: bool = False,
+        note: str | None = None,
+    ):
+        """Record an email signup."""
+        self._cursor().execute(
+            """
+            INSERT INTO signups (email, ip, user_agent, notify_plow, notify_projects, notify_siliconharbour, note)
             VALUES (?, ?, ?, ?, ?, ?, ?)
             """,
-            [zoom, center_lng, center_lat, sw_lng, sw_lat, ne_lng, ne_lat],
+            [
+                email,
+                ip,
+                user_agent,
+                notify_plow,
+                notify_projects,
+                notify_siliconharbour,
+                note,
+            ],
+        )
+
+    def count_recent_signups(self, ip: str, minutes: int = 30) -> int:
+        """Count signups from an IP in the last N minutes."""
+        row = (
+            self._cursor()
+            .execute(
+                """
+            SELECT count(*) FROM signups
+            WHERE ip = ? AND timestamp > now() - INTERVAL (?) MINUTE
+            """,
+                [ip, minutes],
+            )
+            .fetchone()
         )
+        return row[0] if row else 0
 
     def close(self):
         self.conn.close()

src/where_the_plow/main.py

@@ -40,7 +40,7 @@ async def lifespan(app: FastAPI):
 
 
 app = FastAPI(
-    title="Where the Plow",
+    title="where the plow",
     description="Real-time and historical plow tracker for the City of St. John's. "
     "All geo endpoints return GeoJSON FeatureCollections with cursor-based pagination.\n\n"
     "**WARNING:** This API is not stable. Monitor the `openapi.json` file for breaking changes. "