create-extractor: convert to a scheduled gcec task, and retire fetch-page.yml

The auto-implement-extractor pipeline moves from the legacy label-wired
routine (dev/routines/create-extractor/) to a gcec pack task on the
per-repo scheduler, split by what each stage is actually the right place
for:

- precondition (task.mjs) — pure code over the `issues` signal: is any
  open extractor-request eligible? No I/O, no writes, no body parsing.
  Almost every hour answers no and costs nothing. An `extractor-in-progress`
  claim label keeps that correct across hours, since preprocessing runs
  before (and independently of) the dispatch issue's at-most-one-open guard
  and would otherwise re-scaffold a live request.
- preprocessing (prepare.mjs) — everything deterministic, Action-side:
  triage, closing deny/allow/duplicate with their canned messages, claim,
  branch, scaffold, offline baseline, the ScraperAPI page fetch, push, and
  the draft PR. All of it needs issue bodies, GitHub writes, or a network
  fetch — none of which a precondition may do. It requests the agent only
  when a real page recorded and an extract() is left to write.
- agent (task.md) — the judgment only, roughly halving the prose: read the
  page, write extract(), fill the case from a real run, verify, deliver.
  The four-script/JSON-blob hand-off is gone; postconditions.sh now derives
  its names from the branch instead of taking five env vars.

fetch-page.yml is deleted. It existed solely because an agent session
cannot hold SCRAPER_API_KEY, forcing a dispatch → poll → pull round-trip
for one curl; preprocessing runs inside Actions, and the task declares
`agent_preprocessing_secrets: ['SCRAPER_API_KEY']`. Its other caller — the
hand-dispatched add-live-case / gardening flow — becomes a new agentless
`record-page` task: a committed `.url` with no sibling `.html` is the whole
request, and the task records it and opens the PR. Both tasks fetch through
one shared scraperapi.mjs, so the project keeps a single page-fetching
surface.

The vendored mount carries the engine files this depends on (the secrets
contract); the stamp is deliberately left where it is, so the next
baselining still converges the mount to canon head normally.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016NgBmWQmxLFPybrhSTz8i3

Author: claude

.claudinite/local/packs/gcec/RULES.md

@@ -4,7 +4,7 @@ The project's standing working rules — injected at session start while this pa
 is declared. Activity-scoped procedures live in this pack's skills
 (snapshot-approval, merge-and-ci, testing-guide, add-live-case) and surface on
 demand; the extractor-automation domain's standing rules are the "Extractor
-pipeline" section below, and its two routines live under `dev/routines/`.
+pipeline" section below, and its three scheduled tasks live under `tasks/`.
 
 ## Working rules
 
@@ -137,32 +137,35 @@ classification: the `Release` stub (`chrome-extension-release.yml`) is
 unattended and already covered — the reporters fire inside the vendored
 create-package/publish/daily workflows, keyed per operation, with per-repo
 values in `.github/release.config`; `test.yml` is attended PR CI — no reporter;
-`fetch-page.yml` is attended by its dispatcher (see the extractor pipeline
-section below); a **new** unattended workflow adds a failure job per the canon
-action header's recipe.
+`claudinite-scheduler.yml` is the vendored scheduler, which converges its own
+failures to `needs-human` issues; a **new** unattended workflow adds a failure job
+per the canon action header's recipe.
 
 ## Extractor pipeline
 
-Standing rules for the extractor-automation domain — the create-extractor routine
-(an `extractor-request` issue → a PR adding site support), still under
-[`dev/routines/`](../../../../dev/routines/), and the weekly
-fallback-extractor-improvements task, now a gcec pack task under
-[`tasks/`](tasks/fallback-extractor-improvements/) (read a spec only when working
+Standing rules for the extractor-automation domain — the three gcec pack
+[`tasks/`](tasks/): **create-extractor** (an `extractor-request` issue → a PR
+adding site support), **record-page** (records a committed `.url`'s cached page),
+and the weekly **fallback-extractor-improvements** (read a spec only when working
 on that pipeline). Adding a cached live case is the
 [add-live-case](skills/add-live-case/SKILL.md) skill.
 
-- **All page fetching is delegated to ScraperAPI via the fetch-page workflow**
-  (`.github/workflows/fetch-page.yml`): a bare curl through ScraperAPI's
-  residential proxy with `render=true`, so a single-page app records post-render
-  HTML with real data. Bot-blocking from CI/sandbox IPs is the portable rule
-  maintained in the canon; here the escape hatch is the `SCRAPER_API_KEY`
-  **GitHub Actions secret** — a page is recorded by dispatching that workflow (the
-  create-extractor routine does this in its step 4), never by a local fetch (this
-  sandbox is bot-blocked). ScraperAPI is the whole fetching surface — swap the
-  vendor in that one workflow if it underperforms. The aid for a flaky SPA render
-  is the **`Wait-for selector`** a source request can carry
+- **All page fetching goes through [`scraperapi.mjs`](scraperapi.mjs), from a
+  task's preprocessing worker and nowhere else.** A rendered fetch through
+  ScraperAPI's residential proxy (`render=true`, so a single-page app records
+  post-render HTML with real data). Bot-blocking from CI/sandbox IPs is the
+  portable rule maintained in the canon; here the escape hatch is the
+  `SCRAPER_API_KEY` **GitHub Actions secret**, which reaches a worker because its
+  task declares `agent_preprocessing_secrets: ['SCRAPER_API_KEY']` and reaches
+  nothing else — never a local fetch (this sandbox is bot-blocked), and never an
+  agent session (which holds no repo secrets). **Never re-introduce a workflow
+  whose only job is to hold that secret for an agent**: `fetch-page.yml` was
+  exactly that, and the dispatch/poll/pull round-trip it forced on the agent is
+  what preprocessing removed. ScraperAPI is the whole fetching surface — swap the
+  vendor in that one module if it underperforms. The aid for a flaky SPA render is
+  the **`Wait-for selector`** a source request can carry
   (`extension/events-popup/derive-wait-selector.js`, a source-request tool, NOT an
-  event extractor, #603), passed to the workflow as `wait_for_selector`.
+  event extractor, #603), passed through as `wait_for_selector`.
 - **Facebook can't be a cached live case** — a hard HTTP 400 even through the
   proxy, so its extraction stays unit-tests-only
   (`extension-test/event-extractors/extraction.test.js`).
@@ -179,11 +182,12 @@ on that pipeline). Adding a cached live case is the
   from `extract-unsupported.js`'s `pageUsesDayFirstDates`), mirroring
   `derive-timezone.js`'s locale read; unambiguous dates (a part > 12) and the
   `.` / `-` separators are always day-first regardless (#686).
-- **`fetch-page.yml` is attended by its dispatcher** (the workflow-failure
-  classification above): the create-extractor routine dispatches it, polls the
-  run, and on failure labels the issue `extractor-blocked-needs-human` — a red run
-  reaches a human through the routine, not the Actions list, so the workflow
-  carries no failure reporter (and being dispatch-only it never runs unwatched).
+- **An unrecordable page is a dead end, not a failed run.** When a fetch can't
+  produce a page (bot wall, dead URL, empty render), create-extractor's
+  preprocessing labels the request `extractor-blocked-needs-human` with the reason
+  and exits **0** — a task failure would converge to a `needs-human` dispatch
+  issue as well, duplicating the signal and implying the pipeline broke when it
+  correctly declined. Same for record-page: the pages that did record still land.
 - **The fallback-coverage gate is a high-watermark over a changing case set.** It
   ratchets up on an unchanged case set and re-anchors when the set changes,
   compared over the cases the runs **share** — so adding an extractor never fails

.claudinite/local/packs/gcec/scraperapi.mjs

@@ -0,0 +1,41 @@
+// Recording an event page through ScraperAPI — the project's ONE page-fetching
+// surface (the gcec RULES.md extractor-pipeline rule). Lives at the pack root
+// rather than inside a task because two tasks need it: `create-extractor` records
+// the page for a new request, and `record-page` records any case whose `.url` has
+// no committed `.html` yet. Lifting it to their common ancestor keeps one
+// implementation of "how this project fetches a page" — swap the vendor here if
+// ScraperAPI underperforms, and both tasks follow.
+//
+// Only usable from a task's `agent_preprocessing` worker: it needs
+// SCRAPER_API_KEY, which reaches a worker through the task's declared
+// `agent_preprocessing_secrets` and reaches nothing else (an executor session
+// holds no repo secrets). This sandbox is bot-blocked, so a direct fetch from a
+// session would fail even with a key.
+
+import { mkdirSync, writeFileSync, statSync } from 'node:fs';
+import { dirname } from 'node:path';
+
+// The ScraperAPI request URL for a page. `render=true` executes the page's JS so a
+// single-page app records real content instead of an empty shell; the optional
+// wait-for selector (the extension derives it from the live page, #603) makes the
+// snapshot wait for real content to appear first. Both are url-encoded, so their
+// own &/?/# can never leak as sibling ScraperAPI params.
+export function scraperUrl(key, pageUrl, waitSelector) {
+  const params = new URLSearchParams({ api_key: key, render: 'true', url: pageUrl });
+  if (waitSelector) params.set('wait_for_selector', waitSelector);
+  return `https://api.scraperapi.com/?${params}`;
+}
+
+// Record `pageUrl` to `outPath`. Returns the byte count written, or throws with a
+// message written to land verbatim on a GitHub issue — the caller's job is to
+// decide whether an unfetchable page is a dead end (hand it to a human) or a
+// failure, not to re-diagnose it.
+export async function recordPage(key, { url, waitSelector }, outPath, { timeoutMs = 180_000 } = {}) {
+  const res = await fetch(scraperUrl(key, url, waitSelector), { signal: AbortSignal.timeout(timeoutMs) });
+  if (!res.ok) throw new Error(`ScraperAPI returned HTTP ${res.status} for ${url}`);
+  const html = await res.text();
+  if (!html.trim()) throw new Error(`ScraperAPI returned an empty document for ${url}`);
+  mkdirSync(dirname(outPath), { recursive: true });
+  writeFileSync(outPath, html);
+  return statSync(outPath).size;
+}

.claudinite/local/packs/gcec/skills/add-live-case/SKILL.md

@@ -17,29 +17,37 @@ New cached HTML can't be fetched from this sandbox (bot-blocked — see the
 gcec pack's RULES.md, "Extractor pipeline" section), so record the page *before* writing the
 case and read its exact `expected` off the committed file instead of guessing:
 
-1. Commit one new file — but **not** the case file yet:
+1. Land one new file on `main` — but **not** the case file yet:
    `dev/requirements/extractor/data/server-fetched/<name>.url` — plain text,
    just the event page URL. This file stays for good: it is the **single source
-   of truth** for the page's URL (used by the fetch workflow and by
-   `live.test.js`), so the URL is **not** repeated in the case file.
-2. Push the branch, then **dispatch `.github/workflows/fetch-page.yml`** on it
-   (Actions tab / API) with inputs `case_name=<name>` and `url=<the event URL>`
-   (plus `wait_for_selector` when the request carries one). It records the page
-   via ScraperAPI and commits `<name>.html` back to the branch with
-   `[skip ci]`; `test:live` stays green because no case asserts it yet. The
-   workflow is dispatched **per page, never a bulk refresh**, and its commit
-   only touches `data/**`, which `test.yml`'s `paths-ignore` skips.
-3. Pull, then add `dev/requirements/extractor/expected/<name>.json` (same
-   `<name>`, just `description` + `expected`, no `url`) and run
-   `npm run test:live` — it now runs against the local cached HTML, so its
-   output gives you the exact `expected` to paste in. Commit and push.
+   of truth** for the page's URL (it is what the recorder fetches and what
+   `live.test.js` sets the DOM origin from), so the URL is **not** repeated in
+   the case file.
+2. Wait for the **`record-page` task** to record it. There is nothing to
+   dispatch: that task runs on the hourly scheduler, and a committed `.url` with
+   no sibling `.html` *is* the request. It fetches the page through ScraperAPI
+   (whose secret only a task's preprocessing worker can reach — see the gcec
+   pack's RULES.md, "Extractor pipeline") and opens a **Record cached event
+   pages** PR carrying `<name>.html`. Merge it. `test:live` stays green
+   throughout, because no case asserts the page yet.
+3. Add `dev/requirements/extractor/expected/<name>.json` (same `<name>`, just
+   `description` + `expected`, no `url`) and run `npm run test:live` — it now
+   runs against the local cached HTML, so its output gives you the exact
+   `expected` to paste in. Commit and push.
+
+For a **new site** (or a new behavior on a supported one), you usually don't need
+this at all: file an `extractor-request` issue with the event URL and the
+`create-extractor` task does the whole thing — records the page, scaffolds the
+case, writes the extractor, and opens the PR.
 
 `expected.events` is the **complete, exact** array the extractor produces —
 deep-equal on `title`, `start`, `end`, `location`, `ctz`, and `details`, no
 matchers, array length included (one event for an ordinary page, several for a
 listing/series page). See `live.test.js`'s header for how each field derives.
 
-**Gardening:** when an event page is eventually taken down, point
-`<name>.url` at a newer event and refresh its cached HTML the same way. Until a
-cached HTML file exists for a case, `test:live` (and the Tests workflow) fails
-with `Missing cached HTML for "<name>"`.
+**Gardening:** when an event page is eventually taken down, point `<name>.url` at
+a newer event **and delete the stale `<name>.html`** in the same commit — that is
+the whole re-record request; `record-page` fetches the new page on its next run.
+Until a cached HTML file exists for a case, `test:live` (and the Tests workflow)
+fails with `Missing cached HTML for "<name>"`, so land the refresh PR before (or
+with) the case's updated `expected`.

.claudinite/local/packs/gcec/skills/testing-guide/SKILL.md

@@ -53,7 +53,7 @@ matchers, array length included. The page URL lives **only** in
 and the test both read it; never in the case file). Tests run offline against
 the committed cached HTML, loaded into a DOM at the `.url`'s URL so hostname
 detection behaves exactly as in Chrome. `test.yml` never fetches — recording
-goes through the fetch-page workflow (see add-live-case).
+goes through the record-page task (see add-live-case).
 
 ## UI snapshots
 

…es/create-extractor/attach-sample-url.js …ks/create-extractor/attach-sample-url.jsdev/routines/create-extractor/attach-sample-url.js renamed to .claudinite/local/packs/gcec/tasks/create-extractor/attach-sample-url.js dev/routines/create-extractor/attach-sample-url.js renamed to .claudinite/local/packs/gcec/tasks/create-extractor/attach-sample-url.js

@@ -1,6 +1,6 @@
 // Fold a deferred extractor request's event URL into the LEADER issue's body as
-// an extra "sample page" (see 2-triage.js's "sample" disposition
-// and routine.md). Two same-host requests can't run the agent
+// an extra "sample page" (see triage.js's "sample" disposition, applied by
+// prepare.mjs). Two same-host requests can't run the agent
 // concurrently, but the newer one's URL is a second real event page — useful raw
 // material for a more robust extractor. So instead of discarding it, the prepare
 // workflow records it on the leader issue as a checklist item a maintainer (the

…outines/create-extractor/case-quality.js …c/tasks/create-extractor/case-quality.jsdev/routines/create-extractor/case-quality.js renamed to .claudinite/local/packs/gcec/tasks/create-extractor/case-quality.js dev/routines/create-extractor/case-quality.js renamed to .claudinite/local/packs/gcec/tasks/create-extractor/case-quality.js

@@ -1,8 +1,8 @@
 // Deterministic branch / cache-file names for the auto-implement-extractor
-// workflow, derived purely from the event URL's host. The workflow does Phase 1
-// (branch + cache the page) before the agent runs, so these names can't be the
+// task, derived purely from the event URL's host. The task's preprocessing
+// branches and records the page before the agent runs, so these names can't be the
 // agent's judgement call any more — they have to be reproducible from the URL
-// alone. Pure (no I/O), so it's unit-tested, and the workflow and triage agree
+// alone. Pure (no I/O), so it's unit-tested, and preprocessing and triage agree
 // on the names.
 //
 //   slug      — host, www-stripped, a common public suffix removed, dots → "-":

…nes/create-extractor/extractor-naming.js …sks/create-extractor/extractor-naming.jsdev/routines/create-extractor/extractor-naming.js renamed to .claudinite/local/packs/gcec/tasks/create-extractor/extractor-naming.js dev/routines/create-extractor/extractor-naming.js renamed to .claudinite/local/packs/gcec/tasks/create-extractor/extractor-naming.js

@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+#
+# Postcondition for the create-extractor task's AGENT stage (task.md §4): validate
+# the agent's work before the draft PR is marked ready for review.
+#   Exit 0   → the extraction is valid; mark the PR ready (task.md §5).
+#   Non-zero → do NOT mark it ready. The agent bails: close the PR and hand the
+#              issue to a human, quoting the printed reason. Reverts nothing — a
+#              scope violation FAILS the run (the human PR review is the backstop,
+#              per the pipeline's containment model), rather than silently
+#              rewriting the agent's work.
+#
+# Takes NO env. Everything it needs is derivable from the branch preprocessing
+# pushed: the case name from the branch, the source from the recorded page's own
+# host. (It used to read five env vars the agent threaded through from a triage
+# JSON blob — that hand-off is gone now that preprocessing owns the names and
+# commits them, and derived-here can't drift from what was actually built.)
+
+set -uo pipefail
+cd "$(git rev-parse --show-toplevel)"
+TASK_DIR=".claudinite/local/packs/gcec/tasks/create-extractor"
+
+fail() { echo "POSTCONDITION FAILED — do not mark the PR ready, hand the issue to a human: $*" >&2; exit 1; }
+
+BRANCH="$(git rev-parse --abbrev-ref HEAD)"
+case "$BRANCH" in
+  claude/extractor/*) ;;
+  *) fail "not on a claude/extractor/* branch (on '$BRANCH') — work on the draft PR's head branch" ;;
+esac
+CASE_NAME="${BRANCH#claude/extractor/}"
+CASE_FILE="dev/requirements/extractor/expected/$CASE_NAME.json"
+URL_FILE="dev/requirements/extractor/data/server-fetched/$CASE_NAME.url"
+[ -f "$CASE_FILE" ] || fail "no case at $CASE_FILE — was this branch scaffolded by the task's preprocessing?"
+[ -f "$URL_FILE" ]  || fail "no recorded URL at $URL_FILE — was this branch scaffolded by the task's preprocessing?"
+
+# The source that owns this case's host, asked of the sources' real matches() — the
+# same authority triage used, so this can't disagree with what was scaffolded.
+SRC_BASE="$(node "$TASK_DIR/resolve-source.js" "$(cat "$URL_FILE")" 2>/dev/null)"
+[ -n "$SRC_BASE" ] || fail "no source matches this case's host — the scaffold's matches() is missing or was edited"
+SRC="extension/event-extractors/custom/$SRC_BASE.js"
+
+# The last commit preprocessing made on this branch (the scaffold, then the page
+# record). `git log` is newest-first, so the first match is the baseline the agent's
+# work sits on top of; fall back to the newest branch-only commit.
+PREP=$(git log --format='%H' --grep='chore: scaffold' --grep='chore: record' origin/main..HEAD 2>/dev/null | head -1)
+[ -n "$PREP" ] || PREP=$(git rev-list origin/main..HEAD 2>/dev/null | head -1)
+[ -n "$PREP" ] || fail "no preprocessing commit found on the branch — was prepare.mjs run?"
+
+# 1. SCOPE — only the source + the case may differ from what preprocessing left.
+#    Anything else the agent touched (a shared helper, the load lists, a re-recorded
+#    page, a new file) fails the run. The recorded page is INSIDE the baseline now
+#    (preprocessing committed it), so it is no longer an allowed change — a
+#    re-recorded page is exactly the kind of drift this should catch.
+changed="$( { git diff --name-only "$PREP"; git ls-files --others --exclude-standard; } \
+  | sort -u | sed '/^$/d' )"
+offenders="$(printf '%s\n' "$changed" | grep -Fvx -e "$SRC" -e "$CASE_FILE" || true)"
+[ -z "$offenders" ] || fail "out-of-scope changes (only $SRC and $CASE_FILE may change):
+$(printf '  %s\n' $offenders)"
+
+# 2. QUALITY FLOOR — a deterministic backstop to the agent's bail judgment. The
+#    case must be a real, presentable event: not 'empty' (nothing extracted) and
+#    not 'degenerate' (an event with no location — the signature of a listing/tour
+#    page rather than one specific event).
+VERDICT=$(CASE_FILE="$CASE_FILE" node "$TASK_DIR/case-quality.js")
+[ "$VERDICT" = "ok" ] || fail "quality floor: the case is '$VERDICT' — no presentable single event to ship"
+
+# 3. RE-VERIFY — never trust the agent; the whole suite must be green.
+npm run test:live    || fail "test:live is not green"
+npm run test:offline || fail "test:offline is not green"
+
+echo "All postconditions passed — mark the draft PR ready for review (task.md §5)."
+exit 0