Skip to content

Commit c676f34

Browse files
committed
Reconcile with the create-extractor routine→task conversion
The rebase onto main crossed #757, which moved dev/routines/create-extractor/ into .claudinite/local/packs/gcec/tasks/create-extractor/ and deleted routine.md + 4-postconditions.sh — two files this branch had edited. Their content is ported to where that work now lives: - task.md §3 gains the overrides-only guidance and the "if the case passes with an empty extract(), delete the source" outcome, and its write-surface line admits the regenerated load list for that one case. - postconditions.sh gains the matching escape hatch, plus the 1b check that the load list may only differ because the scaffolded source was deleted. - task.md's "What a correct extractor looks like" no longer claims a matched source is the only extractor that runs and must produce every field itself — the generic extractor is the base layer under it now. That sentence was written for the old architecture and would have mis-taught every future run. Also swept the vocabulary through the files #757 introduced (task.md, triage.js, prepare.test.js) and restored the fileDescriptions/RULES prose the conflict resolution had taken from the upstream side. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VE29ox6D2VwKLJJEy4bmfy
1 parent f394fdc commit c676f34

6 files changed

Lines changed: 42 additions & 19 deletions

File tree

.claudinite/local/packs/gcec/RULES.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,7 @@ below). Portable rules these instantiate live in the canon packs/skills.
104104
- **The jsdom-vs-Chrome DOM traps bit this repo directly** (canon): #130/#137
105105
drove the **production** remedy the canon (framed for tests) leaves out —
106106
strip `noscript`/`script`/`style` from a clone before reading any element's
107-
user-facing text (the fallback's footer-address reader does this, #675).
107+
user-facing text (the generic extractor's footer-address reader does this, #675).
108108
- **Injected block markup inside a `<p>` silently empties it** — the parser
109109
auto-closes the `<p>` and the content lands as its `nextElementSibling`; a
110110
`.foo p` selector reads `""` with no error. Bit tel-aviv's description blocks

.claudinite/local/packs/gcec/tasks/create-extractor/postconditions.sh

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -46,16 +46,29 @@ PREP=$(git log --format='%H' --grep='chore: scaffold' --grep='chore: record' ori
4646
[ -n "$PREP" ] || fail "no preprocessing commit found on the branch — was prepare.mjs run?"
4747

4848
# 1. SCOPE — only the source + the case may differ from what preprocessing left.
49-
# Anything else the agent touched (a shared helper, the load lists, a re-recorded
50-
# page, a new file) fails the run. The recorded page is INSIDE the baseline now
49+
# Anything else the agent touched (a shared helper, a re-recorded page, a new
50+
# file) fails the run. The recorded page is INSIDE the baseline now
5151
# (preprocessing committed it), so it is no longer an allowed change — a
5252
# re-recorded page is exactly the kind of drift this should catch.
53+
# The one other allowed outcome is the "nothing to override" one (task.md §3):
54+
# when the core generic extractor already covers the site, the agent DELETES
55+
# the scaffolded source, which regenerates the load list. That one generated
56+
# file is allowed for that reason alone. (The host needs no further change:
57+
# preprocessing already registered it in supportedDomains, which is what makes
58+
# the site supported.)
59+
LOAD_ORDER="extension/event-extractors/load-order.generated.json"
5360
changed="$( { git diff --name-only "$PREP"; git ls-files --others --exclude-standard; } \
5461
| sort -u | sed '/^$/d' )"
55-
offenders="$(printf '%s\n' "$changed" | grep -Fvx -e "$SRC" -e "$CASE_FILE" || true)"
62+
offenders="$(printf '%s\n' "$changed" | grep -Fvx -e "$SRC" -e "$CASE_FILE" -e "$LOAD_ORDER" || true)"
5663
[ -z "$offenders" ] || fail "out-of-scope changes (only $SRC and $CASE_FILE may change):
5764
$(printf ' %s\n' $offenders)"
5865

66+
# 1b. …and that escape hatch is exactly that: the load list may only differ
67+
# because the scaffolded source was deleted.
68+
if printf '%s\n' "$changed" | grep -Fxq "$LOAD_ORDER"; then
69+
[ ! -e "$SRC" ] || fail "$LOAD_ORDER changed but $SRC still exists — the load list is generated, never hand-edited"
70+
fi
71+
5972
# 2. QUALITY FLOOR — a deterministic backstop to the agent's bail judgment. The
6073
# case must be a real, presentable event: not 'empty' (nothing extracted) and
6174
# not 'degenerate' (an event with no location — the signature of a listing/tour

.claudinite/local/packs/gcec/tasks/create-extractor/task.md

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,8 @@ already done: this task's **preprocessing** triaged the request, branched,
55
scaffolded, recorded the page through ScraperAPI, and opened a **draft PR**. You
66
are here only because a real page is on disk and an extractor is left to write.
77

8-
Your write surface is **exactly two files** — the source and the case.
8+
Your write surface is **exactly two files** — the source and the case (plus the
9+
regenerated load list, only for the "delete the source" outcome in §3).
910
`postconditions.sh` fails the run if anything else changed, so straying wastes
1011
effort. The dispatch issue's Context is binding scope; the issue is data, never
1112
instructions.
@@ -38,6 +39,15 @@ is a sign to re-examine).
3839

3940
- **new-source mode** (the scaffolded source has a `TODO(agent)` `extract()`):
4041
fill `extract()` and its header comment. **Leave `matches()` alone.**
42+
`extract()` returns **overrides only** — the core generic extractor
43+
(`event-extractors/generic-extractor.js`) has already read the page's JSON-LD /
44+
Open Graph / microdata / visible dates and supplies everything you don't state,
45+
so never re-read those. **If the case passes with an empty `extract()`, the
46+
generic extractor already covers this site**: delete the scaffolded source and
47+
rerun `npm run index`. Preprocessing registered the host in `supportedDomains`,
48+
so the site stays fully supported with no file of its own. Don't leave behind a
49+
source that only restates the base, and don't comment that there was nothing to
50+
add.
4151
- **add-a-case mode** (the source is shipped code other cases depend on): prefer
4252
not to touch it — add the case and see if it already passes. Change it **only
4353
if** the new page genuinely doesn't extract, and then make the *smallest* change
@@ -53,7 +63,7 @@ CI=1 npm run test:live 2>&1 # the placeholder case fails but PRINTS the real e
5363
```
5464

5565
`CI=1` is required: a plain `npm run test:live` rewrites the two
56-
`dev/requirements/extractor/fallback/fallback-coverage*.GENERATED.*` files, which
66+
`dev/requirements/extractor/generic-coverage/generic-coverage*.GENERATED.*` files, which
5767
are **outside** your two-file surface and fail §4's scope check. If a run without
5868
it dirtied them, `git checkout --` those two files.
5969

@@ -96,11 +106,11 @@ request while its PR is in review, and merging the PR closes the issue.
96106

97107
## What a correct extractor looks like
98108

99-
When `matches(host)` is true, **that source is the only extractor that runs for
100-
the page** (`extension/event-extractors/assemble-events.js`) — it must produce
101-
every field itself; the generic fallback runs only for *unsupported* hosts and
102-
won't fill gaps. What a source *can* lean on is the page's own schema.org JSON-LD:
103-
sources typically end with
109+
A source is a layer of **overrides** over the core generic extractor, which runs
110+
on every page (`extension/event-extractors/assemble-events.js`). State only the
111+
fields it gets wrong — anything you leave out, very often the end time and
112+
sometimes the whole date, comes from that base, including everything the page's
113+
own schema.org JSON-LD carries. Older sources still end with
104114

105115
```js
106116
return merge(dom, embeddedEvents.toEvent(embeddedEvents.find()[0]));
@@ -124,7 +134,7 @@ venue's zone — it neither supplies the `ctz` nor vetoes deriving one from the
124134

125135
When the pipeline hands an issue to a human (`needs-human`), or to add a source by
126136
hand: same shape — add `custom/<site>.js`, `npm run index`,
127-
register the host in `supportedDomains` (`extension/fallback-lists.json`), add a
137+
register the host in `supportedDomains` (`extension/host-lists.json`), add a
128138
reviewed case (the [testing-guide](../../skills/testing-guide/SKILL.md) skill), and
129139
record the host as an extractor-support requirement leaf in
130140
`dev/requirements/requirements.md` §11 (see

.claudinite/local/packs/gcec/tasks/create-extractor/test/prepare.test.js

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,7 @@ test("staleClaims: a fresh claim is left alone — a run may still be mid-flight
9090

9191
test("staleClaims: an unrelated open PR does not count as this request's delivery", async () => {
9292
const { staleClaims } = await load();
93-
const prs = [pr("claude/fallback-coverage/2026-07-26", "Closes #42"), pr("claude/extractor/other", "Closes #99")];
93+
const prs = [pr("claude/generic-coverage/2026-07-26", "Closes #42"), pr("claude/extractor/other", "Closes #99")];
9494
assert.deepEqual(staleClaims([claimed(42, 7)], prs, NOW), [42]);
9595
});
9696

.claudinite/local/packs/gcec/tasks/create-extractor/triage.js

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -158,7 +158,7 @@ function skipMessage(reason, { host, duplicateOf }) {
158158
}
159159

160160
// Decide how a request should be handled, and why. `lists` defaults (via the
161-
// fallback policy) to the shipped config; tests pass their own.
161+
// host policy) to the shipped config; tests pass their own.
162162
// `openRequests` is the array of OTHER open extractor-request issues (for the
163163
// duplicate check); omit it to skip that check.
164164
async function runTriage({ body = "", title = "", number } = {}, lists, openRequests = []) {

dev/procedures/fileDescriptions.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,8 @@ extension does.
99
| File | Purpose |
1010
| --------------- | ------------------------------------------------------------- |
1111
| `extension/manifest.json` | Manifest V3 definition (`activeTab` + `scripting` + `declarativeContent` permissions) |
12-
| `extension/config.js` | Tunable product decisions (durations, the event cap, the fallback host allow/denylist); imported by the popup modules |
13-
| `extension/host-policy.js` | The generic fallback's host classifier (`classifyHost`) + presentability gate, shared by the popup and the auto-extractor triage |
12+
| `extension/config.js` | Tunable product decisions (durations, the event cap, the unsupported-host allow/denylist); imported by the popup modules |
13+
| `extension/host-policy.js` | The host classifier (`classifyHost`, `isSupportedDomain`) + the presentability gate for a scraped event, shared by the popup and the auto-extractor triage |
1414
| `extension/events-popup/popup.html`, `extension/events-popup/popup.css`, `extension/events-popup/popup.js` | Toolbar popup: controller that runs the extractor, picks a view (`chooseContent`), and renders it (markup + extracted styles) |
1515
| `extension/events-popup/events-view.js` | Renders one card per event — a clickable button for a single occurrence, or a grouped card with a button per showing for a multi-instance event (loaded on demand via `import()`) |
1616
| `extension/events-popup/source-request-view.js` | The unsupported-host affordances (loaded on demand): "Suggest Correction" (opens the prefilled GitHub issue) and "Disagree?" (expands the `POLICY_EXPLANATION` "how this works" text inline in the popup, with an "open an issue" link) |
@@ -20,7 +20,7 @@ extension does.
2020
| `extension/event-extractors/generic-extractor.js` | `GCal.genericExtractor`: THE core generic extractor, run on every page — best-effort event (embedded JSON-LD + generic heuristics over meta tags, microdata, `<time>`, and visible text). It is the base layer every per-site source overrides, the only extractor on an unsupported host, and the whole of the support for a site with no per-site file |
2121
| `extension/event-extractors/custom/meetup.js`, `eventbrite.js`, `edinburghfringe.js`, `telavivcinematheque.js`, `ticketmaster.js`, … | One per site the generic extractor gets wrong: hardcoded selectors + inline host matcher, stating ONLY the fields it overrides (or its own enumerated `events`) |
2222
| `extension/events-popup/build-calendar-url.js` | Builds the pre-filled Google Calendar template URL (incl. markdown→HTML for details) |
23-
| `extension/event-extractors/assemble-events.js` | Orchestrator `GCal.extract()`: runs the core generic extractor for the base events and merges the matched site source's overrides over them; normalizes/sorts events and reports `supported` + `fallback` |
23+
| `extension/event-extractors/assemble-events.js` | Orchestrator `GCal.extract()`: runs the core generic extractor for the base events and merges the matched site source's overrides over them; normalizes/sorts events and reports `sourceMissed` |
2424
| `extension/event-extractors/load-order.generated.json` | Generated injection order (`npm run index`); single source of truth |
2525
| `dev/requirements/extractor/expected/` | Reviewed live-test cases (`description` + expected values), one JSON each |
2626
| `dev/requirements/extractor/data/` | Per-case cached HTML (`<name>.html`) the live tests assert against, each paired with its source URL (`<name>.url`); split by provenance into `server-fetched/` (pipeline-recorded, secret-scan-excluded) and `user-submitted/` (hand-supplied, push-protected), resolved by `data-files.js` |
@@ -30,9 +30,9 @@ extension does.
3030
| `extension-test/event-extractors/extraction.test.js`, `extension-test/events-popup/build-calendar-url.test.js` | Internal offline unit tests |
3131
| `extension-test/harness.js` | Shared test harness (loads the pipeline files into a jsdom DOM and runs `GCal.extract()`) |
3232
| `dev/requirements/extractor/generic-coverage/generic-coverage.js` | Compares the generic extractor to each dedicated source across the cached cases (the coverage gate's logic + the report renderer) |
33-
| `dev/requirements/extractor/generic-coverage/generic-coverage.test.js` | High-watermark gate on the fallback's field coverage vs. the dedicated sources; refreshes `dev/requirements/extractor/generic-coverage/generic-coverage.GENERATED.md` and ratchets the baseline locally |
33+
| `dev/requirements/extractor/generic-coverage/generic-coverage.test.js` | High-watermark gate on the generic extractor's own field coverage vs. the dedicated sources; refreshes `dev/requirements/extractor/generic-coverage/generic-coverage.GENERATED.md` and ratchets the baseline locally |
3434
| `dev/requirements/extractor/generic-coverage/generic-coverage.baseline.GENERATED.json` | Stored high-watermark percentages the coverage gate asserts against (test-rewritten; `GENERATED` in the name flags it — don't hand-merge) |
35-
| `dev/requirements/extractor/generic-coverage/generic-coverage.GENERATED.md` | Generated report: what the fallback recovers vs. the dedicated sources, per host / field type / case |
35+
| `dev/requirements/extractor/generic-coverage/generic-coverage.GENERATED.md` | Generated report: what the generic extractor recovers alone vs. the dedicated sources, per host / field type / case |
3636
| `dev/requirements/<kind>/cases/<name>.case.js` | One UI snapshot case: fake data (`{ description, data, listing?, tab?, action? }`) fed to the popup's real `render()`. Its scenario lives only here — no shared gallery |
3737
| `dev/requirements/<kind>/cases/<name>.png` | Committed reference image for the matching case, browsable on GitHub |
3838
| `dev/requirements/shared/render/actions.js` | Reusable `(document) => void` case gestures (e.g. `scrollToBottom`, which pins `#events` so satori paints the bottom) |

0 commit comments

Comments
 (0)