Skip to content

Commit 2f72aa6

Browse files
andrewboldiclaude
andcommitted
feat: consolidate scraper into monorepo, remove migration scripts
- Remove one-time migrate.py and tag_subjects.py (already used) - Add scraper.py: scrapes AoPS wiki and ingests directly into D1 - Supports --subjects-dir for subject tagging - Supports --year for incremental updates - Supports --local-backup for file output - Add pyproject.toml for scraper Python dependencies - No longer depends on amcProblems, amcSubjects, or amcWorker repos Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 9a29fbb commit 2f72aa6

4 files changed

Lines changed: 313 additions & 337 deletions

File tree

worker/migrate.py

Lines changed: 0 additions & 169 deletions
This file was deleted.

worker/pyproject.toml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
[project]
2+
name = "amc-scraper"
3+
version = "2.0.0"
4+
requires-python = ">=3.10"
5+
dependencies = [
6+
"aiohttp",
7+
"tqdm",
8+
]

0 commit comments

Comments
 (0)