Migrate from bash to modern Python with UV

.github/workflows/ci.yml

@@ -0,0 +1,39 @@
+name: CI
+
+on:
+  push:
+    branches:
+      - '**'
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  lint-and-test:
+    runs-on: ubuntu-latest
+    steps:
+    - name: Check out this repo
+      uses: actions/checkout@v4
+
+    - name: Install uv
+      uses: astral-sh/setup-uv@v5
+      with:
+        enable-cache: true
+
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: '3.11'
+
+    - name: Install dependencies
+      run: uv sync --all-extras
+
+    - name: Run linting
+      run: |
+        uv run ruff check .
+        uv run ruff format --check .
+
+    - name: Run type checking
+      run: uv run mypy --package scraper
+
+    - name: Run tests
+      run: uv run pytest --cov-report=term-missing

.github/workflows/scrape.yml

@@ -7,22 +7,62 @@ on:
     - cron: '0 0 * * *' # Run daily at midnight UTC
 
 jobs:
-  scheduled:
+  scrape:
     runs-on: ubuntu-latest
     permissions:
       contents: write
     steps:
     - name: Check out this repo
-      uses: actions/checkout@v3
+      uses: actions/checkout@v4
+
+    - name: Install uv
+      uses: astral-sh/setup-uv@v5
+      with:
+        enable-cache: true
+
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: '3.11'
+
+    - name: Install dependencies
+      run: uv sync
+
     - name: Fetch latest data
-      run: |-
-          chmod +x scrape.sh
-          ./scrape.sh
-    - name: Commit and push if it changed
+      run: uv run python -m scraper.main
+
+    - name: Show changes (dry-run)
+      run: |
+        echo "=== Git Status ==="
+        git status
+        echo ""
+        echo "=== Changes to be committed ==="
+        git add -A
+        git diff --cached --stat
+        echo ""
+        if [ -n "$(git diff --cached)" ]; then
+          echo "=== Detailed diff ==="
+          git diff --cached
+        else
+          echo "No changes detected."
+        fi
+
+    - name: Commit and push changes (master only)
+      if: github.ref == 'refs/heads/master'
       run: |-
         git config user.name "Automated"
         git config user.email "actions@users.noreply.github.com"
         git add -A
         timestamp=$(date -u)
         git commit -m "Latest data: ${timestamp}" || exit 0
         git push
+
+    - name: Dry-run notice (non-master branches)
+      if: github.ref != 'refs/heads/master'
+      run: |
+        echo "=================================================="
+        echo "DRY RUN MODE - Not on master branch"
+        echo "Branch: ${{ github.ref }}"
+        echo "Changes were NOT committed or pushed."
+        echo "See the diff above to preview what would change."
+        echo "=================================================="

.gitignore

@@ -0,0 +1,52 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# Virtual environments
+.venv/
+venv/
+ENV/
+env/
+
+# IDEs
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+
+# Testing and coverage
+.pytest_cache/
+.coverage
+htmlcov/
+.tox/
+
+# Type checking
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Linting
+.ruff_cache/
+
+# UV
+.uv/
+uv.lock

README.md

@@ -10,7 +10,69 @@ You can use standard Git commands like `git log factions_details/` or `git diff
 
 ## Implementation and Data
 
-The data represents the composition and details of various factions within the Swiss Parliament. It is scraped daily from the official parliamentary web service API.
+The data represents the composition and details of various factions within the Swiss Parliament. It is scraped daily from the [OpenParlData API](https://api.openparldata.ch).
 
-A GitHub Actions workflow (`.github/workflows/scrape.yml`) runs a script (`scrape.sh`) daily to fetch the latest faction list and details. The script saves the data for each faction into a separate JSON file within the `factions_details/` directory (e.g., `faction_1.json`). Timestamps are removed from the data before saving to minimize commit noise. Changes are automatically committed, allowing the Git history to track the evolution of faction compositions over time.
+A GitHub Actions workflow (`.github/workflows/scrape.yml`) runs a Python scraper daily to fetch the latest faction list and details. The scraper saves the data for each faction into a separate JSON file within the `factions_details/` directory (e.g., `faction_1.json`). Changes are automatically committed, allowing the Git history to track the evolution of faction compositions over time.
+
+## Development
+
+This project uses modern Python tooling:
+
+- **[UV](https://github.com/astral-sh/uv)** for fast dependency management
+- **[Ruff](https://github.com/astral-sh/ruff)** for linting and formatting
+- **[mypy](https://mypy-lang.org/)** for static type checking
+- **[pytest](https://pytest.org/)** for testing
+
+### Setup
+
+Install dependencies with UV:
+
+```bash
+uv sync
+```
+
+### Running the Scraper
+
+Run the scraper locally:
+
+```bash
+uv run python -m scraper.main
+```
+
+### Development Commands
+
+Run linting:
+
+```bash
+uv run ruff check .
+uv run ruff format .
+```
+
+Run type checking:
+
+```bash
+uv run mypy --package scraper
+```
+
+Run tests:
+
+```bash
+uv run pytest
+```
+
+### Project Structure
+
+```
+.
+├── src/
+│   └── scraper/
+│       ├── __init__.py
+│       ├── main.py          # Main scraping logic
+│       ├── api_client.py    # API client for OpenParlData
+│       ├── models.py        # Pydantic data models
+│       └── mappings.py      # Mapping functions (canton codes, etc.)
+├── tests/                   # Test suite
+├── factions_details/        # Output directory for faction JSON files
+└── pyproject.toml          # Project configuration
+```