Automation: Direction

[github-actions]

Last generated: 2026-01-22T18:28:25.543Z
Provider: openai
Model: gpt-5.2

Summary

Stabilize and simplify CI/automation for scrapy-rotating-proxies by (1) adding a minimal, reliable test + lint pipeline, (2) reducing workflow sprawl to a small set of repo-relevant automations, and (3) tightening packaging/release hygiene. This reduces maintenance toil and makes PR signal clearer.

Direction (what and why)

What

1. Make CI deterministic and repo-specific: run tox across supported Python versions + Scrapy versions, upload coverage, and fail fast on style/typing regressions.
2. Trim/contain automation noise: many .github/workflows/auto-*.yml look like org-wide templates and create ongoing churn. Keep only what’s needed for this library (CI, security scan, labeler if desired).
3. Prevent accidental large/binary artifacts: bfg-1.15.0.jar is committed at repo root; this is a long-term liability and can slow clones and CI.

Why

• Current automation surface area is large and likely unrelated to the library, increasing failure modes and review overhead.
• A tight CI loop (tox-driven) is the most valuable automation for a small Python library.
• Removing/isolating large artifacts and standardizing packaging checks improves reliability for releases and downstream users.

Plan (next 1–3 steps)

1) Add a single, canonical CI workflow that runs tox (and make tox authoritative)

Files

• Add: .github/workflows/ci.yml
• Ensure: tox.ini is the source of truth for envs

Concrete steps

• Create .github/workflows/ci.yml with:
  • Triggers: push, pull_request
  • Matrix: Python 3.9, 3.10, 3.11, 3.12 (adjust if project supports older)
  • Steps:
    • actions/checkout@v4
    • actions/setup-python@v5
    • python -m pip install -U pip tox
    • tox -q
  • Add caching for pip/tox (actions/cache) keyed by tox.ini + setup.cfg/setup.py.

Tox improvements (if needed)

• In tox.ini, define envs explicitly, e.g.:
  • py{39,310,311,312}
  • optional factor envs for Scrapy pins (example: scrapy{24,25,26}) if compatibility is important.
• Add a lint env:
  • Use ruff (fast, low config) and optionally mypy if typing exists.
  • Example deps: ruff, pytest, pytest-cov.

Outcome

• One CI workflow provides clear pass/fail signals; contributors run the same via tox.

2) Reduce workflow sprawl by isolating or removing non-repo-specific automation

Files

• Audit: .github/workflows/*.yml

Concrete steps

• Keep (recommended):
  • ci.yml (new)
  • auto-sec-scan.yml (only if it’s actually running CodeQL/dependency review; otherwise replace with GitHub-native)
• Disable or remove org-template workflows that don’t add value to this repo (e.g. “auto-copilot-*”, “trigger-all-repos”, “autonomous-progress”, etc.).

Safe approach

• First pass: disable rather than delete by adding:
  • on: [] at top of each irrelevant workflow (quick revert possible)
• Second pass (after 1–2 weeks): delete the disabled files.

Outcome

• Fewer unexpected bot actions; less CI noise; lower maintenance.

3) Packaging hygiene + artifact control

Files

• .gitignore
• optionally add .gitattributes
• repo root: bfg-1.15.0.jar

Concrete steps

• Decide whether bfg-1.15.0.jar is needed:
  • If not needed: remove it and add to .gitignore.
  • If needed for maintainer-only workflows: move to a tools/ directory and document in AGENTS.md (or create tools/README.md) with a download link instead of vendoring.
• Add a packaging check job (in ci.yml or separate release-check.yml):
  • python -m pip install -U build twine
  • python -m build
  • python -m twine check dist/*

Outcome

• Smaller repo, faster CI, fewer “it builds locally but not on PyPI” issues.

Risks/unknowns

• Supported Python/Scrapy versions aren’t specified in the prompt. Updating CI matrices may surface failures that require small code fixes. Mitigation: start with the currently working version(s) from tox (or align with setup.py classifiers if present).
• Workflow removals might break org processes (labeling, auto-assign). Mitigation: disable first (on: []), observe for a week, then delete.
• Removing bfg-1.15.0.jar may affect a maintainer workflow. Mitigation: confirm via repo search (git grep bfg-1.15.0.jar) and check docs before removal.

Suggested tests

1. Local
   • tox -q
   • tox -q -e lint (if added)
   • python -m build && twine check dist/*
2. CI verification checklist
   • New ci.yml runs on PRs and pushes
   • Matrix completes successfully (or failures are actionable/real)
   • Coverage upload works (if configured) or is intentionally omitted
   • Disabled workflows no longer trigger
   • Repo clone size/time improves if large artifacts removed
   • pip install . and pip install scrapy-rotating-proxies (from built dist) both work in a clean venv