The repository is intentionally lean. main.py holds the executable entry point and is the right place to wire new pipelines or CLIs before factoring them into dedicated modules. Shared logic belongs under src/painatlas/; keep modules focused on one responsibility so downstream data-mining tasks stay composable. Project metadata lives in pyproject.toml, migrations sit inside migrations/ alongside alembic.ini, and high-level context resides in README.md. When you introduce datasets or notebooks, store them in data/ and notebooks/ folders to keep the root clean.
Create a virtual environment and install the package in editable mode before contributing:
uv venv .venv
source .venv/bin/activate
uv pip install -e .Run database migrations with uv run python -m alembic upgrade head before feature work (the CLI auto-runs this, but explicit calls keep local dev predictable). Always execute Python entry points through uv run, e.g., uv run python main.py ingest ... to ensure the pinned environment is used. Once tests exist, run them via uv run pytest; pytest -k <pattern> targets a subset when iterating.
Secrets belong in a root-level .env file loaded by python-dotenv; do not commit this file.
Follow PEP 8 with 4-space indentation and descriptive, lowercase module names (e.g., scrapers/reddit_client.py). Use snake_case for functions and variables, PascalCase for classes, and reserve all-caps for constants. Prefer type hints on all public functions to document expected payloads and responses. Keep imports sorted by standard library, third-party, and local packages to simplify diffs.
Store tests in a mirrored tests/ hierarchy, e.g., tests/test_reddit_client.py. Each test module should cover a single production module and name cases with test_<behavior>. Favor fixtures for API credentials or sample payloads instead of hard-coded values. Aim for meaningful branch coverage on parsing and classification logic, especially around error handling paths. When adding dependencies, list any test-only requirements under a [project.optional-dependencies.test] section in pyproject.toml.
Write commit subjects in the imperative mood (Add Reddit crawler) with bodies explaining rationale and trade-offs. Group related changes in a single commit to ease reviews. Pull requests must include a summary of changes, steps for reviewers to validate (python main.py, pytest), and link to any tracking issue. Attach screenshots or sample output when altering the CLI experience so reviewers can verify UX quickly.