Remove Codecov in favor of native coverage.py

[patrick91]

Summary

This PR removes the Codecov dependency and replaces it with native coverage.py reporting using GitHub Actions artifacts, following the approach outlined in Ditch Codecov for Python Projects.

Changes:

• Delete .codecov.yml configuration file
• Update .coveragerc to include previously Codecov-ignored files in the omit section
• Replace the upload-coverage job with a new coverage job that:
  • Downloads coverage data from all test runs (Linux + Windows)
  • Combines coverage using coverage combine
  • Generates HTML report and markdown summary visible in GitHub Actions step summary
  • Uploads HTML report as artifact for detailed analysis
• Update unit-tests and Windows tests to upload .coverage files instead of coverage.xml
• Remove --cov-report=xml from noxfile.py (no longer needed since we combine raw coverage data)
• Clean up unused pipx install coverage commands

Benefits:

• Eliminates external service dependency
• Coverage reports visible directly in GitHub Actions workflow summary
• HTML report available as downloadable artifact for detailed analysis
• Simpler configuration

Test plan

• Verify CI workflow runs successfully
• Check that coverage report appears in GitHub Actions step summary
• Verify HTML coverage report is available as artifact

Summary by Sourcery

Replace external Codecov integration with native coverage.py reporting aggregated in CI and surfaced via GitHub Actions artifacts and summaries.

Build:

• Remove Codecov-specific configuration and XML coverage generation in favor of raw coverage data collection across environments.
• Introduce a dedicated coverage job that downloads per-job coverage artifacts, combines them, generates HTML and markdown reports, and uploads the HTML report as a CI artifact.

CI:

• Remove Codecov upload steps from Linux and Windows test workflows and switch to uploading .coverage files as artifacts instead.
• Drop unused coverage installation steps from CI jobs that no longer require a separate coverage tool setup.

Tests:

• Simplify pytest invocations in nox sessions by removing XML coverage report generation flags now that coverage is combined from raw data.

[sourcery-ai]

Reviewer's Guide

Replaces external Codecov integration with a native coverage.py workflow in GitHub Actions by aggregating raw .coverage artifacts from all test jobs, generating Markdown and HTML reports in CI, and cleaning up now-unneeded configuration and options.

File-Level Changes

Change	Details	Files
Switch CI coverage reporting from Codecov uploads to native coverage.py aggregation and reporting in GitHub Actions.	Rename unit-test artifacts to upload raw .coverage data instead of coverage.xml. Add a new coverage job that downloads all coverage artifacts, combines them with coverage combine, generates HTML and Markdown reports, and uploads the HTML report as an artifact. Remove the old upload-coverage job and all uses of the codecov/codecov-action, including Windows-specific Codecov steps. Ensure hidden .coverage files are included in uploaded artifacts and ignored if missing, so coverage aggregation is resilient to partial failures.	.github/workflows/test.yml
Simplify local/CI pytest coverage configuration now that XML reports for Codecov are no longer needed.	Drop the --cov-report=xml flag from COMMON_PYTEST_OPTIONS so tests only produce raw coverage data. Remove redundant --cov-report=xml occurrences in specialized nox sessions that previously generated XML for Codecov.	noxfile.py
Align coverage configuration and remove obsolete Codecov config file.	Update .coveragerc (content not shown in diff) to omit the same files that were previously excluded via Codecov configuration, keeping effective coverage behavior consistent. Delete .codecov.yml since Codecov is no longer used.	.coveragerc .codecov.yml
Remove unused coverage-related setup steps from CI jobs that no longer need global coverage installs.	Drop pipx install coverage from benchmark-related CI jobs, relying on the new coverage aggregation step to manage its own coverage dependency.	.github/workflows/test.yml

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've left some high level feedback:

• The new artifact upload/download flow only uploads a single .coverage file per job and uses merge-multiple: true, so identically named .coverage files from different jobs will overwrite each other; consider either enabling coverage parallel mode (so you get distinct .coverage.* files) and uploading them all, or varying the COVERAGE_FILE/artifact paths per job so coverage combine can truly merge data from all test runs.
• Since the coverage job is marked if: always() and depends on both Linux and Windows test jobs, you may want to guard the combine step with a check that coverage artifacts were actually downloaded (or adjust the if condition) to avoid confusing failures when an upstream job is cancelled or skipped.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The new artifact upload/download flow only uploads a single `.coverage` file per job and uses `merge-multiple: true`, so identically named `.coverage` files from different jobs will overwrite each other; consider either enabling coverage parallel mode (so you get distinct `.coverage.*` files) and uploading them all, or varying the `COVERAGE_FILE`/artifact paths per job so `coverage combine` can truly merge data from all test runs.
- Since the `coverage` job is marked `if: always()` and depends on both Linux and Windows test jobs, you may want to guard the combine step with a check that coverage artifacts were actually downloaded (or adjust the `if` condition) to avoid confusing failures when an upstream job is cancelled or skipped.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[greptile-apps]

Greptile Summary

This PR successfully migrates from Codecov to native coverage.py by removing the external service dependency and implementing coverage combining directly in GitHub Actions.

Key Changes:

• Deleted .codecov.yml and migrated its ignore patterns to .coveragerc omit section
• Replaced upload-coverage job with new coverage job that downloads all .coverage files, combines them using coverage combine, and generates both markdown summary and HTML report
• Updated test jobs to upload raw .coverage files instead of XML reports
• Removed --cov-report=xml flags from all pytest commands in noxfile.py
• Removed unnecessary pipx install coverage commands

Issue Found:

• Critical path mismatch: .coveragerc configures HTML output to coverage_html_report directory but workflow attempts to upload htmlcov (coverage.py default), which will cause artifact upload to fail with no files found

Confidence Score: 3/5

• This PR has a critical configuration bug that will prevent HTML coverage reports from being uploaded
• The migration approach is sound and the changes are well-structured, but there's a directory path mismatch between .coveragerc (specifies coverage_html_report) and the workflow (tries to upload htmlcov). This will cause the HTML report artifact upload step to fail. Once fixed, the PR should work correctly.
• .github/workflows/test.yml needs the HTML directory path corrected to match .coveragerc configuration

Important Files Changed

Filename	Overview
.codecov.yml	File completely removed - no longer needed since switching from Codecov to native coverage.py
.coveragerc	Migrated ignore patterns from .codecov.yml to omit section - properly maintains same coverage exclusions
.github/workflows/test.yml	Replaced Codecov upload with native coverage.py combining and reporting - uploads .coverage files instead of XML, combines coverage from all test runs including Windows
noxfile.py	Removed --cov-report=xml flags from all pytest invocations - no longer needed since coverage is combined from raw .coverage files

Sequence Diagram

sequenceDiagram
    participant GH as GitHub Actions
    participant UT as unit-tests (Linux)
    participant WT as unit-tests-on-windows
    participant COV as coverage job
    participant ART as Artifacts

    Note over GH,ART: Old Flow (Codecov)
    UT->>UT: Run tests with --cov-report=xml
    UT->>ART: Upload coverage.xml
    WT->>WT: Run tests with --cov-report=xml
    WT->>WT: Generate coverage.xml
    WT->>Codecov: Upload to Codecov service
    UT->>Codecov: Upload to Codecov service

    Note over GH,ART: New Flow (Native coverage.py)
    UT->>UT: Run tests (no XML generation)
    UT->>ART: Upload .coverage file (hidden)
    WT->>WT: Run tests (no XML generation)
    WT->>ART: Upload .coverage file (hidden)
    COV->>ART: Download all coverage-data-* artifacts
    COV->>COV: coverage combine (merge all .coverage files)
    COV->>COV: coverage html (generate HTML report)
    COV->>COV: coverage report --format=markdown
    COV->>GH: Append to GITHUB_STEP_SUMMARY
    COV->>ART: Upload html-report artifact

Loading

[greptile-apps]

_{4 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

logic: HTML directory mismatch - .coveragerc specifies coverage_html_report but this uses htmlcov (coverage.py default)

Suggested change

	python -Im coverage html --skip-covered --skip-empty
	python -Im coverage html --skip-covered --skip-empty

Then update line 105 to match:

path: coverage_html_report

Prompt To Fix With AI

This is a comment left during a code review.
Path: .github/workflows/test.yml
Line: 98:98

Comment:
**logic:** HTML directory mismatch - `.coveragerc` specifies `coverage_html_report` but this uses `htmlcov` (coverage.py default)

```suggestion
          python -Im coverage html --skip-covered --skip-empty
```

Then update line 105 to match:
```
path: coverage_html_report
```

How can I resolve this? If you propose a fix, please make it concise.

[bellini666]

🍓🍓🍓

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing remove-codecov (d3cd16c) with main (96a06ba)}