Skip to content

feat(ci): Add Codecov integration and native test result summaries#98

Merged
hristoterezov merged 4 commits intomasterfrom
feat/add-codecov-integration
Nov 3, 2025
Merged

feat(ci): Add Codecov integration and native test result summaries#98
hristoterezov merged 4 commits intomasterfrom
feat/add-codecov-integration

Conversation

@hristoterezov
Copy link
Member

@hristoterezov hristoterezov commented Oct 29, 2025

Summary

This PR enhances the CI workflow with automated reporting for better visibility into test results and code coverage:

Codecov integration - Automated coverage reporting with PR comments
Native test summaries - Test results displayed on workflow runs

What This Adds

1. Codecov Integration

Automatic PR Comments:
Every pull request will get a comment from Codecov showing:

  • Overall coverage percentage
  • Coverage change compared to base branch (±X%)
  • File-by-file breakdown of coverage changes
  • Which lines are covered/uncovered

Codecov Dashboard:

  • Coverage trends over time
  • File/folder coverage breakdown
  • Line-by-line visualization
  • Coverage badges for README

Configuration:

  • Uses codecov/codecov-action@v4 (matches other Jitsi repos like jitsi-videobridge)
  • Authenticates with ${{ secrets.CODECOV_TOKEN }} from organization secrets
  • Set to fail_ci_if_error: false to not block CI if Codecov has issues
  • Verbose logging enabled for initial rollout debugging

2. Native Test Result Summaries

GitHub Job Summary:
Displays on the workflow run page showing:

  • ✅/❌ Pass/fail status with visual indicators
  • 📊 Formatted table with:
    • Total tests run
    • Number of failed tests
    • Number of passed tests

Implementation:

  • Uses GitHub's native $GITHUB_STEP_SUMMARY feature
  • Parses Web Test Runner console output
  • No third-party actions required
  • Runs with if: always() to show results even on failure

Example Output

Test Summary (on workflow run page):

### Test Results 📊

✅ All tests passed!

| Metric | Count |
|--------|-------|
| Total Tests | 42 |
| Failed | 0 |
| Passed | 42 |

Backwards Compatibility

  • ✅ Existing artifact upload is preserved for manual inspection
  • ✅ Test failure still causes CI to fail (exit code preserved)
  • ✅ Coverage thresholds still enforced by Web Test Runner

Test Plan

  • CI runs successfully on this PR
  • Test summary appears on workflow run page
  • Codecov uploads coverage (check for CODECOV_TOKEN availability)
  • Codecov PR comment is posted
  • Artifact still available for download

Related

  • Matches the Codecov setup used in other Jitsi repositories (jitsi-videobridge)
  • Addresses the need for visible test results without downloading artifacts

Adds Codecov integration to the CI workflow to provide automated coverage reporting:
- Uploads coverage data to Codecov after test execution
- Uses organization-level CODECOV_TOKEN for authentication
- Maintains artifact upload as fallback for manual inspection
- Configured with fail_ci_if_error: false to not block CI on Codecov issues
- Enables automatic PR comments with coverage statistics and trends

This integration will provide visibility into code coverage directly in pull requests, showing coverage changes and trends over time without requiring manual artifact downloads.
@codecov
Copy link

codecov bot commented Oct 29, 2025

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

@hristoterezov hristoterezov force-pushed the feat/add-codecov-integration branch 2 times, most recently from bcd0c76 to 77ce564 Compare October 29, 2025 18:10
Enhances CI workflow with automated reporting for better visibility:

**Native Test Result Summaries:**
- Adds GitHub job summary showing test pass/fail status
- Displays formatted table with total/passed/failed test counts
- Uses only native GitHub features ($GITHUB_STEP_SUMMARY)
- No third-party actions required for test reporting

**Benefits:**
- Test results summary on workflow run page
- Maintains existing artifact upload for manual inspection
- All reporting uses native or org-approved tools
@hristoterezov hristoterezov force-pushed the feat/add-codecov-integration branch from 77ce564 to 0b8f79a Compare October 29, 2025 18:16
@saghul
Copy link
Member

saghul commented Oct 30, 2025

CXan we put the long-ish scripts on files in the .github directory and call them instead?

This refactoring improves the maintainability and readability of the CI workflow by extracting complex inline scripts into separate, well-documented files. This approach provides better code organization, easier testing, and the ability to reuse these scripts across multiple workflows if needed.
@hristoterezov
Copy link
Member Author

CXan we put the long-ish scripts on files in the .github directory and call them instead?

@saghul Done!

@hristoterezov hristoterezov requested a review from saghul October 30, 2025 16:24
@hristoterezov hristoterezov merged commit c02dc3c into master Nov 3, 2025
6 checks passed
@hristoterezov hristoterezov deleted the feat/add-codecov-integration branch November 3, 2025 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants