Skip to content

feat(ci): Implement test sharding (5x testing speed goal part-2) #3281

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
may-tas wants to merge 5 commits into
base: develop
Choose a base branch
from
Open

feat(ci): Implement test sharding (5x testing speed goal part-2) #3281

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
2d9b3be
feat: enhance CI workflows with sharding and coverage merging
may-tas Jan 25, 2026
832495e
feat: improve coverage report merging and install lcov in CI workflows
may-tas Jan 25, 2026
c69a19d
fix: ensure coverage report merging only occurs on successful Flutter…
may-tas Jan 25, 2026
274fac5
Merge branch 'develop' into improve-testing-speed-part2
may-tas Jan 29, 2026
85b2ac4
ci: increase total shards and improve Codecov integration in workflows
may-tas Jan 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
135 changes: 130 additions & 5 deletions .github/workflows/pull-request.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ env:
# Test concurrency: defaults to nproc, capped at 4 (override if needed)
# Note: Flutter 3.32.x may have --concurrency flag behavior issues
FLUTTER_TEST_CONCURRENCY: 4
TOTAL_SHARDS: 12

jobs:
Flutter-Codebase-Check:
Expand Down Expand Up @@ -273,6 +274,15 @@ jobs:
#Get all changed files
ALL_CHANGED_FILES=$(git diff --name-only --diff-filter=ACMRT $BASE_SHA ${{github.event.pull_request.head.sha}} | tr '\n' ' ')
echo "all_changed_files=${ALL_CHANGED_FILES}" >> $GITHUB_OUTPUT

FILE_COUNT=$(echo "$ALL_CHANGED_FILES" | wc -w)
echo "all_changed_files_count=${FILE_COUNT}" >> $GITHUB_OUTPUT

if [ -n "$ALL_CHANGED_FILES" ]; then
echo "any_changed=true" >> $GITHUB_OUTPUT
else
echo "any_changed=false" >> $GITHUB_OUTPUT
fi
- env:
CHANGED_FILES: ${{ steps.changed_files.outputs.all_changed_files }}
if: steps.changed_files.outputs.any_changed == 'true' || steps.changed_files.outputs.any_deleted == 'true'
Expand Down Expand Up @@ -302,10 +312,14 @@ jobs:
exit 1

Flutter-Testing:
name: Testing codebase
name: Run tests (Shard ${{ matrix.shard }})
runs-on: ubuntu-latest
needs:
[Flutter-Codebase-Check, Python-Compliance, check-unused-dependencies]
strategy:
fail-fast: false
matrix:
shard: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
Comment on lines +319 to +322
Copy link
Contributor

@coderabbitai coderabbitai bot Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Matrix shard count should reference TOTAL_SHARDS to avoid drift.

The matrix explicitly lists [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] while TOTAL_SHARDS is defined separately as 12. If someone updates TOTAL_SHARDS without updating the matrix (or vice versa), tests will be incorrectly distributed.

Consider using fromJson with a generated sequence or adding a validation step to ensure matrix size equals TOTAL_SHARDS.

♻️ Option: Add validation step to catch drift 
      - name: Validate shard configuration
        run: |
          if [ "${{ matrix.shard }}" -ge "${{ env.TOTAL_SHARDS }}" ]; then
            echo "ERROR: Shard index ${{ matrix.shard }} >= TOTAL_SHARDS (${{ env.TOTAL_SHARDS }})"
            exit 1
          fi

Alternatively, document clearly that both must be updated together.

🤖 Prompt for AI Agents 
In @.github/workflows/pull-request.yml around lines 319 - 322, The matrix
hardcodes shard indices while TOTAL_SHARDS is defined separately, risking drift;
update the workflow so the matrix is derived from TOTAL_SHARDS (e.g., generate
the shard list via fromJson/sequence using env.TOTAL_SHARDS) or add a validation
step that compares matrix.shard against env.TOTAL_SHARDS and fails if
matrix.shard >= TOTAL_SHARDS; refer to TOTAL_SHARDS and matrix.shard to locate
where to change the strategy matrix and where to add the validation check.

steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
Expand All @@ -318,30 +332,141 @@ jobs:
channel: "stable" # or: 'beta', 'dev' or 'master'
- name: Running pub get to fetch dependencies
run: flutter pub get
- name: Codebase testing
- name: Run tests (shard ${{ matrix.shard }}/${{ env.TOTAL_SHARDS }})
run: |
CONCURRENCY="${FLUTTER_TEST_CONCURRENCY:-$(nproc)}"
if [ "$CONCURRENCY" -gt 4 ]; then CONCURRENCY=4; fi
flutter test --concurrency="$CONCURRENCY" --coverage
flutter test --concurrency="$CONCURRENCY" \
--shard-index=${{ matrix.shard }} \
--total-shards=${{ env.TOTAL_SHARDS }} \
--coverage
- name: Remove generated files from coverage
if: always()
run: |
dart pub global activate remove_from_coverage
dart pub global run remove_from_coverage:remove_from_coverage -f coverage/lcov.info -r '\.g\.dart$'
- name: Upload coverage artifact
if: always()
uses: actions/upload-artifact@v4
with:
name: coverage-shard-${{ matrix.shard }}
path: ./coverage/
retention-days: 1
Comment on lines 343 to +354
Copy link
Contributor

@coderabbitai coderabbitai bot Jan 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Pin remove_from_coverage to a fixed version for reproducible CI.

Same concern as push.yml: pinning avoids surprise changes from pub.dev.

🤖 Prompt for AI Agents 
In @.github/workflows/pull-request.yml around lines 343 - 354, Pin the
remove_from_coverage package to a fixed pub version in the "Remove generated
files from coverage" workflow step: change the activation command for
remove_from_coverage to include an explicit version (e.g.
remove_from_coverage@<version>) so CI uses a reproducible release, and keep the
subsequent dart pub global run remove_from_coverage:remove_from_coverage
invocation unchanged; ensure the chosen version is used consistently for
activation.

Comment on lines +344 to +354
Copy link
Contributor

@coderabbitai coderabbitai bot Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

if: always() on coverage steps may upload incomplete artifacts.

When a shard's tests fail, the coverage file may be incomplete or missing entirely. Using if: always() uploads potentially corrupt coverage data that could skew merged results.

Consider using if: success() || failure() (excludes cancelled) or checking for file existence before upload.

🛡️ Proposed fix: Conditional upload with file check 
       - name: Upload coverage artifact
-        if: always()
+        if: (success() || failure()) && hashFiles('coverage/lcov.info') != ''
         uses: actions/upload-artifact@v4
         with:
           name: coverage-shard-${{ matrix.shard }}
           path: ./coverage/
           retention-days: 1
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if: always()
run: |
dart pub global activate remove_from_coverage
dart pub global run remove_from_coverage:remove_from_coverage -f coverage/lcov.info -r '\.g\.dart$'
- name: Upload coverage artifact
if: always()
uses: actions/upload-artifact@v4
with:
name: coverage-shard-${{ matrix.shard }}
path: ./coverage/
retention-days: 1
if: always()
run: |
dart pub global activate remove_from_coverage
dart pub global run remove_from_coverage:remove_from_coverage -f coverage/lcov.info -r '\.g\.dart$'
- name: Upload coverage artifact
if: (success() || failure()) && hashFiles('coverage/lcov.info') != ''
uses: actions/upload-artifact@v4
with:
name: coverage-shard-${{ matrix.shard }}
path: ./coverage/
retention-days: 1
🤖 Prompt for AI Agents 
In @.github/workflows/pull-request.yml around lines 344 - 354, The "Upload
coverage artifact" step currently uses if: always() which can upload incomplete
or missing artifacts; update the step that uses actions/upload-artifact@v4
(named coverage-shard-${{ matrix.shard }}) to only run when appropriate by
replacing if: always() with a safer condition such as if: success() || failure()
(to exclude cancelled runs) or add a pre-check that ensures the ./coverage/
directory or coverage/lcov.info exists before uploading; modify the Upload
coverage artifact step's if clause or add a conditional check so the artifact
upload only happens when valid coverage files are present.


Merge-Coverage:
name: Merge Coverage Reports
runs-on: ubuntu-latest
needs: [Flutter-Testing]
if: success()
Copy link
Contributor

@coderabbitai coderabbitai bot Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

# Check the current state of the pull-request.yml file around line 360
head -370 .github/workflows/pull-request.yml | tail -20

Repository: PalisadoesFoundation/talawa

Length of output: 608

🏁 Script executed:

# Also check what condition is used in push.yml for comparison
grep -n "Merge.*[Cc]overage\|if:.*success\|if:.*cancelled" .github/workflows/push.yml

Repository: PalisadoesFoundation/talawa

Length of output: 308

🏁 Script executed:

# Get more context around line 360 in pull-request.yml
sed -n '350,370p' .github/workflows/pull-request.yml

Repository: PalisadoesFoundation/talawa

Length of output: 649

🏁 Script executed:

# Check if there's a Merge-Coverage job definition to understand what should follow
grep -n "Merge-Coverage\|merge.*coverage" .github/workflows/pull-request.yml -i

Repository: PalisadoesFoundation/talawa

Length of output: 355

Merge-Coverage condition still uses if: success(), which skips coverage when any shard fails.

The if: success() condition at line 360 prevents the Merge-Coverage job from running if any shard fails, losing partial coverage from passing shards. This behavior is present in both pull-request.yml and push.yml. If the goal is to report partial coverage even when some shards fail, consider using if: ${{ !cancelled() }} instead.

🤖 Prompt for AI Agents 
In @.github/workflows/pull-request.yml at line 360, The Merge-Coverage job is
gated by the `if: success()` condition which prevents running when any shard
fails; update the job's conditional to `if: ${{ !cancelled() }}` so
Merge-Coverage runs for partial results (replace `if: success()` with `if: ${{
!cancelled() }}` in the Merge-Coverage job definition), and make the same change
for the equivalent job in the other workflow so both `pull-request.yml` and
`push.yml` use `if: ${{ !cancelled() }}` instead of `if: success()`.

steps:
- name: Checkout the Repository
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Fetch base branch for Codecov comparison
run: |
set -e
echo "Fetching base branch: ${{ github.event.pull_request.base.ref }}"
if ! git fetch origin ${{ github.event.pull_request.base.ref }} 2>&1; then
echo "ERROR: Failed to fetch base branch '${{ github.event.pull_request.base.ref }}' from origin"
exit 1
fi
echo "Successfully fetched base branch"

- uses: actions/setup-java@v4
with:
distribution: "zulu"
java-version: "17.0"
- uses: subosito/flutter-action@v2
with:
flutter-version: ${{ env.FLUTTER_VERSION }}
channel: "stable"
Comment on lines +377 to +384
Copy link
Contributor

@coderabbitai coderabbitai bot Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Java and Flutter setup appear unnecessary in Merge-Coverage job.

The Merge-Coverage job only downloads artifacts, merges lcov files, and uploads to Codecov. Java and Flutter are not used in any of these steps.

Removing these setup steps would reduce job execution time by ~30-60 seconds.

♻️ Proposed removal 
-      - uses: actions/setup-java@v4
-        with:
-          distribution: "zulu"
-          java-version: "17.0"
-      - uses: subosito/flutter-action@v2
-        with:
-          flutter-version: ${{ env.FLUTTER_VERSION }}
-          channel: "stable"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- uses: actions/setup-java@v4
with:
distribution: "zulu"
java-version: "17.0"
- uses: subosito/flutter-action@v2
with:
flutter-version: ${{ env.FLUTTER_VERSION }}
channel: "stable"
🤖 Prompt for AI Agents 
In @.github/workflows/pull-request.yml around lines 377 - 384, In the
Merge-Coverage job remove the unnecessary Java and Flutter setup steps (the
actions/setup-java@v4 block and the subosito/flutter-action@v2 block) since the
job only downloads artifacts, merges lcov files, and uploads to Codecov; delete
those "uses:" entries and their "with:" subkeys (and verify no other steps in
the Merge-Coverage job reference java-version, distribution, or
env.FLUTTER_VERSION) so the job runs without installing Java or Flutter and
reduces execution time.

- name: Download all coverage artifacts
id: download-artifacts
continue-on-error: true
uses: actions/download-artifact@v4
with:
pattern: coverage-shard-*
path: ./coverage-shards/
merge-multiple: false
- name: Check if artifacts were downloaded
id: check-artifacts
run: |
if find coverage-shards -name "lcov.info" -type f | grep -q .; then
echo "artifacts_found=true" >> $GITHUB_OUTPUT
echo "Coverage artifacts found"
else
echo "artifacts_found=false" >> $GITHUB_OUTPUT
echo "No coverage artifacts found"
exit 1
fi
- name: Install lcov
if: steps.check-artifacts.outputs.artifacts_found == 'true'
run: sudo apt-get update && sudo apt-get install -y lcov

- name: Merge coverage reports
if: steps.check-artifacts.outputs.artifacts_found == 'true'
run: |
mkdir -p ./coverage
echo "Merging coverage from shards..."
find coverage-shards -name "lcov.info" -type f > lcov-files.txt
if [ ! -s lcov-files.txt ]; then
echo "ERROR: No lcov.info files found"
exit 1
fi
echo "Found lcov files:"
cat lcov-files.txt

# Properly merge using lcov to avoid duplicate entries
FIRST_FILE=true
while IFS= read -r file; do
if [ "$FIRST_FILE" = true ]; then
cp "$file" ./coverage/lcov.info
FIRST_FILE=false
else
lcov --add-tracefile ./coverage/lcov.info \
--add-tracefile "$file" \
--output-file ./coverage/lcov.info.tmp
mv ./coverage/lcov.info.tmp ./coverage/lcov.info
fi
done < lcov-files.txt

echo "Merged coverage file size: $(wc -c < ./coverage/lcov.info) bytes"
SF_COUNT=$(grep -c "^SF:" ./coverage/lcov.info || echo "0")
echo "Number of files in merged coverage: $SF_COUNT"
if [ "$SF_COUNT" -lt 50 ]; then
echo "::warning::Only $SF_COUNT files in coverage (expected 50+)"
fi

- name: Calculate merge base for Codecov
if: steps.check-artifacts.outputs.artifacts_found == 'true'
id: get-merge-base
run: |
MERGE_BASE=$(git merge-base origin/${{ github.event.pull_request.base.ref }} HEAD)
echo "Merge base commit: $MERGE_BASE"
echo "merge_base=$MERGE_BASE" >> $GITHUB_OUTPUT
git show -s --format=%ci $MERGE_BASE

#######################################################################
# DO NOT DELETE ANY references to env.CODECOV_UNIQUE_NAME in this
# section. They are required for accurate calculations
#######################################################################
- name: Present and upload coverage to Codecov as ${{env.CODECOV_UNIQUE_NAME}}
if: steps.check-artifacts.outputs.artifacts_found == 'true'
uses: codecov/codecov-action@v5
with:
name: "${{env.CODECOV_UNIQUE_NAME}}"
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: false
fail_ci_if_error: true
Copy link
Contributor

@coderabbitai coderabbitai bot Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

cat -n .github/workflows/pull-request.yml | sed -n '450,475p'

Repository: PalisadoesFoundation/talawa

Length of output: 1297

🌐 Web query:

Codecov action fail CI error fork pull request CODECOV_TOKEN

💡 Result:

On fork-based pull requests, GitHub Actions does not provide repository secrets to the workflow run, so secrets.CODECOV_TOKEN is empty. If your Codecov setup requires a token, the Codecov upload step will error/fail on fork PRs. [5][2]

Fix options

  1. Public repo (recommended): use tokenless uploads
  • In Codecov, set Token authentication for public repositories = Not required (org setting; admin-only), then don’t pass CODECOV_TOKEN for uploads. Codecov supports tokenless uploads for public repos in certain cases (and newer uploaders/actions are recommended). [1]
  • Also ensure you’re using a modern action (e.g., codecov/codecov-action@v4), which is designed to work better with fork PRs. [1]
  1. Private repo: you must use a token
  • Codecov requires a token for private repos, so fork PR uploads will fail unless you redesign the workflow to run in a trusted context. [1]
  1. If you choose to expose the token to fork PRs (risky)
  • Some projects run coverage upload on pull_request_target (so secrets are available), but you must treat this as security-sensitive (untrusted code + secrets).
  • Codecov notes that the token mainly allows uploading coverage for that repo (not code access), but an attacker could upload misleading coverage if they get it. [6]

Minimal workflow patterns

A. Upload only when token exists (skips forks safely):

- uses: codecov/codecov-action@v4
  if: ${{ secrets.CODECOV_TOKEN != '' }}
  with:
    token: ${{ secrets.CODECOV_TOKEN }}

(Adding the token is the standard approach when you can rely on secrets.) [4][1]

B. Tokenless (public repo + token auth disabled):

- uses: codecov/codecov-action@v4

(Tokenless behavior depends on your Codecov org/repo settings.) [1]

If you paste the exact Codecov error lines from the job log, I can tell which of the above applies (token required vs. GitHub API/build lookup vs. config). [2]

🌐 Web query:

Codecov GitHub Actions fork PR missing token behavior

💡 Result:

In GitHub Actions, pull requests coming from forks do not get access to the upstream repo’s secrets, so secrets.CODECOV_TOKEN will be empty/missing in that context (this is a GitHub security behavior, and Codecov calls it out in their Action docs). As a result, a Codecov upload that requires a token will fail on fork PRs. [2]

What happens next depends on repo visibility / Codecov settings:

  • Public repo, fork PRs (recommended path): tokenless upload can work.
    Codecov supports tokenless uploads for public repos when the upload is treated as coming from an “unprotected” branch (a branch name that contains a : prefix like forkname:branch). Starting with codecov/codecov-action > v4.0, the Action automatically rewrites fork PR branch names to include :, so fork PR uploads “should work smoothly … out of the box” without exposing a token. [1]

  • Private repo: token is always required.
    Codecov’s token docs state: “For private repositories, all uploads require a token.” So fork PR uploads will fail unless you change your workflow strategy (because the secret token won’t be available to the forked PR run). [1]

  • Public repo, but you’ve required tokens for protected branches:
    A token is required for uploads on protected branches unless the org has disabled token auth for public repos. [1]

If you’re trying to make fork PRs work without exposing CODECOV_TOKEN, typical fixes are:

  • Ensure you’re using codecov/codecov-action@v4+ (or newer), and rely on the fork-PR tokenless behavior for public repos. [1]
  • If you’re on codecov/codecov-action@v5, you can also use OIDC (use_oidc: true) instead of upload tokens (requires permissions: id-token: write). [2]
  • For public repos, admins can optionally configure org settings so token auth is “not required” for public uploads. [1]

🏁 Script executed:

git remote -v

Repository: PalisadoesFoundation/talawa

Length of output: 321

🏁 Script executed:

rg -i "fork\|pull_request_target\|external.contributor" .github/workflows/pull-request.yml

Repository: PalisadoesFoundation/talawa

Length of output: 53

🌐 Web query:

PalisadoesFoundation/talawa repository public or private GitHub

💡 Result:

The PalisadoesFoundation/talawa GitHub repository is public. [1]

Consider handling fork PR coverage uploads differently to prevent blocking external contributions.

With fail_ci_if_error: true, the Codecov upload will fail for fork PRs (which lack access to secrets.CODECOV_TOKEN), blocking the PR. For this public repository using codecov/codecov-action@v5, consider:

  • Setting fail_ci_if_error: false to allow the workflow to continue, or
  • Skipping the upload when the token is unavailable (e.g., if: secrets.CODECOV_TOKEN != ''), or
  • Using OIDC authentication (use_oidc: true) which doesn't require secrets exposure.
🤖 Prompt for AI Agents 
In @.github/workflows/pull-request.yml at line 461, The workflow currently sets
fail_ci_if_error: true for the codecov/codecov-action@v5 step which causes fork
PRs (without secrets.CODECOV_TOKEN) to fail CI; update the step to either set
fail_ci_if_error: false, or add a guard to skip the upload when the token is
missing (use an if: condition checking secrets.CODECOV_TOKEN != '' on the upload
step), or enable OIDC by setting use_oidc: true so uploads can run without
exposing a secret; modify the codecov action configuration accordingly (look for
fail_ci_if_error, use_oidc and the codecov/codecov-action@v5 invocation) to
implement one of these fixes.

verbose: true
exclude: "docs/"
gcov_ignore: "docs/"
files: ./coverage/lcov.info
flags: flutter

- name: Test acceptable level of code coverage
if: steps.check-artifacts.outputs.artifacts_found == 'true'
uses: VeryGoodOpenSource/very_good_coverage@v3
with:
path: "./coverage/lcov.info"
Expand All @@ -350,7 +475,7 @@ jobs:
Android-Build:
name: Testing build for android
runs-on: ubuntu-latest
needs: [Flutter-Testing]
needs: [Merge-Coverage]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
Expand Down
123 changes: 116 additions & 7 deletions .github/workflows/push.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ env:
# Test concurrency: defaults to nproc, capped at 4 (override if needed)
# Note: Flutter 3.32.x may have --concurrency flag behavior issues
FLUTTER_TEST_CONCURRENCY: 4
TOTAL_SHARDS: 12

jobs:
Merge-Conflict-Check:
Expand Down Expand Up @@ -92,10 +93,13 @@ jobs:

Flutter-Testing:
if: ${{ github.actor != 'dependabot[bot]' }}
name: Testing codebase
name: Run tests (Shard ${{ matrix.shard }})
runs-on: ubuntu-latest
needs: Flutter-Codebase-Check
# needs: Update-Documentation
strategy:
fail-fast: false
matrix:
shard: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
Expand All @@ -110,28 +114,133 @@ jobs:
run: cd talawa_lint && flutter pub get && cd ..
- name: Running pub get to fetch dependencies
run: flutter pub get
- name: Codebase testing
- name: Run tests (shard ${{ matrix.shard }}/${{ env.TOTAL_SHARDS }})
run: |
CONCURRENCY="${FLUTTER_TEST_CONCURRENCY:-$(nproc)}"
if [ "$CONCURRENCY" -gt 4 ]; then CONCURRENCY=4; fi
flutter test --concurrency="$CONCURRENCY" --coverage
flutter test --concurrency="$CONCURRENCY" \
--shard-index=${{ matrix.shard }} \
--total-shards=${{ env.TOTAL_SHARDS }} \
--coverage
- name: Remove generated files from coverage
if: always()
run: |
sed -i '/\.g\.dart/d' coverage/lcov.info
dart pub global activate remove_from_coverage
dart pub global run remove_from_coverage:remove_from_coverage -f coverage/lcov.info -r '\.g\.dart$'
Comment on lines 125 to +129
Copy link
Contributor

@coderabbitai coderabbitai bot Jan 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

🧩 Analysis chain

🌐 Web query:

remove_from_coverage pub.dev Dart package versions

💡 Result:

remove_from_coverage on pub.dev currently has 5 published (stable) versions: 2.0.0, 1.0.7, 1.0.3, 1.0.1, 1.0.0. The latest is 2.0.0 (min Dart SDK 2.12). (pub.dev)

Usage (from the package docs), e.g. to remove generated *.g.dart files from coverage/lcov.info:

remove_from_coverage -f coverage/lcov.info -r '\.g\.dart$'

(or pub global run remove_from_coverage:remove_from_coverage ...). (pub.dev)

Citations:

🌐 Web query:

Flutter 3.32.x remove_from_coverage compatibility

💡 Result:

remove_from_coverage is compatible with Flutter 3.32.x.

  • Latest remove_from_coverage (2.0.0) requires Dart SDK >= 2.12 (null-safety) and has no upper bound that would block Dart 3.x, so it will resolve under Flutter 3.32.x’s Dart SDK. [1]
  • Flutter 3.32.0 rolls to Dart 3.8.0 (and 3.32.x bugfix releases use Dart 3.8.x). [2]

Practical note: the package docs still show older pub global run ... examples; on modern Flutter/Dart you can typically use dart run remove_from_coverage ... (as a dev_dependency) or dart pub global activate remove_from_coverage then run it. [3]

Sources:
[1] https://pub.dev/packages/remove_from_coverage/versions
[2] https://docs.flutter.dev/release/release-notes/release-notes-3.32.0
[3] https://pub.dev/documentation/remove_from_coverage/latest/

🌐 Web query:

Dart pub global activate version pinning best practices

💡 Result:

Best practices for version pinning with dart pub global activate

  1. Prefer an explicit version (or tight range) instead of “latest”.
    dart pub global activate <package> <version-constraint> lets you pin exactly (or constrain) what gets installed (for example, markdown 0.6.0, or a range like <3.0.0). This is the simplest way to make installs reproducible across machines and CI runs. [1]

  2. In CI, pin exactly (and install every run).
    Don’t rely on whatever happens to already be in the runner’s global cache. Use a pinned activation step (exact version), then run the tool.

  3. For Git-based global tools, pin to a commit SHA (or tag) with --git-ref.
    Pub uses the repo’s default branch unless you specify otherwise; --git-ref lets you lock to a specific branch/tag/commit (best is a commit SHA for full reproducibility). [1]

  4. Document the tool version in your repo (even if installed globally).
    Put the exact dart pub global activate … <version> (or --git-ref …) command in README, Makefile, or CI config so everyone installs the same version.

  5. Audit and update intentionally.
    You can see what’s globally activated via dart pub global list, and upgrades happen by re-activating (so updates are an explicit action you control). [1]

Note: Dart docs also mention dart install as a newer alternative to dart pub global activate. [1]

Sources
[1] Dart docs: dart pub global (activation with version constraints; --git-ref; listing; updating) https://dart.dev/tools/pub/cmd/pub-global

Pin remove_from_coverage to a specific version (e.g., 2.0.0) for reproducible CI.

Currently, dart pub global activate remove_from_coverage fetches the latest version on every run. Unpinned activations can introduce unexpected behavior changes. Update to dart pub global activate remove_from_coverage 2.0.0 (the latest stable version) and review for updates intentionally.

Current code 
      - name: Remove generated files from coverage
        if: always()
        run: |
          dart pub global activate remove_from_coverage
          dart pub global run remove_from_coverage:remove_from_coverage -f coverage/lcov.info -r '\.g\.dart$'
🤖 Prompt for AI Agents 
In @.github/workflows/push.yml around lines 125 - 129, The GitHub Actions step
"Remove generated files from coverage" currently activates the
`remove_from_coverage` package without a version which risks non-reproducible
CI; update the activation command used in that step so `dart pub global activate
remove_from_coverage` is changed to pin a specific version (e.g., `dart pub
global activate remove_from_coverage 2.0.0`) and keep the subsequent `dart pub
global run remove_from_coverage:remove_from_coverage -f coverage/lcov.info -r
'\.g\.dart$'` invocation unchanged so the workflow consistently uses the
intended `remove_from_coverage` release.

- name: Upload coverage artifact
if: always()
uses: actions/upload-artifact@v4
with:
name: coverage-shard-${{ matrix.shard }}
path: ./coverage/
retention-days: 1

Merge-Coverage:
name: Merge Coverage Reports
runs-on: ubuntu-latest
needs: [Flutter-Testing]
if: success()
steps:
- name: Checkout the Repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Fetch base branch for Codecov comparison
run: |
set -e
echo "Fetching base branch: develop"
if git fetch origin develop:develop 2>/dev/null; then
echo "Successfully fetched develop branch"
else
echo "Warning: Could not fetch develop branch, using current HEAD"
fi
- uses: actions/setup-java@v4
with:
distribution: "zulu"
java-version: "17.0"
- uses: subosito/flutter-action@v2
with:
flutter-version: ${{ env.FLUTTER_VERSION }}
channel: "stable"
- name: Download all coverage artifacts
id: download-artifacts
continue-on-error: true
uses: actions/download-artifact@v4
with:
pattern: coverage-shard-*
path: ./coverage-shards/
merge-multiple: false
- name: Check if artifacts were downloaded
id: check-artifacts
run: |
if find coverage-shards -name "lcov.info" -type f | grep -q .; then
echo "artifacts_found=true" >> $GITHUB_OUTPUT
echo "Coverage artifacts found"
else
echo "artifacts_found=false" >> $GITHUB_OUTPUT
echo "No coverage artifacts found"
exit 1
fi
- name: Install lcov
if: steps.check-artifacts.outputs.artifacts_found == 'true'
run: sudo apt-get update && sudo apt-get install -y lcov

- name: Merge coverage reports
if: steps.check-artifacts.outputs.artifacts_found == 'true'
run: |
mkdir -p ./coverage
echo "Merging coverage from shards..."
find coverage-shards -name "lcov.info" -type f > lcov-files.txt
if [ ! -s lcov-files.txt ]; then
echo "ERROR: No lcov.info files found"
exit 1
fi
echo "Found lcov files:"
cat lcov-files.txt

# Properly merge using lcov to avoid duplicate entries
FIRST_FILE=true
while IFS= read -r file; do
if [ "$FIRST_FILE" = true ]; then
cp "$file" ./coverage/lcov.info
FIRST_FILE=false
else
lcov --add-tracefile ./coverage/lcov.info \
--add-tracefile "$file" \
--output-file ./coverage/lcov.info.tmp
mv ./coverage/lcov.info.tmp ./coverage/lcov.info
fi
done < lcov-files.txt

echo "Merged coverage file size: $(wc -c < ./coverage/lcov.info) bytes"
SF_COUNT=$(grep -c "^SF:" ./coverage/lcov.info || echo "0")
echo "Number of files in merged coverage: $SF_COUNT"
if [ "$SF_COUNT" -lt 50 ]; then
echo "::warning::Only $SF_COUNT files in coverage (expected 50+)"
fi

- name: Calculate merge base for Codecov
run: |
MERGE_BASE=$(git merge-base origin/develop HEAD || echo "HEAD")
echo "MERGE_BASE=$MERGE_BASE" >> $GITHUB_ENV
echo "Using merge base: $MERGE_BASE"
Comment on lines +222 to +226
Copy link
Contributor

@coderabbitai coderabbitai bot Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

🧩 Analysis chain

🏁 Script executed:

rg -n "MERGE_BASE" .github/workflows/push.yml

Repository: PalisadoesFoundation/talawa

Length of output: 258

🏁 Script executed:

sed -n '220,250p' .github/workflows/push.yml

Repository: PalisadoesFoundation/talawa

Length of output: 1280

The MERGE_BASE environment variable is calculated but never used in the workflow.

The merge base is computed and stored in $GITHUB_ENV, but it's not referenced in the subsequent Codecov upload step. Either remove this step if unnecessary, or pass the merge base to Codecov via its action parameters if needed for accurate diff coverage.

🤖 Prompt for AI Agents 
In @.github/workflows/push.yml around lines 222 - 226, The workflow computes
MERGE_BASE in the "Calculate merge base for Codecov" step but never uses it;
either remove that step or propagate the value into the Codecov upload step so
it actually affects upload behavior. Locate the step that sets MERGE_BASE (the
git merge-base call) and either delete it, or modify the Codecov upload step to
consume the value by referencing the environment variable produced (e.g.,
reference MERGE_BASE via the step's inputs or env using the workflow expression
for environment variables) so the computed merge base is passed to Codecov.


#######################################################################
# DO NOT DELETE ANY references to env.CODECOV_UNIQUE_NAME in this
# section. They are required for accurate calculations
#######################################################################
- name: Present and upload coverage to Codecov as ${{env.CODECOV_UNIQUE_NAME}}
if: steps.check-artifacts.outputs.artifacts_found == 'true'
uses: codecov/codecov-action@v5
with:
name: "${{env.CODECOV_UNIQUE_NAME}}"
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: false
fail_ci_if_error: true
verbose: true
exclude: "docs/"
gcov_ignore: "docs/"
files: ./coverage/lcov.info
flags: flutter

Android-Build-and-Release:
if: ${{ github.actor != 'dependabot[bot]' }}
Expand All @@ -140,7 +249,7 @@ jobs:
contents: write
environment: TALAWA_ENVIRONMENT
runs-on: ubuntu-latest
needs: Flutter-Testing
needs: Merge-Coverage
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
Expand Down
Loading
Loading