Skip to content

[WIP] ci: add Codecov bot for coverage reporting#1492

Open
jamesgao-jpg wants to merge 9 commits intozilliztech:mainfrom
jamesgao-jpg:ci/codecov-integration
Open

[WIP] ci: add Codecov bot for coverage reporting#1492
jamesgao-jpg wants to merge 9 commits intozilliztech:mainfrom
jamesgao-jpg:ci/codecov-integration

Conversation

@jamesgao-jpg
Copy link
Collaborator

@jamesgao-jpg jamesgao-jpg commented Mar 10, 2026

Summary

  • Add codecov/codecov-action@v4 upload step to ut.yaml after coverage report generation (tokenless, no secret needed for public repo)
  • Add WITH_COVERAGE flag to Makefile for coverage-enabled builds
  • Remove unused scripts/run_codecov.sh and its Makefile target (it redundantly re-ran tests and was not used by any CI pipeline)
  • Remove branch restriction on Codecov PR comments so coverage results display on all PRs

Builds on PR #1450 which refactored UT and added coverage report generation, but kept pr separate for cleaness.

Test plan

  • Verify Codecov bot posts a coverage comment on this PR after CI runs
  • Confirm coverage data appears on codecov.io for the knowhere repo

foxspy and others added 3 commits March 10, 2026 07:44
- Add test_helpers.h: IndexTestHelper template class for reducing repetitive Build/Search code
- Add test_config_generators.h: Centralized ConfigGenerator class for all index configurations
- Add test_flat.cc: Dedicated Flat index tests (build, search, bitset, range search, serialization)
- Add test_distance_unit.cc: Unit tests for distance computation (L2, IP, L1, Linf)
- Add test_distance_datatypes.cc: Multi-datatype distance tests (fp16, bf16, int8)
- Add test_utility_components.cc: BlockingQueue and BitsetView unit tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
- Merge test_distance_unit.cc into test_distances.cc
- Merge test_distance_datatypes.cc into test_simd.cc
- Add sparse_utils unit tests (SparseRow, MaxMinHeap, GrowableVectorView, DocIdFilterByVector)
- Add HNSW edge case tests (empty graph, heavy filtering, iterator)
- Enable coverage reporting in CI workflow (with_coverage=True)
- Add codecov.yml configuration (70% global, 80% patch targets)
- Add scripts/run_incremental_coverage.sh for local coverage analysis

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
Coverage report generation is handled inline in ut.yaml workflow.
The run_codecov.sh script redundantly re-runs tests and is not
referenced by any CI pipeline.

Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
@sre-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jamesgao-jpg
To complete the pull request process, please assign chasingegg after the PR has been reviewed.
You can assign the PR to them by writing /assign @chasingegg in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jamesgao-jpg jamesgao-jpg changed the title [experimental] ci: add Codecov bot for coverage reporting [WIP] ci: add Codecov bot for coverage reporting Mar 10, 2026
@mergify mergify bot added the dco-passed label Mar 10, 2026
@mergify
Copy link

mergify bot commented Mar 10, 2026

@jamesgao-jpg 🔍 Important: PR Classification Needed!

For efficient project management and a seamless review process, it's essential to classify your PR correctly. Here's how:

  1. If you're fixing a bug, label it as kind/bug.
  2. For small tweaks (less than 20 lines without altering any functionality), please use kind/improvement.
  3. Significant changes that don't modify existing functionalities should be tagged as kind/enhancement.
  4. Adjusting APIs or changing functionality? Go with kind/feature.

For any PR outside the kind/improvement category, ensure you link to the associated issue using the format: “issue: #”.

Thanks for your efforts and contribution to the community!.

Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
…ces.cc

Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
@mergify mergify bot removed the ci-passed label Mar 10, 2026
@codecov
Copy link

codecov bot commented Mar 10, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 32.17%. Comparing base (3c46f4c) to head (2aa8ca9).
⚠️ Report is 522 commits behind head on main.

❌ Your project check has failed because the head coverage (32.17%) is below the target coverage (60.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff            @@
##           main    #1492       +/-   ##
=========================================
+ Coverage      0   32.17%   +32.17%     
=========================================
  Files         0      421      +421     
  Lines         0    52466    +52466     
  Branches      0    12720    +12720     
=========================================
+ Hits          0    16883    +16883     
- Misses        0    33153    +33153     
- Partials      0     2430     +2430     

see 371 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
- TC1: remove topk=120 (Heap/Reservoir handler is orthogonal to refine config)
- TC3: replace dim={31,511,1024} with {31,32} (511 same padding branch as 31, 1024->32 same no-pad branch)
- TC4: remove redundant UINT8+quant=false sub-cases for bf16/fp16 (build covered by quant=true, search covered by BF16/FP16+quant=false)
- TC5: remove scann_gen4 (UINT8+quant=false) from emb list config section (build subset of gen3, search covered by first section)

2. modify codecov.yml to include testing thirdparty libs

Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
@jamesgao-jpg jamesgao-jpg force-pushed the ci/codecov-integration branch from b3d46d5 to 2aa8ca9 Compare March 13, 2026 11:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants