Aggregate codecov across root + sub projects for cumulative report #800

cwperks · 2025-07-16T18:56:09Z

Description

Currently, only tests from the root project count towards the codecov report which shows that only 33% of the lines in this repo are covered by tests.

This PR attempts to aggregate the jacoco reports from root + sub projects (SPI and sample plugin) for one cumulative report to get a more accurate report of code coverage.

Related Issues

Resolves #612

Check List

New functionality includes testing.
New functionality has been documented.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Craig Perkins <[email protected]>

…, but metadata exists Signed-off-by: Craig Perkins <[email protected]>

Signed-off-by: Craig Perkins <[email protected]>

…e report Signed-off-by: Craig Perkins <[email protected]>

Signed-off-by: Craig Perkins <[email protected]>

codecov · 2025-07-16T19:21:31Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.74%. Comparing base (e88586e) to head (bf8075f).
⚠️ Report is 4 commits behind head on main.

❌ Your project status has failed because the head coverage (69.74%) is below the target coverage (75.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@              Coverage Diff              @@
##               main     #800       +/-   ##
=============================================
+ Coverage     31.43%   69.74%   +38.30%     
=============================================
  Files            34       38        +4     
  Lines          1527     1735      +208     
  Branches        137      156       +19     
=============================================
+ Hits            480     1210      +730     
+ Misses         1010      435      -575     
- Partials         37       90       +53

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Craig Perkins <[email protected]>

cwperks · 2025-07-28T12:14:12Z

70% now :)

cwperks · 2025-07-28T17:27:22Z

sample-extension-plugin/build.gradle

        }
    }
-    systemProperty 'tests.security.manager', 'true'
+    systemProperty 'tests.security.manager', 'false'


I had to disable this bc the agent was preventing dumpCoverage from writing to build/jacoco.

I plan to figure out how to re-enable this in a subsequent PR.

Signed-off-by: Craig Perkins <[email protected]>

…pensearch-project#800) * WIP on allowing interval schedule to operate in seconds Signed-off-by: Craig Perkins <[email protected]> * Support defining IntervalSchedule in seconds Signed-off-by: Craig Perkins <[email protected]> * Use seconds in bwc test Signed-off-by: Craig Perkins <[email protected]> * Update runThenListJobs test Signed-off-by: Craig Perkins <[email protected]> * Consistent create watcher job Signed-off-by: Craig Perkins <[email protected]> * Add test that disables watcher job and verifies that it stops running, but metadata exists Signed-off-by: Craig Perkins <[email protected]> * Disable security manager for bwc test Signed-off-by: Craig Perkins <[email protected]> * Always test with security manager Signed-off-by: Craig Perkins <[email protected]> * Add java agent to SPI Signed-off-by: Craig Perkins <[email protected]> * Ensure that document is searchable immediately Signed-off-by: Craig Perkins <[email protected]> * Attempt to aggregate codecov across root + sub projects for cumulative report Signed-off-by: Craig Perkins <[email protected]> * Fix error Signed-off-by: Craig Perkins <[email protected]> * Try this again Signed-off-by: Craig Perkins <[email protected]> * Use codecov-action Signed-off-by: Craig Perkins <[email protected]> * Fix flakiness Signed-off-by: Craig Perkins <[email protected]> * Fix flakiness Signed-off-by: Craig Perkins <[email protected]> * Make sure coverage can be written after node exit using JMX Signed-off-by: Craig Perkins <[email protected]> * Fix forbidden Signed-off-by: Craig Perkins <[email protected]> * Simplify Signed-off-by: Craig Perkins <[email protected]> * Skip JMX for multinode tests Signed-off-by: Craig Perkins <[email protected]> * Use 0.8.13 Signed-off-by: Craig Perkins <[email protected]> * declare task dependency Signed-off-by: Craig Perkins <[email protected]> * Finalize test with jacoco Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Jeremy Dupras <[email protected]>

cwperks added 13 commits July 10, 2025 09:40

WIP on allowing interval schedule to operate in seconds

9b4e66f

Signed-off-by: Craig Perkins <[email protected]>

Support defining IntervalSchedule in seconds

f1d5629

Signed-off-by: Craig Perkins <[email protected]>

Use seconds in bwc test

38a80a3

Signed-off-by: Craig Perkins <[email protected]>

Merge branch 'main' into seconds

ad92df3

Update runThenListJobs test

89c3a7a

Signed-off-by: Craig Perkins <[email protected]>

Consistent create watcher job

1c89413

Signed-off-by: Craig Perkins <[email protected]>

Add test that disables watcher job and verifies that it stops running…

87eb506

…, but metadata exists Signed-off-by: Craig Perkins <[email protected]>

Merge branch 'main' into add-test-for-disable

c2cc448

Disable security manager for bwc test

34782a6

Signed-off-by: Craig Perkins <[email protected]>

Always test with security manager

f3b12ef

Signed-off-by: Craig Perkins <[email protected]>

Add java agent to SPI

6cee9ab

Signed-off-by: Craig Perkins <[email protected]>

Ensure that document is searchable immediately

28c5268

Signed-off-by: Craig Perkins <[email protected]>

Attempt to aggregate codecov across root + sub projects for cumulativ…

ade4f96

…e report Signed-off-by: Craig Perkins <[email protected]>

github-project-automation bot added this to Engineering Effectiveness Board Jul 16, 2025

Fix error

7884f89

Signed-off-by: Craig Perkins <[email protected]>

cwperks added 13 commits July 16, 2025 17:00

Merge branch 'main' into jacoco-coverage

eb283bd

Merge branch 'main' into jacoco-coverage

d318a65

Try this again

93ee699

Signed-off-by: Craig Perkins <[email protected]>

Use codecov-action

b3bac24

Signed-off-by: Craig Perkins <[email protected]>

Fix flakiness

1af1211

Signed-off-by: Craig Perkins <[email protected]>

Fix flakiness

5374017

Signed-off-by: Craig Perkins <[email protected]>

Make sure coverage can be written after node exit using JMX

267f948

Signed-off-by: Craig Perkins <[email protected]>

Fix forbidden

05adfa6

Signed-off-by: Craig Perkins <[email protected]>

Simplify

3e1c0d7

Signed-off-by: Craig Perkins <[email protected]>

Skip JMX for multinode tests

3b9139a

Signed-off-by: Craig Perkins <[email protected]>

Use 0.8.13

853ab93

Signed-off-by: Craig Perkins <[email protected]>

declare task dependency

3c0fea2

Signed-off-by: Craig Perkins <[email protected]>

Finalize test with jacoco

6bcab9e

Signed-off-by: Craig Perkins <[email protected]>

cwperks marked this pull request as ready for review July 28, 2025 12:15

cwperks requested review from dbwiddis, joshpalis, kaituo, prudhvigodithi, saratvemulapalli and vibrantvarun as code owners July 28, 2025 12:15

cwperks commented Jul 28, 2025

View reviewed changes

cwperks mentioned this pull request Jul 30, 2025

Rest API to list all locks with option to get a specific lock #802

Merged

5 tasks

Merge branch 'main' into jacoco-coverage

bf8075f

Signed-off-by: Craig Perkins <[email protected]>

cwperks changed the title ~~Attempt to aggregate codecov across root + sub projects for cumulative report~~ Aggregate codecov across root + sub projects for cumulative report Aug 2, 2025

prudhvigodithi approved these changes Aug 7, 2025

View reviewed changes

github-project-automation bot moved this to 👀 In Review in Engineering Effectiveness Board Aug 7, 2025

cwperks merged commit 0bab682 into opensearch-project:main Aug 7, 2025
23 of 25 checks passed

github-project-automation bot moved this from 👀 In Review to ✅ Done in Engineering Effectiveness Board Aug 7, 2025

cwperks mentioned this pull request Sep 24, 2025

[RFC] Include Integration tests in code coverage reports for a PR opensearch-project/OpenSearch#19382

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Aggregate codecov across root + sub projects for cumulative report #800

Aggregate codecov across root + sub projects for cumulative report #800

Uh oh!

cwperks commented Jul 16, 2025 •

edited

Loading

Uh oh!

codecov bot commented Jul 16, 2025 •

edited

Loading

Uh oh!

cwperks commented Jul 28, 2025

Uh oh!

cwperks Jul 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Aggregate codecov across root + sub projects for cumulative report #800

Aggregate codecov across root + sub projects for cumulative report #800

Uh oh!

Conversation

cwperks commented Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

codecov bot commented Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

cwperks commented Jul 28, 2025

Uh oh!

cwperks Jul 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cwperks commented Jul 16, 2025 •

edited

Loading

codecov bot commented Jul 16, 2025 •

edited

Loading