Add inline diff coverage check to Azure Pipeline by hdwhdw · Pull Request #597 · sonic-net/sonic-gnmi

hdwhdw · 2026-02-27T20:20:41Z

Why I did it

The coverage.sonic-net.sonic-gnmi.build GitHub Check is posted by a black-box GitHub App ("Azure-Pipelines-Wrapper") with no code visibility. Moving diff coverage enforcement into the public azure-pipelines.yml follows infrastructure-as-code principles and makes the check transparent and maintainable.

How I did it

Added a new DiffCoverageCheck job in the Build stage that:

Depends on both PureCIJob and build jobs
Downloads coverage artifacts published by both upstream jobs
Runs diff-cover against the combined coverage data
Fails the check if new/modified lines have less than 80% coverage
Only runs on PR builds (condition: eq(variables['Build.Reason'], 'PullRequest'))

Other changes:

Publish coverage XML as pipeline artifacts from PureCIJob and build job
Add fetchDepth: 0 to build job checkout for git merge-base support
Remove unused DIFF_COVER_* variables from MemoryLeakJob and build job
Add DIFF_COVER_THRESHOLD top-level variable (80%)
Upgrade PublishCodeCoverageResults from v1 to v2
Add diff-cover Makefile target for local testing

PureCIJob ──publish coverage-pure.xml──┐
                                       ├──> DiffCoverageCheck (diff-cover)
build ─────publish coverage.xml────────┘

How to verify it

Validated in build 1048370. The DiffCoverageCheck job ran and correctly reported diff coverage on the dummy test packages (which were intentionally under-tested):

Diff Coverage
Diff: origin/master...HEAD, staged and unstaged changes
-------------
testpkg/integration/helper.go (38.5%): Missing lines 19-20,22,27-30,33
testpkg/pure/math.go (30.8%): Missing lines 18,23-24,26,31-32,34-35,37
-------------
Total:   26 lines
Missing: 17 lines
Coverage: 34%

The dummy test packages have been removed in the latest commit since they served only as validation vehicles.

After merging, the Azure-Pipelines-Wrapper GitHub App can be uninstalled from the repository.

Which release branch to backport (provide reason below if selected)

Description for the changelog

Replace black-box Azure-Pipelines-Wrapper GitHub App with inline diff coverage check in Azure Pipeline YAML.

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

Replace the black-box Azure-Pipelines-Wrapper GitHub App with an inline DiffCoverageCheck job that runs diff-cover against combined pure and integration coverage artifacts. Changes: - Add DiffCoverageCheck job depending on PureCIJob and build - Publish coverage XML as pipeline artifacts from both jobs - Add fetchDepth: 0 to build job for git merge-base support - Remove unused DIFF_COVER_* variables from MemoryLeakJob and build - Add DIFF_COVER_THRESHOLD top-level variable (80%) - Upgrade PublishCodeCoverageResults to v2 - Add diff-cover Makefile target for local testing - Add dummy test packages for PR validation Signed-off-by: Dawei Huang <daweihuang@microsoft.com>

mssonicbld · 2026-02-27T20:20:49Z

/azp run

azure-pipelines · 2026-02-27T20:20:59Z

Azure Pipelines successfully started running 1 pipeline(s).

Copilot

Pull request overview

This PR replaces the Azure-Pipelines-Wrapper GitHub App with a first-party Azure Pipelines job that runs diff-cover against combined pure + integration coverage artifacts to enforce inline diff coverage on PRs.

Changes:

Adds a DiffCoverageCheck job that downloads published coverage artifacts and runs diff-cover with a configurable threshold.
Publishes pure and integration Cobertura XML files as pipeline artifacts and upgrades PublishCodeCoverageResults to v2.
Adds a local make diff-cover target and introduces dummy packages wired into pure/integration test package lists.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
`azure-pipelines.yml`	Adds diff-cover gating job, publishes coverage artifacts, sets `fetchDepth: 0`, and upgrades coverage publishing task version.
`pure.mk`	Adds `testpkg/pure` to the “pure” package set so it runs in pure CI/coverage.
`Makefile`	Adds `testpkg/integration` to integration basic packages and introduces a `diff-cover` convenience target.
`testpkg/pure/math.go`	Dummy pure functions to generate diff lines for coverage validation.
`testpkg/pure/math_test.go`	Dummy tests with intentionally incomplete coverage.
`testpkg/integration/helper.go`	Dummy integration-like helper functions to generate diff lines for coverage validation.
`testpkg/integration/helper_test.go`	Dummy tests with intentionally incomplete coverage.

testpkg/pure/math_test.go

Copilot · 2026-02-27T20:27:13Z

testpkg/integration/helper.go

+// Repeat returns s repeated n times.
+func Repeat(s string, n int) string {
+	result := ""
+	for i := 0; i < n; i++ {
+		result += s
+	}
+	return result
+}


Repeat builds the result via result += s inside a loop, which causes repeated allocations/copies (quadratic behavior as n grows). Use strings.Repeat(s, n) or a strings.Builder/[]byte preallocation approach to keep runtime and allocations predictable.

Dismissed — dummy packages have been removed.

testpkg/integration/helper_test.go

azure-pipelines.yml

pure.mk

Makefile

With a single checkout step, Azure Pipelines checks out directly into the working directory without a subdirectory. Signed-off-by: Dawei Huang <daweihuang@microsoft.com>

mssonicbld · 2026-02-27T21:06:43Z

/azp run

azure-pipelines · 2026-02-27T21:06:52Z

Azure Pipelines successfully started running 1 pipeline(s).

The DiffCoverageCheck job was validated in build 1048370. Removing the temporary testpkg/ packages and their Makefile/pure.mk entries. Signed-off-by: Dawei Huang <daweihuang@microsoft.com>

mssonicbld · 2026-02-27T21:54:47Z

/azp run

azure-pipelines · 2026-02-27T21:54:56Z

Azure Pipelines successfully started running 1 pipeline(s).

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

Makefile

azure-pipelines.yml

…re coverage in Makefile - Remove codeCoverageTool input from PublishCodeCoverageResults@2 (v2 auto-detects) - Add succeeded() guard to DiffCoverageCheck condition to skip on upstream failure - Include test-results/coverage-pure.xml in Makefile diff-cover target for parity with pipeline Signed-off-by: Dawei Huang <daweihuang@microsoft.com>

mssonicbld · 2026-02-27T22:02:22Z

/azp run

azure-pipelines · 2026-02-27T22:02:33Z

Azure Pipelines successfully started running 1 pipeline(s).

…lity The Azure-Pipelines-Wrapper GitHub App needs these variables to complete its check. They can be removed once the app is uninstalled. Signed-off-by: Dawei Huang <daweihuang@microsoft.com>

hdwhdw · 2026-02-27T23:31:25Z

/azp run

azure-pipelines · 2026-02-27T23:31:34Z

Azure Pipelines successfully started running 1 pipeline(s).

Copilot AI review requested due to automatic review settings February 27, 2026 20:20

Copilot started reviewing on behalf of hdwhdw February 27, 2026 20:21 View session

Copilot AI reviewed Feb 27, 2026

View reviewed changes

Fix DiffCoverageCheck: remove cd sonic-gnmi for single checkout

f4ead75

With a single checkout step, Azure Pipelines checks out directly into the working directory without a subdirectory. Signed-off-by: Dawei Huang <daweihuang@microsoft.com>

Remove dummy test packages after validating diff coverage pipeline

ab43d1d

The DiffCoverageCheck job was validated in build 1048370. Removing the temporary testpkg/ packages and their Makefile/pure.mk entries. Signed-off-by: Dawei Huang <daweihuang@microsoft.com>

hdwhdw requested a review from Copilot February 27, 2026 21:55

Copilot started reviewing on behalf of hdwhdw February 27, 2026 21:55 View session

Copilot AI reviewed Feb 27, 2026

View reviewed changes

Makefile Outdated Show resolved Hide resolved

azure-pipelines.yml Outdated Show resolved Hide resolved

azure-pipelines.yml Outdated Show resolved Hide resolved

hdwhdw requested review from ryanzhu706 and zbud-msft February 27, 2026 23:00

Restore DIFF_COVER_* variables in build job for wrapper app compatibi…

0e7e5a6

…lity The Azure-Pipelines-Wrapper GitHub App needs these variables to complete its check. They can be removed once the app is uninstalled. Signed-off-by: Dawei Huang <daweihuang@microsoft.com>

hdwhdw removed request for ryanzhu706 and zbud-msft February 27, 2026 23:02

hdwhdw requested review from Ryangwaite, ryanzhu706 and zbud-msft February 28, 2026 00:54

Conversation

hdwhdw commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why I did it

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

Uh oh!

mssonicbld commented Feb 27, 2026

Uh oh!

azure-pipelines bot commented Feb 27, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

hdwhdw Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mssonicbld commented Feb 27, 2026

Uh oh!

azure-pipelines bot commented Feb 27, 2026

Uh oh!

mssonicbld commented Feb 27, 2026

Uh oh!

azure-pipelines bot commented Feb 27, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mssonicbld commented Feb 27, 2026

Uh oh!

azure-pipelines bot commented Feb 27, 2026

Uh oh!

hdwhdw commented Feb 27, 2026

Uh oh!

azure-pipelines bot commented Feb 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

hdwhdw commented Feb 27, 2026 •

edited

Loading