Refactor: Coverage Script Cleanup & Auto-Ratchet by arodage · Pull Request #350 · open-edge-platform/image-composer-tool

arodage · 2026-01-16T00:27:45Z

Merge Checklist

All boxes should be checked before merging the PR

The changes in the PR have been built and tested
Ready to merge

Description

This PR refactors the test coverage infrastructure to reduce temp file clutter and adds an automatic coverage threshold ratchet mechanism.

Coverage Script (scripts/run_coverage_tests.sh):

Refactored to use a temp directory with automatic cleanup
Reduced output from 40+ files to just 2 files: coverage.out and coverage_report.txt
Fixed SIGPIPE (exit 141) issue when head closes pipes early
Removed confusing "Functions with Lowest Coverage" section from report

GitHub Workflow (.github/workflows/unit-test-and-coverage-gate.yml):

Simplified the workflow
Consolidated 8 artifact uploads to 1
Added clean tabular markdown summary in job output
Added automatic coverage threshold ratchet:
- When coverage improves by 0.5%+, automatically updates .coverage-threshold
- Commits the update to the PR branch (not main)
- Works for both pull_request and workflow_dispatch events

Earthfile:

Updated to read threshold from .coverage-threshold file
Allows override via --COV_THRESHOLD arg

New File: .coverage-threshold:

Single source of truth for coverage threshold
Currently set to 64.2% and should be automatically updated on PR creation

Any Newly Introduced Dependencies

None

How Has This Been Tested?

Script tested locally, produces correct output files
Ratchet mechanism verified in CI - automatically updated threshold from 64.2% to 65.7%
All existing unit tests pass

Copilot

Pull request overview

This PR refactors the test coverage infrastructure to streamline file management and implement an automatic coverage threshold ratcheting mechanism. The changes reduce temporary file clutter from 40+ files to just 2 output files while adding intelligent threshold management that automatically raises the bar when coverage improves.

Changes:

Refactored coverage script to use a temporary directory with automatic cleanup, reducing output files from 40+ to 2
Added automatic coverage threshold ratcheting in the GitHub workflow that updates .coverage-threshold when coverage improves by 0.5%+
Updated Earthfile to read threshold from .coverage-threshold file with override capability

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
scripts/run_coverage_tests.sh	Refactored to use temp directory with cleanup, consolidated output files, and improved argument handling
Earthfile	Updated to read threshold from `.coverage-threshold` file with fallback and override support
.github/workflows/unit-test-and-coverage-gate.yml	Simplified workflow, consolidated artifacts, added markdown summary formatting, and implemented auto-ratchet mechanism
.coverage-threshold	New file storing the current coverage threshold value

Comments suppressed due to low confidence (1)

scripts/run_coverage_tests.sh:1

The fallback value 65.0 in the Earthfile differs from the script's default 64.2 on line 23. These should be synchronized to avoid confusion, or document why they differ.

#!/bin/bash

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Use temp directory for intermediate coverage files (auto-cleanup on exit) - Reduce output from 40+ files to 2: coverage.out, coverage_report.txt - Consolidate reports into single coverage_report.txt with all details - Simplify workflow from 228 to 128 lines with single artifact upload - Add tabular markdown summary with emojis in GitHub Actions summary - Fix coverage threshold default to match script (64.2%) - Remove redundant legacy artifact uploads and debug job

- Replace coverage_total.txt, coverage_packages.txt, test_raw.log with single coverage_report.txt - Update comment for test-debug target

The coverage_report.txt uses **label:** format, updated sed to handle this

- Earthfile now accepts COV_THRESHOLD arg (default: 64.2) - Script accepts COV_THRESHOLD as first argument - Removes duplication: single source of default in Earthfile - Allows workflow to override threshold for manual runs - Improve package coverage report: show package summary + top 10 uncovered functions - Shorten function paths for readability in report

Wrap piped commands with head in subshell with || true to ignore SIGPIPE

Directory-level coverage table provides sufficient detail

- Add .coverage-threshold file (65.7%) as single source of truth - Earthfile reads threshold from file if not provided via arg - Workflow auto-updates threshold on main when coverage increases - Uses 0.5% buffer below actual coverage to avoid flaky failures - Threshold only ratchets up, never down (prevents regression)

- Runs on pull_request events instead of push to main - Commits threshold update to the PR's head branch - Threshold gets merged to main naturally when PR is merged

- Use single-line commit message - Quote github.head_ref references - Replace arrow character with ASCII

- Compare new_threshold vs old_threshold (not current vs old+0.1) - Add set -x for debugging - Move git config before fetch/checkout - Show proposed new threshold in logs

- Changed condition from event_name == 'pull_request' to ref != 'refs/heads/main' - Dynamically determine branch name based on event type - Works for both PR and manual dispatch events

* Add system-packages-only DOT filtering * Addressed copilot comments * Fix test cases for DownloadPackagesComplete signature change - Update DownloadPackagesComplete calls in debutils/download_test.go to include new pkgSources and systemRootsOnly parameters - Update DownloadPackagesComplete calls in rpmutils/download_test.go to include new parameters - Fix TestMergeSystemConfigWithImmutability to include SecureBootDBKey, making the immutability config non-empty per implementation design

Signed-off-by: Mah, Yock Gen <yock.gen.mah@intel.com>

* docs: add ADR for dep-analyzer tool Propose a dependency graph analyzer tool (dep-analyzer) that enables: - Slicing DOT dependency graphs by root package and depth - Reverse dependency analysis ("what depends on X?") - Graph statistics and root/leaf package listing - Multi-format output (DOT, SVG, PNG, PDF) This ADR documents the design decisions and rationale before implementation. * Accepted copilot suggestion Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* feat(scripts): add dep-analyzer.sh for DOT graph analysis Add a comprehensive DOT dependency graph analyzer tool that: - Slices dependency graphs by depth using BFS traversal - Supports forward and reverse (--reverse) dependency analysis - Lists root packages (--list-roots) with no incoming dependencies - Lists leaf packages (--list-leaves) with no outgoing dependencies - Provides graph statistics (--stats) - Outputs multiple formats: dot, svg, png, pdf - Highlights root node in output graphs (--highlight-root) This tool complements the DOT file generation feature in os-image-composer and enables detailed package dependency analysis. Related: ADR in docs/architecture/adr-dep-analyzer.md * copilot suggestion Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * refactor(scripts): extract HIGHLIGHT_PENWIDTH constant Address code review feedback by defining penwidth as a named constant at the top of the script for easier adjustment and better documentation. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Initial draft of the image inspect / compare ADR * Update adr-image-inspect.md added class diagram * Add ImageSummary struct, error handling section, verbose flag, and fix similarity logic * Update adr-image-inspect.md * Update adr-image-inspect.md --------- Co-authored-by: Rodage, Alpesh Ramesh <alpesh.ramesh.rodage@intel.com>

* Add ADR: Template-Enriched RAG for AI-Powered Template Generation This ADR proposes a Template-Enriched RAG architecture for AI-powered OS image template generation. Key features include: - Self-describing templates with embedded metadata for semantic matching - Single search path with hybrid scoring (semantic + keyword + package) - Embedding cache strategy for fast startup - Conversational interaction for iterative template refinement - Query classification with adaptive scoring weights - Future agentic capabilities for auto-validation and self-correction The design enables natural language template generation grounded in real, working examples while maintaining simplicity and extensibility. Technical Area: AI/ML, Template Generation Status: Proposed * Add complete ADR content for Template-Enriched RAG The initial commit had empty content. This commit adds the full ADR document with: - Problem statement and context - Template metadata schema - High-level architecture with Mermaid diagrams - Query classification and hybrid scoring - Embedding cache strategy - Conversational interaction flow - Agentic capabilities (future enhancement) - Implementation phases and configuration * removed approval section

- Use simpler sed patterns for better readability - Add validation for coverage value before bc calculation - Add git pull --rebase after checkout to handle branch updates

- Add persist-credentials: false to checkout action - Use environment variables instead of direct template interpolation to prevent potential template injection attacks from: - inputs.cov_threshold - steps.config.outputs.cov_threshold - steps.config.outputs.build_id - github.head_ref - github.ref_name

Push trigger only on main to avoid running twice when pushing to a branch with an open PR (pull_request event handles that).

Signed-off-by: Mah, Yock Gen <yock.gen.mah@intel.com>

* Initial commit of image inspection functionality * Update internal/image/imageinspect/imageinspect.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update testtables_test.go conditionally skipping if file does not exist * Update imageinspect_integration_test.go Conditionally skipping integration test if file does not exist * Update bootloader_pe.go lint issue * Update testtables_test.go lint issue * Update imageinspect.go lint issue * Update imageinspect_core_test.go added additional testcases not requiring an actual image file --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Alpesh <alpesh.ramesh.rodage@intel.com>

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Signed-off-by: Teoh Suh Haw <suh.haw.teoh@intel.com>

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Signed-off-by: Teoh Suh Haw <suh.haw.teoh@intel.com>

- Normalize both coverage and threshold to 1 decimal place before comparison to avoid precision mismatch (e.g., 64.29% displays as 64.3% but fails >= 64.3) - Update threshold to 64.2% to match current coverage on main

Copilot AI review requested due to automatic review settings January 16, 2026 00:27

Copilot AI reviewed Jan 16, 2026

View reviewed changes

Comment thread .github/workflows/unit-test-and-coverage-gate.yml Outdated

Comment thread .github/workflows/unit-test-and-coverage-gate.yml Outdated

Comment thread .github/workflows/unit-test-and-coverage-gate.yml Outdated

Comment thread .github/workflows/unit-test-and-coverage-gate.yml

arodage changed the title ~~refactor: reduce coverage script temp files and add auto-ratchet~~ Refactor: Reduce coverage script temp files and add auto-ratchet Jan 16, 2026

arodage changed the title ~~Refactor: Reduce coverage script temp files and add auto-ratchet~~ Refactor: Coverage Script Cleanup & Auto-Ratchet Jan 16, 2026

yockgen approved these changes Jan 20, 2026

View reviewed changes

arodage and others added 25 commits January 20, 2026 18:53

chore: update Earthfile to use new coverage output file names

913c2d7

- Replace coverage_total.txt, coverage_packages.txt, test_raw.log with single coverage_report.txt - Update comment for test-debug target

fix: correct sed patterns to parse markdown-formatted coverage report

4d73b63

The coverage_report.txt uses **label:** format, updated sed to handle this

fix: prevent SIGPIPE (exit 141) when head closes pipe early

1212434

Wrap piped commands with head in subshell with || true to ignore SIGPIPE

chore: remove functions coverage section from report

9029791

Directory-level coverage table provides sufficient detail

fix: update coverage threshold on PR branch instead of main

ebfa8bc

- Runs on pull_request events instead of push to main - Commits threshold update to the PR's head branch - Threshold gets merged to main naturally when PR is merged

fix: fix YAML syntax error in workflow

03897c1

- Use single-line commit message - Quote github.head_ref references - Replace arrow character with ASCII

resetting coverage to current threshold

932d57e

fix: correct coverage ratchet logic and add debug output

ecf3c44

- Compare new_threshold vs old_threshold (not current vs old+0.1) - Add set -x for debugging - Move git config before fetch/checkout - Show proposed new threshold in logs

feat: run coverage ratchet for manual workflow_dispatch runs too

05a45d6

- Changed condition from event_name == 'pull_request' to ref != 'refs/heads/main' - Dynamically determine branch name based on event type - Works for both PR and manual dispatch events

chore: auto-update coverage threshold to 65.2% (was 64.2%)

568bb97

Update .coverage-threshold

04f0c0f

fix ubuntu missing package for dlstreamer

82bdf58

Rebasing with latest changes from main and updating config.go unit tests

894091c

Signed-off-by: Mah, Yock Gen <yock.gen.mah@intel.com>

Fixing unit test issue

686387b

Signed-off-by: Mah, Yock Gen <yock.gen.mah@intel.com>

Fixing unit test issue attempt 2

f6eacef

Signed-off-by: Mah, Yock Gen <yock.gen.mah@intel.com>

Fixing unit test issue attempt 3

0525cd3

Signed-off-by: Mah, Yock Gen <yock.gen.mah@intel.com>

Fixing unit test issue attempt 4

bc721fe

Signed-off-by: Mah, Yock Gen <yock.gen.mah@intel.com>

Imporving configurations merging logic

ca6ba41

Signed-off-by: Mah, Yock Gen <yock.gen.mah@intel.com>

arodage and others added 18 commits January 20, 2026 18:54

fix: use printf for consistent decimal formatting in coverage threshold

558c524

fix: address Copilot review comments

a8b95e6

- Use simpler sed patterns for better readability - Add validation for coverage value before bc calculation - Add git pull --rebase after checkout to handle branch updates

chore: auto-update coverage threshold to 65.3% (was 64.2%)

d00e4d9

feat: trigger unit tests on push to any branch

ea794de

fix: avoid duplicate workflow runs on PR branches

1be9cec

Push trigger only on main to avoid running twice when pushing to a branch with an open PR (pull_request event handles that).

Enabling gpg key ignoring for debian repo marked as [trusted=yes]

b429346

Signed-off-by: Mah, Yock Gen <yock.gen.mah@intel.com>

Update internal/ospackage/debutils/verify.go for better logic

320c980

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Add separate github workflow for azl3 raw image build

39713c6

Signed-off-by: Teoh Suh Haw <suh.haw.teoh@intel.com>

Add separate github workflow for azl3 iso image build

77578ef

Signed-off-by: Teoh Suh Haw <suh.haw.teoh@intel.com>

Add separate github workflow for exlr,emt and ubuntu image build

3f23b12

Signed-off-by: Teoh Suh Haw <suh.haw.teoh@intel.com>

Update boot_tester.yml

699c93d

Signed-off-by: Teoh Suh Haw <suh.haw.teoh@intel.com>

Remove validate.sh and PR tester workflow

5484d3c

Signed-off-by: Teoh Suh Haw <suh.haw.teoh@intel.com>

Update .github/workflows/build-azl3-raw.yml

4ffebbf

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update gitleaks

1cf7177

Signed-off-by: Teoh Suh Haw <suh.haw.teoh@intel.com>

fix: normalize coverage values and update threshold

203bd7b

- Normalize both coverage and threshold to 1 decimal place before comparison to avoid precision mismatch (e.g., 64.29% displays as 64.3% but fails >= 64.3) - Update threshold to 64.2% to match current coverage on main

arodage force-pushed the refactor/coverage-script-cleanup branch 2 times, most recently from 0be5065 to 203bd7b Compare January 21, 2026 00:02

Merge branch 'main' into refactor/coverage-script-cleanup

53b2cde

arodage merged commit 252993d into main Jan 21, 2026
17 checks passed

arodage deleted the refactor/coverage-script-cleanup branch January 21, 2026 00:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor: Coverage Script Cleanup & Auto-Ratchet#350

Refactor: Coverage Script Cleanup & Auto-Ratchet#350
arodage merged 44 commits intomainfrom
refactor/coverage-script-cleanup

arodage commented Jan 16, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

arodage commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Checklist

Description

Any Newly Introduced Dependencies

How Has This Been Tested?

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

arodage commented Jan 16, 2026 •

edited

Loading