Skip to content

Commit 2786999

Browse files
imaqsoodclaude
andcommitted
(MODULES-11831) Add SimpleCov coverage gate with baseline mode
Sync a coverage gate to all modules via `pdk update`: * moduleroot/.github/workflows/coverage.yml.erb - thin caller that delegates to puppetlabs/cat-github-actions module_coverage.yml, passing threshold and baseline_mode from .sync.yml. * moduleroot/spec/spec_helper.rb.erb - SimpleCov hook that starts before any application code loads. Gated behind the COVERAGE env var and the `simplecov` config flag, so local `rake spec` runs are unaffected and there is no double-load or json/json_pure conflict. * config_defaults.yml - add `simplecov: true`, a managed coverage.yml entry (threshold 80, baseline_mode true), and the simplecov / simplecov-lcov gems. baseline_mode with no `.coverage_baseline` file is a no-op gate, so the rollout is non-breaking. * README.md - document the new keys and the two coverage modes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 6c994fc commit 2786999

4 files changed

Lines changed: 75 additions & 0 deletions

File tree

README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -200,6 +200,17 @@ release_schedule:
200200

201201
In this example the automated release prep workflow is triggered every Saturday at 3 am.
202202

203+
### .github/workflows/coverage.yml
204+
205+
> The coverage workflow enforces Ruby unit-test coverage on pull requests. It runs `bundle exec rake spec` with [SimpleCov](https://github.com/simplecov-ruby/simplecov) enabled (driven by the `simplecov` key on `spec/spec_helper.rb`), uploads the LCOV report as a build artifact, and posts the coverage result as a PR comment. It delegates to the reusable `puppetlabs/cat-github-actions/.github/workflows/module_coverage.yml` workflow. The gate runs in one of two modes:
206+
207+
| Key | Description |
208+
| :------------- |:--------------|
209+
|threshold|In absolute mode, the minimum line coverage percentage required for the build to pass. Defaults to `80`.|
210+
|baseline_mode|When `true` (the default), the build gates against a `.coverage_baseline` file in the repo root instead of `threshold`: it fails only if coverage drops below the recorded baseline, and stays green (suggesting a new baseline) when coverage rises. When no `.coverage_baseline` file exists the baseline is `0`, so the gate is a no-op and rollout is non-breaking. Set to `false` to gate against `threshold` instead.|
211+
212+
To start ratcheting coverage in baseline mode, commit a `.coverage_baseline` file containing the current line coverage percentage (for example `82.5`). To switch a module to a fixed target, set `baseline_mode: false` and tune `threshold` in `.sync.yml`. To opt a module out entirely, set both `simplecov: false` (on `spec/spec_helper.rb`) and `unmanaged: true` on `.github/workflows/coverage.yml`.
213+
203214
### .pdkignore
204215

205216
>A .pdkignore file in your repo allows you to specify files to ignore when building a module package with `pdk build`.
@@ -323,6 +334,7 @@ is_pe: false
323334
|strict_variables| Defines the [Puppet Strict Variables configuration parameter](https://puppet.com/docs/puppet/5.4/configuration.html#strict_variables). Defaults to `true` however due to `puppetlabs_spec_helper` forced override (https://github.com/puppetlabs/puppetlabs_spec_helper/blob/070ecb79a63cb8fa10f46532c413c055e2697682/lib/puppetlabs_spec_helper/module_spec_helper.rb#L71). Set to `false` to align with true default or with `STRICT_VARIABLES=no` environment setting.|
324335
|coverage_report|Enable [rspec-puppet coverage reports](https://rspec-puppet.com/documentation/coverage/). Defaults to `false`|
325336
|minimum_code_coverage_percentage|The desired code coverage percentage required for tests to pass. Defaults to `0`|
337+
|simplecov|Enables the [SimpleCov](https://github.com/simplecov-ruby/simplecov) hook used by the coverage workflow to measure Ruby line coverage of `lib/`. The hook only activates when the `COVERAGE` environment variable is set to `yes` (the coverage workflow sets this), so local `rake spec` runs are unaffected. Defaults to `true`. This is separate from `coverage_report`, which measures Puppet resource coverage.|
326338

327339
## Further Notes
328340

config_defaults.yml

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -555,8 +555,12 @@ Gemfile:
555555
version: '= 3.12.1'
556556
- gem: 'pry'
557557
version: '~> 0.10'
558+
- gem: 'simplecov'
559+
version: '~> 0.22'
558560
- gem: 'simplecov-console'
559561
version: '~> 0.9'
562+
- gem: 'simplecov-lcov'
563+
version: '~> 0.8'
560564
- gem: 'puppet-debugger'
561565
version: '~> 1.6'
562566
- gem: 'rubocop'
@@ -615,6 +619,16 @@ spec/spec_helper.rb:
615619
mock_with: ":rspec"
616620
strict_level: ":warning"
617621
strict_variables: true
622+
# Enables the SimpleCov hook used by the coverage workflow. Active only when
623+
# the COVERAGE env var is set, so local `rake spec` runs are unaffected.
624+
simplecov: true
625+
# The coverage workflow is managed (not unmanaged) so `pdk update` rolls it out
626+
# to every module. baseline_mode with no `.coverage_baseline` file is a no-op
627+
# gate (baseline 0), so deploying it is non-breaking; commit a baseline to start
628+
# ratcheting, or set `baseline_mode: false` to gate against `threshold`.
629+
.github/workflows/coverage.yml:
630+
threshold: 80
631+
baseline_mode: true
618632
.github/workflows/ci.yml:
619633
unmanaged: true
620634
.github/workflows/nightly.yml:
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
<% coverage = @configs['.github/workflows/coverage.yml'] || {} -%>
2+
name: "coverage"
3+
4+
on:
5+
pull_request:
6+
branches:
7+
- "main"
8+
workflow_dispatch:
9+
10+
permissions:
11+
contents: read
12+
pull-requests: write
13+
14+
concurrency:
15+
group: ${{ github.workflow }}-${{ github.ref }}
16+
cancel-in-progress: true
17+
18+
jobs:
19+
Coverage:
20+
uses: "puppetlabs/cat-github-actions/.github/workflows/module_coverage.yml@main"
21+
with:
22+
threshold: <%= coverage['threshold'] || 80 %>
23+
baseline_mode: <%= coverage['baseline_mode'] ? true : false %>

moduleroot/spec/spec_helper.rb.erb

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,31 @@
11
# frozen_string_literal: true
22

3+
<%- if @configs['simplecov'] -%>
4+
# SimpleCov must start before any application code is required so that every
5+
# line is tracked. Gated behind the COVERAGE env var so local `rake spec` runs
6+
# are unaffected; the coverage workflow sets COVERAGE=yes.
7+
if ENV['COVERAGE'] == 'yes'
8+
require 'simplecov'
9+
require 'simplecov-lcov'
10+
11+
SimpleCov::Formatter::LcovFormatter.config do |config|
12+
config.report_with_single_file = true
13+
end
14+
SimpleCov.formatters = SimpleCov::Formatter::MultiFormatter.new(
15+
[
16+
SimpleCov::Formatter::SimpleFormatter,
17+
SimpleCov::Formatter::LcovFormatter,
18+
],
19+
)
20+
SimpleCov.start do
21+
track_files 'lib/**/*.rb'
22+
add_filter '/spec/'
23+
add_filter '/vendor/'
24+
add_filter '/.vendor/'
25+
end
26+
end
27+
28+
<%- end -%>
329
<%- if @configs['mock_with'] -%>
430
RSpec.configure do |c|
531
c.mock_with <%= @configs['mock_with'] %>

0 commit comments

Comments
 (0)