Skip to content

ci: reduce GitHub Actions consumption across CI workflows#11007

Merged
cesararroba merged 13 commits intomasterfrom
ci/actions-optimization
May 5, 2026
Merged

ci: reduce GitHub Actions consumption across CI workflows#11007
cesararroba merged 13 commits intomasterfrom
ci/actions-optimization

Conversation

@cesararroba
Copy link
Copy Markdown
Contributor

@cesararroba cesararroba commented May 5, 2026

Context

The org hit the 50,000 private-Actions-minutes monthly quota last cycle (~405% of quota at the 30-day mark). prowler is a public repo (Actions free in this repo) but every workflow file here is auto-mirrored to the private prowler-cloud repo, where minutes are billed. Optimizations applied here therefore reduce prowler-cloud billable consumption directly via the existing workflow-sync.

This PR groups eight thematic CI changes that, together, are estimated to remove ~21–25k Actions minutes / 30 days at current PR cadence (per audit baseline; see "Description" for per-change figures).

Description

Eight commits, one per theme. Per-change estimates are conservative midpoints from the 30-day baseline (window 2026-04-05 → 2026-05-05).

  1. ci(ui): cancel in-progress E2E runs on superseded PR pushes — adds concurrency.cancel-in-progress to ui-e2e-tests-v2.yml. Estimated ~3,500 min/30d (was the highest-value missing-concurrency hit at 12,328 min/30d total).
  2. ci: add workflow-level paths filters to skip irrelevant PR runs — adds paths: filters under push: and pull_request: for 8 SDK/API/UI test/quality/security workflows so PRs that don't touch each component skip them. Estimated ~6,000–7,000 min/30d combined.
  3. ci(container-checks): add paths filters, build amd64-only on PRs, use cache mode=min — three co-located changes to the four *-container-checks.yml files. amd64-only on PRs alone saves ~3,000 min/30d on SDK and ~2,500 + ~1,800 min/30d on API/UI; paths filters and cache-mode change add ~1–2k more.
  4. ci(container-build-push): use cache mode=min on PR builds — same conditional-mode pattern on the four *-container-build-push.yml files. Behavioural no-op today (no PR triggers) but kept for consistency and future-proofing. ~0 min/30d immediate; secondary cache-eviction relief.
  5. ci: add nightly arm64 container build verification — new nightly-arm64-container-builds.yml that builds sdk/api/ui/mcp on ubuntu-24.04-arm at 04:00 UTC + on demand. Restores arm64 coverage dropped from PR matrix in commit 3. Cost: ~120 min/30d total across 4 components.
  6. ci: shallow-clone PR helpers and fetch only the base refpr-conflict-checker.yml, pr-check-changelog.yml, pr-check-compliance-mapping.yml switch from fetch-depth: 0 (full ~8k-commit history) to fetch-depth: 1 plus an explicit base-ref fetch. Estimated ~1,900 min/30d combined.
  7. ci(secrets): scan only the PR/push diff with TruffleHogfind-secrets.yml reduces fetch-depth: 0 → 50 and splits the scan step into PR (--since-commit base.sha) and push (--since-commit github.event.before) variants. Estimated ~1,000 min/30d.
  8. ci(mcp): bump version from changelog and skip republish if already on PyPI — addresses the chronic mcp-pypi-release HTTP 400 failures with both a structural fix (new mcp-bump-version.yml reads mcp_server/CHANGELOG.md for the target version) and a safety-net check (mcp-pypi-release.yml skips publish when the version already exists on PyPI). Cost-neutral; reliability win.

Updates after initial push

  • Commit 9 (697d14214) — ci(container-checks): drop multi-arch matrix; arm64 covered by nightly. Removed the arch matrix from the four *-container-checks.yml files entirely (was already amd64-only on PRs after commit 3; arm64 on master pushes was redundant given the nightly cron from commit 5). Additional ~1,000 min/30d savings on master pushes.
  • Commit 10 (4043830ff) — ci: remove EOL v3 and v4.x release branch references. The v3 and v4.6 release branches are EOL (last human commit ~13 months ago); removed dead trigger entries.
  • Commit 11 (03ff9cf26) — ci(mcp): drop bump-version workflow; pre-flight PyPI check is the safety net. Per user decision (Option C): dropped the new mcp-bump-version.yml introduced in commit 8. The pre-flight PyPI existence check in mcp-pypi-release.yml is sufficient to prevent the recurring HTTP 400; MCP version stays manually bumped.
  • Commit 12 (8a6355b29) — ci(secrets): rely on TruffleHog action's built-in PR/push diff handling. Regression fix: the --since-commit flag added in commit 7 is duplicated by the TruffleHog action's own diff-range handling and broke the workflow. Reverted to letting the action handle the range.
  • Commit 13 (6d1380c87) — ci(zizmor): silence secrets-outside-env on nightly-arm64 builds. Adds nightly-arm64-container-builds.yml to the existing secrets-outside-env ignore list in .github/zizmor.yml, matching the pattern already used by the other *-container-build-push workflows that legitimately need SLACK_BOT_TOKEN / SLACK_PLATFORM_DEPLOYMENTS without an environment scope.

⚠️ CRITICAL — Branch protection check required before merge

Commits 2 and 3 add paths: filters to workflows that produce job names commonly enrolled as required status checks. If any of the following are required in branch protection, PRs that don't match the new path filters will leave their required check stuck in pending and block merge.

Workflows whose job names should be verified against branch-protection required-checks:

  • sdk-tests.ymlsdk-tests (3.10), sdk-tests (3.11), sdk-tests (3.12)
  • sdk-code-quality.ymlsdk-code-quality
  • sdk-security.ymlsdk-security-scans
  • sdk-check-duplicate-test-names.ymlcheck-duplicate-test-names
  • api-tests.ymlapi-tests
  • api-code-quality.ymlapi-code-quality
  • api-security.ymlapi-security-scans
  • ui-tests.ymlui-tests
  • sdk-container-checks.ymlsdk-dockerfile-lint, sdk-container-build-and-scan
  • api-container-checks.ymlapi-dockerfile-lint, api-container-build-and-scan
  • ui-container-checks.ymlui-dockerfile-lint, ui-container-build-and-scan
  • mcp-container-checks.ymlmcp-dockerfile-lint, mcp-container-build-and-scan

Note: commit 9 dropped the arch matrix from the four *-container-checks.yml files, so <workflow>-container-build-and-scan is now a single static job (no matrix suffix). The names listed above already reflect the post-commit-9 shape.

If any are required, two mitigations are available:

  • Remove them from required-status-checks in branch protection; or
  • Add an always-green sentinel job in a separate workflow file with no paths: filter that always passes for the same job-name set.

Steps to review

Review per commit:

  • Commit 1 (d198da17a): single new concurrency: block on ui-e2e-tests-v2.yml. Confirm the group key resolves correctly across pull_request and push events (pull_request.number || github.ref).
  • Commit 2 (beea27aa0): 8 files, only paths: additions under existing push: / pull_request: triggers. Verify the path globs match each component's actual source tree.
  • Commit 3 (6c1828461): 4 files, three changes each (paths, matrix, cache-to). The fromJSON() ternary on matrix.include is the load-bearing line — verify it parses correctly and that the push/release branch keeps both arches.
  • Commit 4 (211285b4c): 4 files, one-line conditional swap for cache-to. No-op today; sanity-check the conditional parses.
  • Commit 5 (2e2a82c7d): new workflow. Verify the schedule cron, the Slack notification block uses existing secrets, and fail-fast: false is set.
  • Commit 6 (992a3dc5e): 3 files. Confirm the explicit git fetch --depth=1 origin "${BASE_REF}" step runs before tj-actions/changed-files in each.
  • Commit 7 (58c34cf21): one file. Verify the PR scan uses pull_request.base.sha and the push scan handles the zero-SHA fallback for first pushes. (Superseded by commit 12 — see Updates section.)
  • Commit 8 (0fbd45293): one new workflow + one modified. The new mcp-bump-version.yml mirrors the api-bump-version.yml skeleton; verify the changelog regex matches the existing ## [x.y.z] (Prowler v…) shape used in mcp_server/CHANGELOG.md. The mcp-pypi-release.yml change is a single pre-flight curl block + two if: gates on Build/Publish. (Bump workflow removed by commit 11 — see Updates section.)

All 23 modified/created workflows pass actionlint and zizmor (the latter required two env:-routed shell expansions in the new bump workflow to silence template-injection findings — applied during this PR).

Checklist

Community Checklist
  • This feature/issue is listed in here or roadmap.prowler.com
  • Is it assigned to me, if not, request it via the issue/feature in here or Prowler Community Slack

SDK/CLI

  • Are there new checks included in this PR? No — CI workflow changes only; no SDK behaviour change.

UI

  • All issue/task requirements work as expected on the UI
  • Screenshots/Video of the functionality flow (if applicable) - Mobile (X < 640px) — N/A
  • Screenshots/Video of the functionality flow (if applicable) - Table (640px > X < 1024px) — N/A
  • Screenshots/Video of the functionality flow (if applicable) - Desktop (X > 1024px) — N/A
  • Ensure new entries are added to CHANGELOG.md, if applicable.

API

  • All issue/task requirements work as expected on the API
  • Endpoint response output (if applicable) — N/A
  • EXPLAIN ANALYZE output for new/modified queries or indexes (if applicable) — N/A
  • Performance test results (if applicable) — N/A
  • Any other relevant evidence of the implementation (if applicable) — N/A
  • Verify if API specs need to be regenerated.
  • Check if version updates are required (e.g., specs, Poetry, etc.).
  • Ensure new entries are added to CHANGELOG.md, if applicable.

License

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Add a concurrency group to ui-e2e-tests-v2.yml so a new push to
the same PR cancels any in-flight E2E run. UI E2E is the highest
wall-clock workflow without cancel-in-progress today.
Add paths filters to the SDK, API, and UI test/quality/security
workflows so PRs that don't touch each component's source tree
no longer trigger them. Same paths applied to push and pull_request.

Files: sdk-tests, sdk-code-quality, sdk-security,
sdk-check-duplicate-test-names, api-tests, api-code-quality,
api-security, ui-tests.
… cache mode=min

Three co-located changes to sdk-/api-/ui-/mcp-container-checks.yml:

- Add workflow-level paths filters (component source + workflow file
  + Dockerfiles where applicable) so unrelated PRs skip the workflow.
- Drop the arm64 matrix leg on pull_request events; push and release
  events keep amd64+arm64. arm64 PR coverage is preserved by the new
  nightly arm64 build verification workflow (separate commit).
- Switch buildx cache-to mode from 'max' to 'min' on pull_request
  events to reduce per-PR cache footprint and ease eviction pressure
  at the 10 GiB per-repo cache cap. Push/release keep mode=max.
Apply the same conditional cache-to mode pattern used in
*-container-checks.yml for consistency. These workflows currently
have no pull_request trigger, so the conditional resolves to 'max'
on every existing event — behavioural no-op today, future-proof
when/if PR triggers are added.
Schedule a nightly cron (04:00 UTC) plus workflow_dispatch that
builds the sdk, api, ui, and mcp container images on
ubuntu-24.04-arm runners. This restores the arm64 coverage that
was dropped from PR-time matrix in *-container-checks.yml so
arm64 regressions are caught within ~24h of merge.

Notify on failure via existing Slack secrets. fail-fast: false
so one component's failure does not mask the others. Cache scoped
to arm64 with mode=min — does not interfere with PR/push caches.
Replace fetch-depth: 0 (full history clone) with fetch-depth: 1 in
the PR conflict checker and the PR changelog/compliance-mapping
checkers, then explicitly fetch the PR base ref so
tj-actions/changed-files still resolves the diff range.

These workflows only need the diff against the base branch, not the
entire ~8k-commit history.
Reduce fetch-depth from 0 (full history) to 50 commits — TruffleHog
needs the diff range, not the entire history. Then split the scan
step into PR and push variants so each scans only its own delta:

- PR: --since-commit $base.sha --branch $head.ref
- Push: --since-commit $github.event.before, with a fallback to
  github.sha when event.before is the zero SHA (first push to a
  new branch).
… PyPI

Two related fixes for the mcp-pypi-release HTTP 400 'file already
exists' failures.

Structural fix — new mcp-bump-version.yml: trigger on
release.published. Resolves the target MCP version from
mcp_server/CHANGELOG.md by matching '(Prowler v<release_tag>)' —
the same regex prepare-release.yml uses for release notes. Skips
when no MCP entry exists for the released Prowler version, and
when pyproject.toml is already at the changelog version. Mirrors
api-bump-version's master + version-branch PR pattern.

Safety net — mcp-pypi-release.yml: pre-flight PyPI version check
gates the build and publish steps. If the version in
mcp_server/pyproject.toml already exists on PyPI, the workflow
emits a ::notice:: and skips publish instead of failing the run.
Covers releases with no MCP changes, unmerged bump PRs, and
workflow_dispatch re-runs.
@cesararroba cesararroba requested a review from a team as a code owner May 5, 2026 14:02
@github-actions github-actions Bot added the github_actions Pull requests that update GitHub Actions code label May 5, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 5, 2026

✅ All necessary CHANGELOG.md files have been updated.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 5, 2026

Conflict Markers Resolved

All conflict markers have been successfully resolved in this pull request.

Comment thread .github/workflows/mcp-bump-version.yml Fixed
Comment thread .github/workflows/mcp-bump-version.yml Fixed
Comment thread .github/workflows/mcp-bump-version.yml Fixed
Comment thread .github/workflows/nightly-arm64-container-builds.yml Fixed
Comment thread .github/workflows/nightly-arm64-container-builds.yml Fixed
@codecov
Copy link
Copy Markdown

codecov Bot commented May 5, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.65%. Comparing base (7c6d658) to head (6d1380c).
⚠️ Report is 7 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master   #11007       +/-   ##
===========================================
+ Coverage   59.14%   93.65%   +34.50%     
===========================================
  Files           8      230      +222     
  Lines         399    33936    +33537     
===========================================
+ Hits          236    31782    +31546     
- Misses        163     2154     +1991     
Flag Coverage Δ
api 93.65% <ø> (?)
prowler-py3.10-kubernetes ?
prowler-py3.11-kubernetes ?
prowler-py3.12-kubernetes ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
prowler ∅ <ø> (∅)
api 93.65% <ø> (∅)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cesararroba cesararroba merged commit 9827768 into master May 5, 2026
35 of 37 checks passed
@cesararroba cesararroba deleted the ci/actions-optimization branch May 5, 2026 15:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

github_actions Pull requests that update GitHub Actions code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants