Skip to content

feat(sdk): discover external universal compliance frameworks via entry points#11490

Merged
StylusFrost merged 61 commits into
masterfrom
PROWLER-1444-multi-provider-compliance-entry-points
Jun 9, 2026
Merged

feat(sdk): discover external universal compliance frameworks via entry points#11490
StylusFrost merged 61 commits into
masterfrom
PROWLER-1444-multi-provider-compliance-entry-points

Conversation

@StylusFrost

@StylusFrost StylusFrost commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Context

External plug-ins can register per-provider compliance via the prowler.compliance entry point, but there was no way to ship multi-provider (universal-schema) frameworks. This adds that, building on the universal compliance support already in the SDK.

Scope

  • Includes: prowler.compliance.universal entry-point group, built-in precedence on name collisions, fatal flag on load_compliance_framework
  • Excludes: dashboard rendering, API consumption

Autonomy

  • CI is expected to pass for this PR branch
  • This PR has one deliverable scope
  • This PR can be rolled back without unrelated changes
  • Tests cover this unit

Description

  • get_bulk_compliance_frameworks_universal now scans a dedicated prowler.compliance.universal entry point group, so external plug-ins can ship universal-schema frameworks. Kept separate from the per-provider prowler.compliance group, so the legacy per-provider loader never parses a universal JSON.
  • Built-ins load first and win on a framework-name collision. Multiple packages registering under the same provider name are merged, not collided.
  • load_compliance_framework gains a fatal flag: an external JSON that fails the legacy schema is skipped with a warning instead of aborting the run (previously a sys.exit that the surrounding except Exception could not catch).

SDK-only change in prowler/lib/check/compliance_models.py plus tests.

Steps to review

  1. A universal-schema framework registered under prowler.compliance.universal is returned by get_bulk_compliance_frameworks_universal.
  2. Built-ins win on a name collision; multiple packages under the same provider name are merged.
  3. A universal JSON wrongly registered under prowler.compliance is skipped with a warning and prowler aws --compliance does not crash.

Checklist

  • Are there new checks included in this PR? No
  • Review if the code is being covered by tests.
  • Ensure new entries are added to CHANGELOG.md, if applicable.

License

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Summary by CodeRabbit

  • New Features

    • Added support for external and multi-provider compliance frameworks that can be dynamically registered and discovered
  • Improvements

    • Invalid external compliance files are now logged as warnings and skipped, rather than halting execution

StylusFrost and others added 30 commits April 15, 2026 13:22
…very

  - Implemented dynamic loading of external providers via entry points, allowing for greater flexibility in provider integration.
  - Added functionality to discover compliance directories from entry points, enabling external compliance frameworks to be loaded seamlessly.
  - Refactored check module resolution to prioritize built-in checks while falling back to entry points if necessary.
  - Improved compliance framework loading to include both built-in and external sources, ensuring comprehensive compliance coverage.
  - Enhanced CLI argument parsing to support external providers, improving user experience and configurability.
  - Introduced extensive unit tests to validate dynamic loading, compliance discovery, and overall integration of external providers.
…iders

load_and_validate_config_file only detected the namespaced format for 5
hardcoded providers (aws, gcp, azure, kubernetes, m365). For every other
built-in (github, nhn, vercel, cloudflare, iac, llm, image, mongodbatlas,
oraclecloud, openstack, alibabacloud, googleworkspace) and for any
external plug-in, the full YAML was returned wrapped instead of the
provider's own block.

Replace the hardcoded list with a dynamic check: if the file has a
top-level key matching the provider and its value is a dict, unwrap it.
Keep the legacy flat format for AWS only (historical, pre-multicloud)
and identify it by the absence of nested-dict top-level values, which
prevents cross-provider config leakage when a namespaced file has no
section for the requested provider.
…covery

Conflict in prowler/config/config.py resolved by combining both branches:
- HEAD: external compliance discovery via entry points (PROWLER-1391)
- master: multi-provider framework JSONs scanned at top-level compliance/ (#10300)

Order: built-in per-provider -> built-in multi-provider -> external entry points.
Built-ins first so they win on name collisions against external registrations.

Supporting external plug-ins to register multi-provider frameworks is tracked
in PROWLER-1444.
…er-contract-dynamic-discovery

# Conflicts:
#	prowler/config/config.py
…very

Calling importlib.util.find_spec on prowler.providers.{provider}.services
for an external provider propagates ModuleNotFoundError when the parent
package prowler.providers.{provider} does not exist, instead of returning
None. This caused recover_checks_from_provider, _resolve_check_module and
Scan.scan to fail with "No module named 'prowler.providers.{external}'"
even though the plug-in registered its checks via entry points correctly.

Gate the built-in branch on Provider.is_builtin (which already wraps the
find_spec in try/except) and reuse _resolve_check_module from Scan.scan
so external providers fall through to the entry-point lookup.
@StylusFrost StylusFrost requested a review from a team as a code owner June 7, 2026 11:52
…ints

External plug-ins ship multi-provider (universal-schema) frameworks through a
dedicated prowler.compliance.universal entry point group, separate from the
per-provider prowler.compliance group. Both get_bulk_compliance_frameworks_universal
(loading) and get_available_compliance_frameworks (listing / --compliance
choices) scan the new group. Built-ins load first and win on a name collision;
multiple packages under the same provider are merged. load_compliance_framework
gains fatal=False so the legacy external path skips a non-legacy JSON with a
warning instead of aborting the run.
@StylusFrost StylusFrost force-pushed the PROWLER-1444-multi-provider-compliance-entry-points branch from 841bb72 to 40da359 Compare June 7, 2026 11:57
@StylusFrost StylusFrost self-assigned this Jun 7, 2026
pedrooot
pedrooot previously approved these changes Jun 8, 2026

@pedrooot pedrooot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing work 👏🏼

Base automatically changed from PROWLER-1391-provider-contract-dynamic-discovery to master June 8, 2026 15:47
@StylusFrost StylusFrost dismissed pedrooot’s stale review June 8, 2026 15:47

The base branch was changed.

@StylusFrost StylusFrost requested a review from a team as a code owner June 8, 2026 15:47
…provider-compliance-entry-points

# Conflicts:
#	prowler/CHANGELOG.md
#	prowler/config/config.py
#	prowler/lib/check/compliance_models.py
#	tests/providers/external/test_dynamic_provider_loading.py
@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Conflict Markers Resolved

All conflict markers have been successfully resolved in this pull request.

@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

✅ All necessary CHANGELOG.md files have been updated.

@codecov

codecov Bot commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 74.28571% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.30%. Comparing base (1f7caa6) to head (cd65963).
⚠️ Report is 2 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (1f7caa6) and HEAD (cd65963). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (1f7caa6) HEAD (cd65963)
api 1 0
Additional details and impacted files
@@             Coverage Diff             @@
##           master   #11490       +/-   ##
===========================================
- Coverage   93.97%   74.30%   -19.68%     
===========================================
  Files         241      110      -131     
  Lines       35441     8522    -26919     
===========================================
- Hits        33305     6332    -26973     
- Misses       2136     2190       +54     
Flag Coverage Δ
api ?
prowler-py3.10-config 74.30% <74.28%> (?)
prowler-py3.10-external 21.93% <57.14%> (?)
prowler-py3.10-lib 73.99% <74.28%> (?)
prowler-py3.11-config 74.30% <74.28%> (?)
prowler-py3.11-external 21.93% <57.14%> (?)
prowler-py3.11-lib 73.99% <74.28%> (?)
prowler-py3.12-config 74.30% <74.28%> (?)
prowler-py3.12-external 21.93% <57.14%> (?)
prowler-py3.12-lib 73.99% <74.28%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
prowler 74.26% <74.28%> (∅)
api ∅ <ø> (∅)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

🔒 Container Security Scan

Image: prowler:3335083
Last scan: 2026-06-09 10:58:27 UTC

📊 Vulnerability Summary

Severity Count
🔴 Critical 13
Total 13

8 package(s) affected

⚠️ Action Required

Critical severity vulnerabilities detected. These should be addressed before merging:

  • Review the detailed scan results
  • Update affected packages to patched versions
  • Consider using a different base image if updates are unavailable

📋 Resources:

cesararroba
cesararroba previously approved these changes Jun 8, 2026
@coderabbitai

coderabbitai Bot commented Jun 9, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 4eee2915-d2ca-4714-b8df-07e7c6f9970d

📥 Commits

Reviewing files that changed from the base of the PR and between 7e60e8f and cd65963.

📒 Files selected for processing (5)
  • prowler/CHANGELOG.md
  • prowler/config/config.py
  • prowler/lib/check/compliance_models.py
  • tests/lib/check/universal_compliance_models_test.py
  • tests/providers/external/test_dynamic_provider_loading.py

📝 Walkthrough

Walkthrough

This PR extends Prowler's compliance framework system to support external universal (multi-provider) frameworks via entry points. The implementation adds graceful error handling for invalid external schemas, discovers frameworks through the prowler.compliance.universal entry-point group, and includes comprehensive tests validating discovery, deduplication, and bulk-loading behavior.

Changes

Universal Compliance Frameworks

Layer / File(s) Summary
Framework loading error tolerance foundation
prowler/lib/check/compliance_models.py
load_compliance_framework() gains fatal: bool = True parameter and returns Optional[Compliance]; when fatal=False, invalid schemas log warning and return None instead of exiting. Compliance.get_bulk() calls the loader with fatal=False to tolerate external framework errors.
Universal framework discovery via entry points
prowler/config/config.py, prowler/lib/check/compliance_models.py
get_available_compliance_frameworks() discovers universal frameworks from prowler.compliance.universal entry points, resolves module paths, scans for JSON files, and filters by supports_provider(). get_bulk_compliance_frameworks_universal() loads discovered JSONs with collision avoidance against built-in frameworks.
Universal bulk loading test suite
tests/lib/check/universal_compliance_models_test.py
TestGetBulkUniversalEntryPoints validates bulk loading: external frameworks are discovered, built-in frameworks override external on name collision, all JSONs in a single entry-point directory are loaded, and frameworks from multiple packages merge by provider.
Integration tests for discovery and filtering
tests/providers/external/test_dynamic_provider_loading.py
Tests verify get_available_compliance_frameworks includes universal frameworks for both specific and unspecified providers; Compliance.get_bulk correctly excludes non-legacy external universal-schema frameworks.
Changelog documentation
prowler/CHANGELOG.md
Version 5.30.0 release notes document support for registering external multi-provider compliance frameworks via prowler.compliance.universal entry point group.

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 68.75% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: adding discovery of external universal compliance frameworks via entry points, which is the primary objective of this SDK-focused PR.
Description check ✅ Passed The PR description covers all required template sections: Context explains the motivation, Description details the changes with specific scope, Steps to review provide validation guidance, and Checklist is completed with license confirmation.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch PROWLER-1444-multi-provider-compliance-entry-points

Comment @coderabbitai help to get the list of available commands and usage tips.

@StylusFrost StylusFrost merged commit 6c559fb into master Jun 9, 2026
27 of 29 checks passed
@StylusFrost StylusFrost deleted the PROWLER-1444-multi-provider-compliance-entry-points branch June 9, 2026 11:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants