feat: add SBOM quality compliance scoring (NTIA/CISA/BSI) with policy enforcement by jfjrh2014 · Pull Request #27 · rezmoss/sbomlyze

jfjrh2014 · 2026-06-03T23:48:54Z

What

Addresses Issue #5 — score SBOM quality based on NTIA/CISA/BSI compliance.

Adds a --compliance flag that evaluates SBOM completeness against three recognized standards, plus compliance-based policy rules that let CI pipelines enforce minimum quality thresholds.

Changes

New package: `internal/compliance`

NTIA Minimum Elements (2021): 7 checks — supplier, name, version, unique identifiers, dependency relationship, SBOM author, timestamp
CISA FSCT-3 / 2025 Minimum Elements: 11 checks — NTIA base + PURL, license, hash, CPE
BSI TR-03183-2: 11 checks — strictest requirements including SHA-512 hashes, SPDX license identifiers, dependency completeness
Per-framework 0-100 score and overall score (average)
Text report with pass/fail per check and remediation descriptions

Policy integration (`internal/policy`)

New EvaluateCompliance() function with 4 policy rules:

Rule	Description
`min_ntia_score`	Fail if NTIA score below threshold (0-100)
`min_cisa_score`	Fail if CISA score below threshold
`min_bsi_score`	Fail if BSI TR-03183 score below threshold
`min_overall_compliance`	Fail if overall compliance below threshold

Usage: sbomlyze a.json b.json --compliance --policy policy.json

Output format integration

JSON: Compliance report embedded in output object ("compliance" field)
Markdown: Scoring table + collapsible failed check details
Text: Full report with per-check icons and descriptions

CLI

--compliance flag to enable scoring
Example policy: examples/policies/compliance.json

Testing

10 new compliance evaluation tests (NTIA/CISA/BSI basic, missing fields, SHA-512, overall scoring)
7 new compliance policy tests (threshold enforcement, zero thresholds, nil frameworks)
3 new CLI tests (--compliance flag parsing)
All existing tests pass (12 packages, 0 failures)

Example

# Score a single SBOM
sbomlyze image.json --compliance

# Enforce minimum scores in CI
sbomlyze image.json --compliance --policy compliance-policy.json

# With diff + compliance + markdown
sbomlyze before.json after.json --compliance --format markdown --policy policy.json

Policy file:

{
  "min_ntia_score": 85,
  "min_cisa_score": 70,
  "min_overall_compliance": 75,
  "deny_integrity_drift": true
}

Files Changed

File	Change
`internal/compliance/compliance.go`	New — compliance evaluation engine
`internal/compliance/compliance_test.go`	New — compliance tests
`internal/policy/policy.go`	Added `EvaluateCompliance()` + 4 score threshold fields
`internal/policy/policy_test.go`	Added compliance policy tests
`internal/cli/options.go`	Added `Compliance` field + `--compliance` flag parser
`internal/cli/options_test.go`	Added `--compliance` flag tests
`internal/output/markdown.go`	Added compliance scoring section to Markdown output
`cmd/sbomlyze/main.go`	Wired compliance into single-file and diff modes
`examples/policies/compliance.json`	New — example compliance policy

Adds --compliance flag that evaluates SBOM completeness against three recognized standards: - NTIA Minimum Elements (2021): 7 checks covering supplier, name, version, unique identifiers, dependencies, author, timestamp - CISA FSCT-3 / 2025 Minimum Elements: 11 checks adding PURL, license, hash, and CPE coverage on top of NTIA baseline - BSI TR-03183-2: 11 checks with strictest requirements including SHA-512 hashes, SPDX license identifiers, dependency completeness Also adds compliance-based policy rules for CI enforcement: - min_ntia_score: enforce minimum NTIA score threshold - min_cisa_score: enforce minimum CISA score threshold - min_bsi_score: enforce minimum BSI TR-03183 score threshold - min_overall_compliance: enforce minimum overall score Compliance report integrated into: - Text output (table with per-check pass/fail) - JSON output (embedded in response object) - Markdown diff output (scoring table with failed check details) Includes example policy file (examples/policies/compliance.json). Closes rezmoss#5

Copilot

Pull request overview

Adds an SBOM “quality/completeness” compliance scoring engine (NTIA/CISA/BSI) and allows CI policy enforcement via new minimum-score thresholds, exposing results across CLI and output formats.

Changes:

Introduces internal/compliance to evaluate NTIA/CISA/BSI checks and compute per-standard + overall scores.
Extends policy model with compliance score thresholds and evaluation (EvaluateCompliance), plus tests.
Adds --compliance CLI flag and wires compliance into main execution + output (JSON embedding and new Markdown section).

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
`internal/compliance/compliance.go`	New compliance evaluation engine + reporting helpers.
`internal/compliance/compliance_test.go`	New unit tests for scoring behavior across standards.
`internal/policy/policy.go`	Adds compliance threshold fields + evaluation function.
`internal/policy/policy_test.go`	Adds tests for JSON loading and compliance policy enforcement.
`internal/cli/options.go`	Adds `--compliance` flag parsing.
`internal/cli/options_test.go`	Adds CLI tests for `--compliance`.
`internal/output/markdown.go`	Adds Markdown compliance scoring section + new generator variant.
`cmd/sbomlyze/main.go`	Wires compliance evaluation + JSON embedding + policy threshold checks.
`examples/policies/compliance.json`	Adds example policy including compliance thresholds.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

rezmoss · 2026-06-04T17:53:48Z

+func componentCheck(id, name, desc string, passedCount, total int) CheckResult {
+	passed := passedCount == total
+	details := fmt.Sprintf("%d/%d components", passedCount, total)
+	if passed {
+		details = fmt.Sprintf("All %d components", total)
+	}
+	return CheckResult{
+		ID:          id,
+		Name:        name,
+		Description: desc,
+		Passed:      passed,
+		Details:     details,
+	}
+}


Fixed in 15dabf1. dep-relation is now an SBOM-level check (hasAnyDependency(comps)), so leaf components without deps no longer fail it. A new helper sbomDepSummary reports the count of components with deps at SBOM scope.

rezmoss · 2026-06-04T17:53:55Z

+	hasTimestamp := info.SchemaVersion != ""
+
+	checks := []CheckResult{
+		componentCheck("ntia-name", "Component Name",
+			"NTIA requires a component name for each entry (NTIA Minimum Elements §2.2)",
+			withName, total),
+		componentCheck("ntia-version", "Component Version",
+			"NTIA requires a version identifier for each component (NTIA Minimum Elements §2.3)",
+			withVersion, total),
+		componentCheck("ntia-supplier", "Supplier Name",
+			"NTIA requires the supplier/author name for each component (NTIA Minimum Elements §2.1)",
+			withSupplier, total),
+		componentCheck("ntia-unique-id", "Other Unique Identifiers",
+			"NTIA requires additional identifiers (PURL, CPE, etc.) for lookups (NTIA Minimum Elements §2.4)",
+			withUniqueID, total),
+		componentCheck("ntia-dep-relation", "Dependency Relationship",
+			"NTIA requires dependency relationships to be described (NTIA Minimum Elements §2.5)",
+			withDeps, total),
+		sbomCheck("ntia-author", "SBOM Author",
+			"NTIA requires identifying the entity that created the SBOM (NTIA Minimum Elements §2.6)",
+			hasAuthor, info.ToolName),
+		sbomCheck("ntia-timestamp", "SBOM Timestamp",
+			"NTIA requires a timestamp for when the SBOM was assembled (NTIA Minimum Elements §2.7)",
+			hasTimestamp, info.SchemaVersion),


Fixed in 15dabf1. NTIA timestamp now uses info.SBOMTimestamp (populated from CycloneDX metadata.timestamp, SPDX creationInfo.created, or empty for Syft). SchemaVersion is no longer abused as a timestamp.

rezmoss · 2026-06-04T17:53:59Z

+	hasAuthor := info.ToolName != ""
+	hasTimestamp := info.SchemaVersion != ""
+
+	checks := []CheckResult{
+		componentCheck("cisa-name", "Component Name",
+			"CISA requires a component name for each entry",
+			withName, total),
+		componentCheck("cisa-version", "Component Version",
+			"CISA requires a version identifier for each component",
+			withVersion, total),
+		componentCheck("cisa-supplier", "Supplier Name",
+			"CISA requires supplier identification for each component",
+			withSupplier, total),
+		componentCheck("cisa-unique-id", "Unique Identifiers",
+			"CISA requires additional identifiers for vulnerability lookups",
+			withUniqueID, total),
+		componentCheck("cisa-dep-relation", "Dependency Relationship",
+			"CISA requires dependency relationships to be described",
+			withDeps, total),
+		componentCheck("cisa-purl", "Package URL (PURL)",
+			"CISA strongly recommends PURLs for precise component identification (CISA 2025 §3.1)",
+			withPURL, total),
+		componentCheck("cisa-license", "License Information",
+			"CISA recommends license data for compliance and risk assessment (CISA 2025 §3.2)",
+			withLicense, total),
+		componentCheck("cisa-hash", "Integrity Hashes",
+			"CISA recommends cryptographic hashes for tamper detection (CISA 2025 §3.3)",
+			withHash, total),
+		componentCheck("cisa-cpe", "CPE Identifiers",
+			"CISA recommends CPEs for vulnerability database correlation (CISA 2025 §3.4)",
+			withCPE, total),
+		sbomCheck("cisa-author", "SBOM Author",
+			"CISA requires identifying the entity that created the SBOM",
+			hasAuthor, info.ToolName),
+		sbomCheck("cisa-timestamp", "SBOM Timestamp",
+			"CISA requires a timestamp for SBOM assembly",
+			hasTimestamp, info.SchemaVersion),


Fixed in 15dabf1. CISA timestamp now uses info.SBOMTimestamp. See reply on ntia-timestamp — same fix applies here.

rezmoss · 2026-06-04T17:54:04Z

+	hasAuthor := info.ToolName != ""
+	hasTimestamp := info.SchemaVersion != ""
+
+	checks := []CheckResult{
+		componentCheck("bsi-name", "Component Name",
+			"BSI TR-03183-2 §5.2.2 requires a component name",
+			withName, total),
+		componentCheck("bsi-version", "Component Version",
+			"BSI TR-03183-2 §5.2.2 requires a version identifier",
+			withVersion, total),
+		componentCheck("bsi-creator", "Component Creator",
+			"BSI TR-03183-2 §5.2.2 requires the component creator (supplier) contact",
+			withSupplier, total),
+		componentCheck("bsi-unique-id", "Unique Identifiers",
+			"BSI TR-03183-2 §5.2.2 requires additional identifiers (CPE, PURL, SWID, etc.)",
+			withUniqueID, total),
+		componentCheck("bsi-purl", "Package URL (PURL)",
+			"BSI TR-03183-2 recommends PURLs for unambiguous package identification",
+			withPURL, total),
+		componentCheck("bsi-license", "Distribution Licenses",
+			"BSI TR-03183-2 §5.2.2 requires SPDX license identifiers for each component",
+			withLicense, total),
+		componentCheck("bsi-hash", "Integrity Hash",
+			"BSI TR-03183-2 §5.2.2 recommends cryptographic hashes for integrity",
+			withHash, total),
+		componentCheck("bsi-sha512", "SHA-512 Hash",
+			"BSI TR-03183-2 §5.2.2 specifically requires SHA-512 checksums",
+			withSHA512, total),
+		componentCheck("bsi-dep-relation", "Dependency Relationship",
+			"BSI TR-03183-2 §5.2.2 requires dependency enumeration with completeness indication",
+			withDeps, total),
+		sbomCheck("bsi-author", "SBOM Author/Creator",
+			"BSI TR-03183-2 §5.2.1 requires the email/URL of the SBOM creator",
+			hasAuthor, info.ToolName),
+		sbomCheck("bsi-timestamp", "SBOM Timestamp",
+			"BSI TR-03183-2 §5.2.1 requires the date/time of SBOM compilation",
+			hasTimestamp, info.SchemaVersion),


Fixed in 15dabf1. BSI timestamp now uses info.SBOMTimestamp.

rezmoss · 2026-06-04T17:54:09Z

 		case "html":
 			fmt.Println(output.GenerateHTMLStats(stats, sbomInfo, findings))
+			if complianceReport != nil {
+				compliance.PrintReport(*complianceReport)
+			}


Fixed in 15dabf1. HTML stats output now uses output.GenerateHTMLStatsWithCompliance(...) which inlines the compliance section before the footer. No plaintext appended after </html>.

rezmoss · 2026-06-04T17:54:13Z

+	var complianceReport *compliance.Report
+	if opts.Compliance {
+		// Evaluate compliance on the "after" SBOM (second file)
+		r := compliance.Evaluate(comps2, info2)
+		complianceReport = &r
+	}


Fixed in 15dabf1. Markdown diff path now uses the existing output.GenerateMarkdownWithOverviewAndCompliance(...) and passes the compliance report.

rezmoss · 2026-06-04T17:54:19Z

+		if len(complianceViolations) > 0 {
+			output.PrintViolations(complianceViolations)
+			if policy.HasErrors(complianceViolations) {
+				os.Exit(1)
+			}
 		}


@jfjrh2014 could you plz review all above

Fixed in 15dabf1. In single-file mode, policy violations are written via output.PrintViolationsTo(w, ...) where w is os.Stderr for --format json and --format html, so structured output streams on stdout stay clean.

- Use SBOMTimestamp/SBOMAuthor for compliance checks (was: SchemaVersion/ToolName) - Dep-relation is now SBOM-level, not per-component (leaf components no longer fail) - HTML stats output inlines compliance section (was: appended plaintext AFTER html) - Markdown diff output wires compliance report (was: silently dropped) - Policy violations go to stderr for JSON/HTML formats (was: corrupted stdout stream) - Add SPDX parser metadata extraction (creationInfo.created, creators) - Author helper prefers SBOMAuthor, falls back to ToolName - Add HTML compliance writer + GenerateHTMLWithCompliance + GenerateHTMLStatsWithCompliance - Add PrintViolationsTo(io.Writer) helper - Update tests + snapshots for new SPDX metadata

Copilot

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.

rezmoss · 2026-06-05T21:25:20Z

+	if pol.MinNTIAScore > 0 && report.NTIA != nil && report.NTIA.Score < pol.MinNTIAScore {
+		violations = append(violations, Violation{
+			Rule:     "min_ntia_score",
+			Message:  fmt.Sprintf("NTIA compliance score %d < minimum %d (%d/%d checks passed)", report.NTIA.Score, pol.MinNTIAScore, report.NTIA.Passed, report.NTIA.Total),
+			Severity: SeverityError,
+		})
+	}


@jfjrh2014 plz review

rezmoss · 2026-06-05T21:25:27Z

+	if pol.MinCISAScore > 0 && report.CISA != nil && report.CISA.Score < pol.MinCISAScore {
+		violations = append(violations, Violation{
+			Rule:     "min_cisa_score",
+			Message:  fmt.Sprintf("CISA compliance score %d < minimum %d (%d/%d checks passed)", report.CISA.Score, pol.MinCISAScore, report.CISA.Passed, report.CISA.Total),
+			Severity: SeverityError,
+		})
+	}


@jfjrh2014 plz review

rezmoss · 2026-06-05T21:25:31Z

+	if pol.MinBSIScore > 0 && report.BSI != nil && report.BSI.Score < pol.MinBSIScore {
+		violations = append(violations, Violation{
+			Rule:     "min_bsi_score",
+			Message:  fmt.Sprintf("BSI TR-03183 compliance score %d < minimum %d (%d/%d checks passed)", report.BSI.Score, pol.MinBSIScore, report.BSI.Passed, report.BSI.Total),
+			Severity: SeverityError,
+		})
+	}


@jfjrh2014 plz review

rezmoss · 2026-06-05T21:25:35Z

+// --- BSI TR-03183-2 ---
+// Strictest standard: requires SHA-512 hashes, SPDX license identifiers,
+// component creator contact, filename, dependency completeness flag,
+// and binary properties (executable, archive, structured).
+


@jfjrh2014 plz review

rezmoss · 2026-06-05T21:25:39Z

+	t.Run("skips nil framework results", func(t *testing.T) {
+		pol := Policy{
+			MinNTIAScore: 80,
+			MinCISAScore: 80,
+			MinBSIScore:  80,
+		}
+		report := compliance.Report{
+			NTIA: nil, // empty SBOM = nil NTIA
+			Overall: 0,
+		}
+
+		violations := EvaluateCompliance(pol, report)
+		// Should only get min_overall_compliance (0 < 80 would trigger if set),
+		// but NTIA, CISA, BSI are nil so they're skipped
+		if len(violations) != 0 {
+			t.Errorf("expected 0 violations (nil frameworks are skipped), got %d", len(violations))
+		}
+	})


@jfjrh2014 plz review

rezmoss requested a review from Copilot June 4, 2026 17:28

Copilot started reviewing on behalf of rezmoss June 4, 2026 17:28 View session

Copilot AI reviewed Jun 4, 2026

View reviewed changes

rezmoss requested a review from Copilot June 5, 2026 21:02

Copilot started reviewing on behalf of rezmoss June 5, 2026 21:03 View session

Copilot AI reviewed Jun 5, 2026

View reviewed changes

Conversation

jfjrh2014 commented Jun 3, 2026

What

Changes

New package: internal/compliance

Policy integration (internal/policy)

Output format integration

CLI

Testing

Example

Files Changed

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rezmoss Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

New package: `internal/compliance`

Policy integration (`internal/policy`)

rezmoss Jun 4, 2026 •

edited

Loading