Skip to content

fix: include streamed artifacts in SBOM output for --stream scans#672

Merged
mldangelo merged 3 commits intopromptfoo:mainfrom
Daketey:main
Mar 10, 2026
Merged

fix: include streamed artifacts in SBOM output for --stream scans#672
mldangelo merged 3 commits intopromptfoo:mainfrom
Daketey:main

Conversation

@Daketey
Copy link
Contributor

@Daketey Daketey commented Mar 10, 2026

Summary

Fixed issue #671 where scan --stream --sbom did not include streamed model artifacts as SBOM components.

User impact:

Before: streaming scans could emit an SBOM with only a top-level/source component.
After: streamed artifact paths are captured and included, so SBOM output correctly lists model files/components discovered during streaming.
Also added a test in test_cli.py to verify streamed assets appear in SBOM output, and fixed line-length formatting in that new test.

Validation

  • uv run ruff format --check modelaudit/ tests/
  • uv run ruff check modelaudit/ tests/
  • uv run mypy modelaudit/
  • uv run pytest -n auto -m "not slow and not integration" --maxfail=1

Checklist

  • I followed the security-first guidelines in AGENTS.md.
  • I did not weaken detection behavior.
  • I added/updated tests when behavior changed.
  • I updated docs where needed.

Summary by CodeRabbit

  • Bug Fixes

    • SBOM generation for streaming scans now reliably captures all scanned components (including cases with no reported asset paths), improving SBOM completeness for HuggingFace models, cloud storage, and other streamed sources.
  • Tests

    • Added a regression test to verify SBOMs produced during streaming scans include all expected components for HuggingFace streaming workflows.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 10, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: ecc98be2-e7a6-4c0d-aca8-1e64f97c1d6d

📥 Commits

Reviewing files that changed from the base of the PR and between 0f4d4a0 and 77661a4.

📒 Files selected for processing (2)
  • modelaudit/cli.py
  • tests/test_cli.py

Walkthrough

Adds a nested helper _track_streaming_paths_for_sbom inside scan_command to collect concrete artifact paths from streaming scan results into scanned_paths for SBOM generation; the helper is invoked after multiple streaming scan flows. Adds a regression test to verify SBOM includes all streaming components.

Changes

Cohort / File(s) Summary
SBOM Streaming Path Tracking
modelaudit/cli.py
Adds nested helper _track_streaming_paths_for_sbom(streaming_result, fallback_path) and calls to it after HuggingFace, cloud storage, and PyTorch Hub streaming scans to ensure artifact paths are recorded for SBOM generation.
Regression Test (duplicate added)
tests/test_cli.py
Adds test_scan_huggingface_streaming_sbom_contains_all_components (appears duplicated) which mocks streaming download/scan, produces streamed assets, runs cli scan --stream --sbom, asserts exit code 0, and verifies SBOM components include all streamed filenames.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐇 I hopped through streams both near and far,
Collected each artifact, each little star.
Paths now tracked, the SBOM sings,
Components counted, joy it brings.
A rabbit's nod to tidy things.

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title accurately and concisely summarizes the main change: fixing SBOM output to include streamed artifacts when using --stream scans.
Docstring Coverage ✅ Passed Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@modelaudit/cli.py`:
- Around line 955-965: The helper _track_streaming_paths_for_sbom collects
streamed artifact paths but is only used for HuggingFace/local-directory; update
the PyTorch Hub and cloud-storage code paths that call final_scan_and_delete (or
otherwise merge streaming_result) to also invoke
_track_streaming_paths_for_sbom(streaming_result, fallback_path) right after you
obtain the streaming_result and before finalizing/deleting, so scanned_paths is
populated for SBOM; ensure scanned_paths is in scope (or passed/returned) where
you add the call and keep the same fallback_path behavior.

In `@tests/test_cli.py`:
- Around line 781-783: The test function
test_scan_huggingface_streaming_sbom_contains_all_components is missing type
hints; update its signature to include a return type of -> None, annotate
tmp_path as tmp_path: Path, and type the mocked fixtures (e.g.,
mock_scan_streaming, mock_download_streaming, mock_is_hf_url) as Mock or Any
(from typing or unittest.mock) so the signature looks like: def
test_scan_huggingface_streaming_sbom_contains_all_components(mock_scan_streaming:
Mock, mock_download_streaming: Mock, mock_is_hf_url: Mock, tmp_path: Path) ->
None; ensure necessary imports (Path, Mock/Any) are present at the top of the
test file.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 3c6e1521-c6cd-4e91-87b2-bdd774910997

📥 Commits

Reviewing files that changed from the base of the PR and between 9d77d50 and 0f4d4a0.

📒 Files selected for processing (2)
  • modelaudit/cli.py
  • tests/test_cli.py

@mldangelo
Copy link
Member

Thank you @Daketey! Please reach out if you'd like some promptfoo swag!

@mldangelo mldangelo merged commit 48d8d54 into promptfoo:main Mar 10, 2026
25 checks passed
mldangelo added a commit that referenced this pull request Mar 10, 2026
@Daketey
Copy link
Contributor Author

Daketey commented Mar 10, 2026

@mldangelo Thanks for the Merge. Let me know the best way to reach out to you!

@mldangelo
Copy link
Member

@mldangelo Thanks for the Merge. Let me know the best way to reach out to you!

email me at michael @ promptfoo.dev

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants