Skip to content

Conversation

@AdamKorcz
Copy link
Contributor

What kind of change does this PR introduce?

This adds support for Prow for the Code review check and the SAST check. It allows Scorecard to consider approvals in the format that Prow users leave as well as whether security SAST tools run in Prow - Scorecard already had limited support for this but does not extract the reviews to consider them in the CI check. This PR fixes that. The CI fix is fairly simple. The SAST fix has a bit more to it; It allows Scorecard to parse Prow config files and look for SAST tools. A limitation in this is that most projects don't place their Prow config files in the same repository. As such, this won't work with a lot of projects, however, I think we should leave it since there may be project with which it will work. In addition, it will be easy to add support for projects to specify where their Prow configuration file is. The PR also adds support for reading Prow CI logs and look for the invocation of security SAST tools. This requires Scorecard to first get the URL of the logs, download them and then scan them. The log scanning part could be optimized in the future, but it does a fair job for now. For example, when I run it on Kubernetes (see blow), I only see that it runs golangci which seems correct, judging from the the logs, its workflows and this old comment indicating that Kubernetes does not run the popular gosec.

This can be tested on Kubernetes:

Before this PR:

go run main.go --repo=github.com/kubernetes/kubernetes --checks=SAST --show-details --verbosity=debug
RESULTS
-------
Aggregate score: 0.0 / 10

Check scores:
|--------|------|--------------------------------------------------------------|--------------------------------------------------------|-----------------------------------------------------------------|
| SCORE  | NAME |                            REASON                            |                        DETAILS                         |                   DOCUMENTATION / REMEDIATION                   |
|--------|------|--------------------------------------------------------------|--------------------------------------------------------|-----------------------------------------------------------------|
| 0 / 10 | SAST | SAST tool is not run on all commits -- score normalized to 0 | Warn: 0 commits out of 30 are checked with a SAST tool | https://github.com/ossf/scorecard/blob/main/docs/checks.md#sast |
|--------|------|--------------------------------------------------------------|--------------------------------------------------------|-----------------------------------------------------------------|

After this PR:

go run main.go --repo=github.com/kubernetes/kubernetes --checks=SAST --show-details --verbosity=debug
RESULTS                                                                                                                                                                                                                                                                                                                                                                       
-------                                                                                                                                                                                                                                                                                                                                                                       
Aggregate score: 10.0 / 10                                                                                                                                                                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                                                                                                                              
Check scores:                                                                                                                                                                                                                                                                                                                                                                 
|---------|------|---------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------|
|  SCORE  | NAME |             REASON              |                                                                                       DETAILS                                                                                       |                   DOCUMENTATION / REMEDIATION                   |
|---------|------|---------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------|
| 10 / 10 | SAST | SAST tool is run on all commits | Debug: Starting SAST check - looking for static analysis tools...                                                                                                                   | https://github.com/ossf/scorecard/blob/main/docs/checks.md#sast |
|         |      |                                 | Debug: [CheckRuns] Analyzing 30 commit(s) for SAST tools                                                                                                                            |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135330/pull-kubernetes-verify/2005573653218988032       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135912/pull-kubernetes-verify/2005556029835710464       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135954/pull-kubernetes-verify/2005331042004635648       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/134350/pull-kubernetes-verify/2005058754957021184       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135954/pull-kubernetes-verify/2005331042004635648       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135946/pull-kubernetes-verify/2004672198282842112       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135933/pull-kubernetes-verify/2004634951894437888       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135947/pull-kubernetes-linter-hints/2004600212831604736 |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135947/pull-kubernetes-verify/2004600212139544576       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135947/pull-kubernetes-linter-hints/2004600212831604736 |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135947/pull-kubernetes-verify/2004600212139544576       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135938/pull-kubernetes-linter-hints/2004487936187305984 |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135938/pull-kubernetes-verify/2004487935398776832       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135946/pull-kubernetes-verify/2004672198282842112       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135938/pull-kubernetes-linter-hints/2004487936187305984 |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135938/pull-kubernetes-verify/2004487935398776832       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135933/pull-kubernetes-verify/2004634951894437888       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135918/pull-kubernetes-linter-hints/2003715284485017600 |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135918/pull-kubernetes-verify/2003715283960729600       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135918/pull-kubernetes-linter-hints/2003715284485017600 |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135918/pull-kubernetes-verify/2003715283960729600       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135912/pull-kubernetes-verify/2005556029835710464       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135903/pull-kubernetes-linter-hints/2003444229807804416 |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135903/pull-kubernetes-verify/2003444228973137920       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135903/pull-kubernetes-verify/2003444228973137920       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135903/pull-kubernetes-linter-hints/2003444229807804416 |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135886/pull-kubernetes-verify/2003348989432303616       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135805/pull-kubernetes-verify/2003248322042662912       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135391/pull-kubernetes-linter-hints/1991867715425406976 |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135391/pull-kubernetes-verify/1991867714586546176       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135859/pull-kubernetes-linter-hints/2003159379993432064 |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135859/pull-kubernetes-linter-hints/2003159379993432064 |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135859/pull-kubernetes-verify/2003159379167154176       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135859/pull-kubernetes-verify/2003159379167154176       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135862/pull-kubernetes-linter-hints/2002930730329444352 |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135862/pull-kubernetes-verify/2002930729570275328       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135501/pull-kubernetes-verify/2003135576772972544       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135886/pull-kubernetes-verify/2003348989432303616       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135807/pull-kubernetes-verify/2002931727290339328       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135869/pull-kubernetes-verify/2002743487660822528       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135829/pull-kubernetes-linter-hints/2002729177307942912 |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135829/pull-kubernetes-verify/2002729176473276416       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135829/pull-kubernetes-verify/2002729176473276416       |                                                                 |
|         |      |                                 | Debug: [Prow Logs] SAST tool 'golangci' detected in job output: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/135829/pull-kubernetes-linter-hints/2002729177307942912 |                                                                 |
|         |      |                                 | Debug: [Prow] Scanning for Prow configuration files...                                                                                                                              |                                                                 |
|         |      |                                 | Debug: [Prow] Scanning pattern: .prow.yaml                                                                                                                                          |                                                                 |
|         |      |                                 | Debug: [Prow] Scanning pattern: .prow/*.yaml                                                                                                                                        |                                                                 |
|         |      |                                 | Debug: [Prow] Scanning pattern: prow/*.yaml                                                                                                                                         |                                                                 |                                                                  
|         |      |                                 | Debug: [Prow] No Prow config files with SAST tools found                                                                                                                            |                                                                 |                                                                  
|         |      |                                 | Debug: SAST check complete: found 0 workflow(s), analyzed 30 commit(s)                                                                                                              |                                                                 |                                                                  
|         |      |                                 | Info: all commits (30) are checked with a SAST tool                                                                                                                                 |                                                                 |                                                                  
|---------|------|---------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------| 

What is the current behavior?

What is the new behavior (if this is a feature change)?**

  • Tests for the changes have been added (for bug fixes/features)

Which issue(s) this PR fixes

Special notes for your reviewer

Does this PR introduce a user-facing change?

For user-facing changes, please add a concise, human-readable release note to
the release-note

(In particular, describe what changes users might need to make in their
application as a result of this pull request.)


Signed-off-by: Adam Korczynski <adam@adalogics.com>
@AdamKorcz AdamKorcz requested a review from a team as a code owner December 29, 2025 19:11
@AdamKorcz AdamKorcz removed the request for review from a team December 29, 2025 19:11
@dosubot dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Dec 29, 2025
@codecov
Copy link

codecov bot commented Dec 29, 2025

Codecov Report

❌ Patch coverage is 32.60870% with 217 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.49%. Comparing base (353ed60) to head (c160fbe).
⚠️ Report is 303 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4897      +/-   ##
==========================================
+ Coverage   66.80%   67.49%   +0.68%     
==========================================
  Files         230      252      +22     
  Lines       16602    15968     -634     
==========================================
- Hits        11091    10777     -314     
+ Misses       4808     4325     -483     
- Partials      703      866     +163     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

… logs

Signed-off-by: Adam Korczynski <adam@adalogics.com>
Signed-off-by: Adam Korczynski <adam@adalogics.com>
Signed-off-by: Adam Korczynski <adam@adalogics.com>
@spencerschrock
Copy link
Member

but does not extract the reviews to consider them in the CI check

I assume you mean Code-Review?

This requires Scorecard to first get the URL of the logs, download them and then scan them

Do we need to worry about rate limiting? How widely used is Prow, is this limited to certain LF projects? Only K8s?

I haven't looked at implementation yet, but is there any chance of splitting these into their components? Or are the changes largely intertwined?

@github-actions
Copy link

This pull request has been marked stale because it has been open for 10 days with no activity

@github-actions github-actions bot added the Stale label Jan 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL This PR changes 1000+ lines, ignoring generated files. Stale

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants