Skip to content

Create SBOMs and scan for vulnerablities #20801

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 8, 2025
Merged

Create SBOMs and scan for vulnerablities #20801

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
5eb2202
Add sbom.enabled=true
corneliusludmann May 6, 2025
a6440b4
Replace trivy scan in build.yml
corneliusludmann May 6, 2025
4ecf00e
Only scan relevent packages
corneliusludmann May 7, 2025
1041fda
Add vulnerability ignore rules
corneliusludmann May 7, 2025
e24b507
Add a daily vulnerability scan
corneliusludmann May 8, 2025
50e44e8
Update dev image
corneliusludmann May 8, 2025
9cb9b9c
Remove not needed env vars
corneliusludmann May 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actions/delete-preview/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32399
FROM eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32507

COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
2 changes: 1 addition & 1 deletion .github/actions/deploy-gitpod/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32399
FROM eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32507

COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
2 changes: 1 addition & 1 deletion .github/actions/deploy-monitoring-satellite/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32399
FROM eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32507

COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
2 changes: 1 addition & 1 deletion .github/actions/preview-create/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32399
FROM eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32507

COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
174 changes: 137 additions & 37 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,17 @@ on:
type: string
description: "Whether to recreate the VM"
default: "false"
simulate_scheduled_run:
required: false
type: boolean
description: "Simulate a scheduled run"
default: false
schedule:
# Run at midnight UTC every day
# Purpose: This scheduled run performs regular vulnerability scans of the codebase
# and sends notifications to Slack when new critical vulnerabilities are found.
# The scan results are used to maintain security standards and address issues promptly.
- cron: '0 0 * * *'

jobs:
create-runner:
Expand All @@ -36,6 +47,7 @@ jobs:
cancel-in-progress: true
outputs:
is_main_branch: ${{ (github.head_ref || github.ref) == 'refs/heads/main' }}
is_scheduled_run: ${{ github.event_name == 'schedule' || inputs.simulate_scheduled_run == true }}
version: ${{ steps.branches.outputs.sanitized-branch-name }}-gha.${{github.run_number}}
preview_enable: ${{ contains( steps.pr-details.outputs.pr_body, '[x] /werft with-preview') || (steps.output.outputs.with_integration_tests != '') }}
preview_name: ${{ github.head_ref || github.ref_name }}
Expand Down Expand Up @@ -98,14 +110,15 @@ jobs:
name: Build previewctl
if: |
(needs.configuration.outputs.pr_no_diff_skip != 'true') &&
(needs.configuration.outputs.preview_enable == 'true')
(needs.configuration.outputs.preview_enable == 'true') &&
(needs.configuration.outputs.is_scheduled_run != 'true')
needs: [ configuration, create-runner ]
concurrency:
group: ${{ github.ref == 'refs/heads/main' && github.run_id || github.sha }}-build-previewctl
cancel-in-progress: ${{ needs.configuration.outputs.is_main_branch == 'false' }}
runs-on: ${{ needs.create-runner.outputs.label }}
container:
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32399
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32507
steps:
- uses: actions/checkout@v4
- name: Setup Environment
Expand All @@ -126,7 +139,8 @@ jobs:
if: |
(needs.configuration.outputs.pr_no_diff_skip != 'true') &&
(needs.configuration.outputs.preview_enable == 'true') &&
(needs.configuration.outputs.is_main_branch != 'true')
(needs.configuration.outputs.is_main_branch != 'true') &&
(needs.configuration.outputs.is_scheduled_run != 'true')
runs-on: ${{ needs.create-runner.outputs.label }}
concurrency:
group: ${{ github.ref == 'refs/heads/main' && github.run_id || github.sha }}-infrastructure
Expand Down Expand Up @@ -154,6 +168,8 @@ jobs:
name: Build Gitpod
needs: [ configuration, create-runner ]
runs-on: ${{ needs.create-runner.outputs.label }}
outputs:
affected_packages: ${{ steps.check_vulnerabilities.outputs.affected_packages }}
concurrency:
group: ${{ github.ref == 'refs/heads/main' && github.run_id || github.sha }}-build-gitpod
cancel-in-progress: ${{ needs.configuration.outputs.is_main_branch == 'false' }}
Expand All @@ -172,7 +188,7 @@ jobs:
ports:
- 6379:6379
container:
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32399
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32507
env:
DB_HOST: "mysql"
DB_PORT: "23306"
Expand Down Expand Up @@ -299,7 +315,7 @@ jobs:

exit $RESULT
- name: Tag the release
if: github.ref == 'refs/heads/main'
if: github.ref == 'refs/heads/main' && needs.configuration.outputs.is_scheduled_run != 'true'
run: |
git config --global user.name $GITHUB_USER
git config --global user.email $GITHUB_EMAIL
Expand All @@ -309,36 +325,99 @@ jobs:
GITHUB_USER: roboquat
GITHUB_EMAIL: [email protected]
VERSION: ${{ needs.configuration.outputs.version }}

trivy-scan:
name: "Scan Images for Vulnerabilities"
needs:
- configuration
- build-gitpod
- create-runner
runs-on: ${{ needs.create-runner.outputs.label }}
container:
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32399
steps:
- uses: actions/checkout@v4
- name: Setup Environment
uses: ./.github/actions/setup-environment
with:
identity_provider: ${{ github.ref == 'refs/heads/main' && secrets.CORE_DEV_PROVIDER || secrets.DEV_PREVIEW_PROVIDER }}
service_account: ${{ github.ref == 'refs/heads/main' && secrets.CORE_DEV_SA || secrets.DEV_PREVIEW_SA }}
leeway_segment_key: ${{ secrets.LEEWAY_SEGMENT_KEY }}
- name: Scan Images for Vulnerabilities
- name: Scan for Vulnerabilities
id: scan
shell: bash
env:
INSTALLER_IMAGE_BASE_REPO: ${{needs.configuration.outputs.image_repo_base}}
NODE_OPTIONS: "--max_old_space_size=4096"
JAVA_HOME: /home/gitpod/.sdkman/candidates/java/current
VERSION: ${{needs.configuration.outputs.version}}
PR_NO_CACHE: ${{needs.configuration.outputs.build_no_cache}}
PR_NO_TEST: ${{needs.configuration.outputs.build_no_test}}
NPM_AUTH_TOKEN: "${{ secrets.NPM_AUTH_TOKEN }}"
PUBLISH_TO_NPM: ${{ needs.configuration.outputs.publish_to_npm == 'true' || needs.configuration.outputs.is_main_branch == 'true' }}
JB_MARKETPLACE_PUBLISH_TOKEN: "${{ secrets.JB_MARKETPLACE_PUBLISH_TOKEN }}"
PUBLISH_TO_JBPM: ${{ needs.configuration.outputs.publish_to_jbmp == 'true' || needs.configuration.outputs.is_main_branch == 'true' }}
CODECOV_TOKEN: "${{ secrets.CODECOV_TOKEN }}"
LEEWAY_REMOTE_CACHE_BUCKET: ${{needs.configuration.outputs.leeway_cache_bucket}}
IMAGE_REPO_BASE: ${{needs.configuration.outputs.image_repo_base}}/build

# SCM tokens for integration tests
GITPOD_TEST_TOKEN_BITBUCKET: "${{ secrets.GITPOD_TEST_TOKEN_BITBUCKET }}"
GITPOD_TEST_TOKEN_BITBUCKET_SERVER: "${{ secrets.GITPOD_TEST_TOKEN_BITBUCKET_SERVER }}"
GITPOD_TEST_TOKEN_BITBUCKET_SERVER_WRITE: "${{ secrets.GITPOD_TEST_TOKEN_BITBUCKET_SERVER_WRITE }}"
GITPOD_TEST_TOKEN_BITBUCKET_SERVER_READ: "${{ secrets.GITPOD_TEST_TOKEN_BITBUCKET_SERVER_READ }}"
Copy link
Member

@geropl geropl May 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@corneliusludmann Do we really need all of these env vars? 🤔

Generally, I think producing the SBOMs should be part of the build itself - and only the scan separate - where we would not need all of these, because we effectively pull the required SBOMs from all the packages..?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would also allow us to have the "scan" as separate workflow that we trigger syncronously - and get rid of the invasive && needs.configuration.outputs.is_scheduled_run != 'true' checks... 🫧

Copy link
Contributor Author

@corneliusludmann corneliusludmann May 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, I think producing the SBOMs should be part of the build itself - and only the scan separate - where we would not need all of these, because we effectively pull the required SBOMs from all the packages..?

In general, I agree. Actually, we generate the SBOMs during build already. However, when we want to pull the cached packages Leeway calculates the internal version (aka hash) of the packages based on the package manifest, which also includes the value of envs. When the envs are not the same, Leeway would rebuild the packages.

To be honest, I don't know which variables exactly are considered. Do you think it's worth to try it out and find the right set of variables needed for it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the envs that are not needed as discussed and it works. ✔️

GITPOD_TEST_TOKEN_GITHUB: "${{ secrets.GITPOD_TEST_TOKEN_GITHUB }}"
GITPOD_TEST_TOKEN_GITLAB: "${{ secrets.GITPOD_TEST_TOKEN_GITLAB }}"
GITPOD_TEST_TOKEN_AZURE_DEVOPS: "${{ secrets.GITPOD_TEST_TOKEN_AZURE_DEVOPS }}"
run: |
./scripts/trivy/trivy-scan-images.sh ${{ needs.configuration.outputs.version }} CRITICAL
exit $?
[[ "$PR_NO_CACHE" = "true" ]] && CACHE="none" || CACHE="remote"
[[ "$PR_NO_TEST" = "true" ]] && TEST="--dont-test" || TEST=""
[[ "${PUBLISH_TO_NPM}" = 'true' ]] && NPM_PUBLISH_TRIGGER=$(date +%s%3N) || NPM_PUBLISH_TRIGGER="false"

sboms_dir=$(mktemp -d)
CI= leeway sbom export components:needs-vuln-scan --with-dependencies --output-dir "$sboms_dir" \
-Dversion=$VERSION \
--docker-build-options network=host \
--max-concurrent-tasks 1 \
-DlocalAppVersion=$VERSION \
-DpublishToNPM="${PUBLISH_TO_NPM}" \
-DnpmPublishTrigger="${NPM_PUBLISH_TRIGGER}" \
-DpublishToJBMarketplace="${PUBLISH_TO_JBPM}" \
-DimageRepoBase=$IMAGE_REPO_BASE

scans_dir=$(mktemp -d)
CI= leeway sbom scan components:needs-vuln-scan --with-dependencies --output-dir "$scans_dir" \
-Dversion=$VERSION \
--docker-build-options network=host \
--max-concurrent-tasks 1 \
-DlocalAppVersion=$VERSION \
-DpublishToNPM="${PUBLISH_TO_NPM}" \
-DnpmPublishTrigger="${NPM_PUBLISH_TRIGGER}" \
-DpublishToJBMarketplace="${PUBLISH_TO_JBPM}" \
-DimageRepoBase=$IMAGE_REPO_BASE || RESULT=$?

{
echo "leeway_sboms_dir=$sboms_dir"
echo "leeway_vulnerability_reports_dir=$scans_dir"
} >> $GITHUB_OUTPUT

cat "$scans_dir/vulnerability-summary.md" >> $GITHUB_STEP_SUMMARY

exit $RESULT
- name: Check for Critical Vulnerabilities
if: needs.configuration.outputs.is_scheduled_run == 'true'
id: check_vulnerabilities
shell: bash
run: |
# Parse vulnerability-stats.json from the scans directory
CRITICAL_PACKAGES=$(jq -r '.[] | select(.critical > 0) | "\(.name): \(.critical) critical vulnerabilities"' "${{ steps.scan.outputs.leeway_vulnerability_reports_dir }}/vulnerability-stats.json")

# If there are critical packages, list them and fail the build
if [ -n "$CRITICAL_PACKAGES" ]; then
echo "::error::Critical vulnerabilities found in the following packages:"
echo "$CRITICAL_PACKAGES" | tee -a $GITHUB_STEP_SUMMARY
echo "affected_packages<<EOF" >> $GITHUB_OUTPUT
echo "$CRITICAL_PACKAGES" >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
exit 1
else
echo "No critical vulnerabilities found."
fi
- name: Upload SBOMs
uses: actions/upload-artifact@v4
with:
name: sboms
path: ${{ steps.scan.outputs.leeway_sboms_dir }}
- name: Upload vulnerability reports
uses: actions/upload-artifact@v4
with:
name: vulnerability-reports
path: ${{ steps.scan.outputs.leeway_vulnerability_reports_dir }}
install-app:
runs-on: ${{ needs.create-runner.outputs.label }}
needs: [ configuration, build-gitpod, trivy-scan, create-runner ]
if: ${{ needs.configuration.outputs.is_main_branch == 'true' }}
needs: [ configuration, build-gitpod, create-runner ]
if: ${{ needs.configuration.outputs.is_main_branch == 'true' && needs.configuration.outputs.is_scheduled_run != 'true' }}
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -375,9 +454,9 @@ jobs:
- configuration
- build-previewctl
- build-gitpod
- trivy-scan
- infrastructure
- create-runner
if: needs.configuration.outputs.is_scheduled_run != 'true'
runs-on: ${{ needs.create-runner.outputs.label }}
concurrency:
group: ${{ github.ref == 'refs/heads/main' && github.run_id || github.sha }}-install
Expand Down Expand Up @@ -428,7 +507,7 @@ jobs:
name: "Install Monitoring Satellite"
needs: [ infrastructure, build-previewctl, create-runner ]
runs-on: ${{ needs.create-runner.outputs.label }}
if: needs.configuration.outputs.with_monitoring == 'true'
if: needs.configuration.outputs.with_monitoring == 'true' && needs.configuration.outputs.is_scheduled_run != 'true'
concurrency:
group: ${{ github.ref == 'refs/heads/main' && github.run_id || github.sha }}-monitoring
cancel-in-progress: true
Expand Down Expand Up @@ -458,8 +537,8 @@ jobs:
- create-runner
runs-on: ${{ needs.create-runner.outputs.label }}
container:
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32399
if: needs.configuration.outputs.with_integration_tests != ''
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32507
if: needs.configuration.outputs.with_integration_tests != '' && needs.configuration.outputs.is_scheduled_run != 'true'
concurrency:
group: ${{ needs.configuration.outputs.preview_name }}-integration-test
cancel-in-progress: true
Expand Down Expand Up @@ -489,7 +568,7 @@ jobs:
- configuration
- build-gitpod
- create-runner
if: needs.configuration.outputs.is_main_branch == 'true'
if: needs.configuration.outputs.is_main_branch == 'true' && needs.configuration.outputs.is_scheduled_run != 'true'
uses: ./.github/workflows/workspace-integration-tests.yml
with:
version: ${{ needs.configuration.outputs.version }}
Expand All @@ -501,7 +580,7 @@ jobs:
- configuration
- build-gitpod
- create-runner
if: needs.configuration.outputs.is_main_branch == 'true'
if: needs.configuration.outputs.is_main_branch == 'true' && needs.configuration.outputs.is_scheduled_run != 'true'
uses: ./.github/workflows/code-updates.yml
secrets: inherit

Expand All @@ -511,10 +590,31 @@ jobs:
- configuration
- build-gitpod
- create-runner
if: needs.configuration.outputs.is_main_branch == 'true'
if: needs.configuration.outputs.is_main_branch == 'true' && needs.configuration.outputs.is_scheduled_run != 'true'
uses: ./.github/workflows/jetbrains-updates.yml
secrets: inherit

notify-scheduled-failure:
name: "Notify on scheduled run failure"
if: needs.configuration.outputs.is_scheduled_run == 'true' && failure()
needs:
- configuration
- build-gitpod
- workspace-integration-tests-main
- ide-code-updates
- ide-jb-updates
runs-on: ubuntu-latest
steps:
- name: Slack Notification
uses: rtCamp/action-slack-notify@v2
env:
SLACK_WEBHOOK: ${{ secrets.WORKSPACE_SLACK_WEBHOOK }}
SLACK_ICON_EMOJI: ":x:"
SLACK_USERNAME: "Scheduled Build"
SLACK_COLOR: "danger"
SLACK_MESSAGE: "⚠️ Security Alert: Daily vulnerability scan detected critical vulnerabilities in the following packages:\n${{ needs.build-gitpod.outputs.affected_packages }}"
SLACK_FOOTER: "<${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View Workflow Logs>"

delete-runner:
if: always()
needs:
Expand All @@ -523,11 +623,11 @@ jobs:
- build-previewctl
- infrastructure
- build-gitpod
- trivy-scan
- install-app
- install
- monitoring
- integration-test
- notify-scheduled-failure
uses: gitpod-io/gce-github-runner/.github/workflows/delete-vm.yml@main
secrets:
gcp_credentials: ${{ secrets.SELF_HOSTED_GITHUB_RUNNER_GCP_CREDENTIALS }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/code-nightly.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
runs-on: ${{ needs.create-runner.outputs.label }}
needs: [create-runner]
container:
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32399
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32507
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup-environment
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/ide-integration-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ jobs:
runs-on: ${{ needs.create-runner.outputs.label }}
needs: [create-runner]
container:
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32399
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32507
outputs:
name: ${{ steps.configuration.outputs.name }}
version: ${{ steps.configuration.outputs.version }}
Expand Down Expand Up @@ -131,7 +131,7 @@ jobs:
needs: [configuration, infrastructure, create-runner]
runs-on: ${{ needs.create-runner.outputs.label }}
container:
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32399
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32507
volumes:
- /var/tmp:/var/tmp
- /tmp:/tmp
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/jetbrains-auto-update-template.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
update-jetbrains:
runs-on: ${{ needs.create-runner.outputs.label }}
container:
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32399
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32507
needs: [ create-runner ]
steps:
- uses: actions/checkout@v2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/jetbrains-integration-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ jobs:
gcp_credentials: ${{ secrets.SELF_HOSTED_GITHUB_RUNNER_GCP_CREDENTIALS }}
jetbrains-smoke-test-linux:
container:
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32399
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32507
runs-on: ${{ needs.create-runner.outputs.label }}
needs: [create-runner]
steps:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/preview-env-check-regressions.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ jobs:
if: ${{ needs.configuration.outputs.skip == 'false' }}
runs-on: ${{ needs.create-runner.outputs.label }}
container:
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32399
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32507
volumes:
- /var/tmp:/var/tmp
- /tmp:/tmp
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/preview-env-gc.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
runs-on: ${{ needs.create-runner.outputs.label }}
needs: [create-runner]
container:
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32399
image: eu.gcr.io/gitpod-core-dev/dev/dev-environment:main-gha.32507
outputs:
names: ${{ steps.set-matrix.outputs.names }}
count: ${{ steps.set-matrix.outputs.count }}
Expand Down
Loading
Loading