build: migrate tarball integrity checksums from sha1 to sha512 #170
Workflow file for this run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Build ONIE (kvm_x86_64) | |
| # Builds the generic amd64 ONIE image for the kvm_x86_64 target, intended | |
| # to run as a VM under KVM/QEMU. The build follows the upstream-recommended | |
| # containerized flow (see contrib/build-env and machine/kvm_x86_64/INSTALL), | |
| # with secure boot enabled: the workflow generates demonstration signing | |
| # keys and self-signs the shim before building. | |
| # Run on every push and pull request: the point of this workflow is to | |
| # confirm the change still builds, and the kvm_x86_64 image depends on far | |
| # more than the build files (installer/, rootconf/, patches/, ...), so path | |
| # filtering would risk skipping validation on build-affecting changes. | |
| on: | |
| workflow_dispatch: | |
| inputs: | |
| oce_scope: | |
| description: >- | |
| OCE compliance test scope: "default" (a fast representative | |
| subset, also what push/PR runs) or "full" (the entire OCE | |
| installer+updater sweep, tests 3-121 -- slow, many VM boots). | |
| type: choice | |
| default: default | |
| options: | |
| - default | |
| - full | |
| push: | |
| pull_request: | |
| # Cancel an in-progress run when a newer commit is pushed to the same ref, | |
| # so stacked pushes don't pile up concurrent builds. | |
| concurrency: | |
| group: ${{ github.workflow }}-${{ github.ref }} | |
| cancel-in-progress: true | |
| jobs: | |
| build: | |
| name: Build kvm_x86_64 | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Checkout | |
| uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | |
| # The GCC 14 / Debian 13 toolchain build plus the full image (restored | |
| # x-tools cache + every package + kernel + recovery ISO) exceeds the | |
| # ~14 GB free on a stock ubuntu-latest runner. Reclaim ~25 GB of | |
| # preinstalled toolchains we do not use so the build has headroom. | |
| - name: Free up runner disk space | |
| run: | | |
| echo "Disk before cleanup:"; df -h / | |
| sudo rm -rf /usr/local/lib/android /opt/hostedtoolcache /usr/share/dotnet \ | |
| /usr/local/.ghcup /usr/share/swift /usr/local/share/powershell \ | |
| /usr/local/lib/node_modules || true | |
| echo "Disk after cleanup:"; df -h / | |
| - name: Set up Docker Buildx | |
| uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0 | |
| - name: Determine host UID/GID | |
| id: ids | |
| run: | | |
| echo "uid=$(id -u)" >> "$GITHUB_OUTPUT" | |
| echo "gid=$(id -g)" >> "$GITHUB_OUTPUT" | |
| - name: Build the ONIE build-environment image | |
| uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0 | |
| with: | |
| context: .github/onie-build | |
| tags: onie-build-env:latest | |
| build-args: | | |
| UID=${{ steps.ids.outputs.uid }} | |
| GID=${{ steps.ids.outputs.gid }} | |
| load: true | |
| cache-from: type=gha | |
| cache-to: type=gha,mode=max | |
| # Cache the crosstool-NG cross toolchain -- the long pole of a cold | |
| # build. We cache the whole stamp chain (so make does not rebuild on a | |
| # missing root stamp) but only the parts the warm, no-op path actually | |
| # needs -- NOT the multi-GB crosstool-NG build scratch or ct-ng source | |
| # tree, which are only used while building the toolchain. On a cache | |
| # hit "make xtools" is a no-op and the real build only uses the | |
| # installed toolchain, so dropping the scratch keeps the cache blob | |
| # small and fast/reliable to restore. | |
| # | |
| # build/stamp-project -- top-of-chain project stamp | |
| # build/download -- toolchain tarballs + download stamps | |
| # build/crosstool-ng/stamp -- crosstool-NG build stamps | |
| # build/x-tools/*/stamp -- xtools build stamps | |
| # build/x-tools/*/install -- the installed cross toolchain | |
| # build/x-tools/*/build/.config -- xtools config (an xtools-build prereq) | |
| # | |
| # The key is tied to every input that can change the toolchain -- | |
| # including the Dockerfile, since it defines the compiler/host | |
| # environment the toolchain is built in. No loose restore-keys: any | |
| # input change forces a clean rebuild rather than restoring a | |
| # mismatched toolchain. | |
| - name: Restore cross-toolchain cache | |
| id: xtools-cache | |
| uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 | |
| with: | |
| path: | | |
| build/stamp-project | |
| build/download | |
| build/crosstool-ng/stamp | |
| build/x-tools/*/stamp | |
| build/x-tools/*/install | |
| build/x-tools/*/build/.config | |
| key: onie-xtools-kvm_x86_64-${{ hashFiles('build-config/make/xtools.make', 'build-config/make/crosstool-ng.make', 'build-config/make/compiler.make', 'build-config/conf/crosstool/**', 'patches/crosstool-NG/**', '.github/onie-build/Dockerfile') }} | |
| # ONIE drives its build with stamp files compared by mtime. A fresh | |
| # checkout gives every repo source file a current mtime, which is newer | |
| # than the restored stamps -- so make would consider the toolchain | |
| # stale and rebuild it (e.g. build/.../.config depends on the | |
| # checked-out conf/crosstool config). A cache hit means the key | |
| # matched, i.e. every toolchain input is unchanged and the restored | |
| # toolchain is valid, so bump the restored artifacts ahead of the | |
| # checkout to make "make xtools" a genuine no-op. | |
| - name: Mark restored toolchain up to date | |
| if: steps.xtools-cache.outputs.cache-hit == 'true' | |
| # Restoring build/stamp-project makes make skip the project-stamp | |
| # recipe, which is what normally creates the project output dirs -- | |
| # so recreate them (build/images is an output, not part of the | |
| # cache, and the kernel install copies vmlinuz into it). | |
| # | |
| # Then touch only the files make compares -- the stamp files and the | |
| # toolchain .config. Do NOT touch the toolchain tree itself: it | |
| # contains read-only binaries and symlinks to host tools (bash, | |
| # make, ...) that touch would follow and fail to update. | |
| # | |
| # Touch them all to ONE identical timestamp (-t), not "now" per file. | |
| # The xtools build stamp depends on .config (xtools.make), and a bare | |
| # `find -exec touch {} +` batches files across calls that can straddle a | |
| # one-second boundary -- leaving .config (touched last) newer than the | |
| # build stamp. make then re-runs `ct-ng build`, which fails because the | |
| # ct-ng binary is intentionally not in the cache. Equal mtimes mean no | |
| # cached target is ever "newer-than" a cached prerequisite, so `make | |
| # xtools` is a reliable no-op; the timestamp is "now" (after checkout), | |
| # so it's also newer than the just-checked-out toolchain config sources. | |
| run: | | |
| mkdir -p build build/images build/download | |
| ts="$(date +%Y%m%d%H%M.%S)" | |
| find build/stamp-project build/download \ | |
| build/crosstool-ng/stamp \ | |
| build/x-tools/*/stamp \ | |
| build/x-tools/*/build/.config \ | |
| -type f -exec touch -t "$ts" {} + | |
| - name: Build cross toolchain | |
| run: | | |
| docker run --rm --privileged \ | |
| -v "${PWD}:/onie" \ | |
| onie-build-env \ | |
| bash -lc 'cd build-config && make -j"$(nproc)" \ | |
| MACHINE=kvm_x86_64 \ | |
| xtools' | |
| # Save only on a cache miss, and only after the toolchain build above | |
| # succeeded (default if: success()), so a broken toolchain is never | |
| # cached. Running before the full build means a later-stage failure | |
| # still preserves the toolchain for the next run. | |
| - name: Save cross-toolchain cache | |
| if: steps.xtools-cache.outputs.cache-hit != 'true' | |
| uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 | |
| with: | |
| path: | | |
| build/stamp-project | |
| build/download | |
| build/crosstool-ng/stamp | |
| build/x-tools/*/stamp | |
| build/x-tools/*/install | |
| build/x-tools/*/build/.config | |
| key: ${{ steps.xtools-cache.outputs.cache-primary-key }} | |
| - name: Build ONIE | |
| run: | | |
| docker run --rm --privileged \ | |
| -v "${PWD}:/onie" \ | |
| onie-build-env \ | |
| bash -lc 'cd build-config && \ | |
| make MACHINE=kvm_x86_64 signing-keys-generate && \ | |
| make MACHINE=kvm_x86_64 shim-self-sign && \ | |
| make -j"$(nproc)" MACHINE=kvm_x86_64 all recovery-iso demo' | |
| - name: Upload kvm recovery image | |
| uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 | |
| with: | |
| name: onie-kvm-recovery-iso | |
| path: build/images/onie-recovery-x86_64-kvm_x86_64-r0.iso | |
| retention-days: 7 | |
| # The demonstration PK/KEK/db that signed this build's shim/grub/kernel. | |
| # The boot-test job enrolls them into an OVMF varstore to validate the | |
| # secure-boot signature chain. These are the SAME keys this run signed | |
| # with, so the enrolled db matches the shipped image (regenerating would | |
| # not). Demo keys only -- non-sensitive (see machine-security.make). | |
| - name: Upload secure-boot demo keys | |
| uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 | |
| with: | |
| name: onie-kvm-sb-keys | |
| # Exclude the gpg-agent sockets that signing-keys-generate leaves in | |
| # the gpg-keys dirs -- upload-artifact can't zip sockets and warns | |
| # ENTRYNOTSUPPORTED; only the .pem/.der key material is needed. | |
| path: | | |
| encryption/machines/kvm_x86_64/keys | |
| !encryption/machines/kvm_x86_64/keys/**/S.gpg-agent* | |
| retention-days: 7 | |
| # Artifacts for the install-test job: the recovery kernel/initrd (booted | |
| # directly in install mode), the ONIE updater (to embed ONIE on a blank | |
| # disk first), and the demo OS installer (the NOS discovery installs). | |
| - name: Upload install-test images | |
| uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 | |
| with: | |
| name: onie-kvm-install-images | |
| path: | | |
| build/images/kvm_x86_64-r0.vmlinuz | |
| build/images/kvm_x86_64-r0.initrd | |
| build/images/onie-updater-x86_64-kvm_x86_64-r0 | |
| build/images/demo-installer-x86_64-kvm_x86_64-r0.bin | |
| retention-days: 7 | |
| # Boot the built recovery ISO headlessly under QEMU+OVMF and assert ONIE gets | |
| # past GRUB into the OS (serial-console milestones, no kernel panic). ONIE is | |
| # serial end-to-end (GRUB+kernel+userspace on ttyS0) so a serial capture | |
| # suffices -- no screen scraping/OCR. See emulation/ci-boot-test.sh. | |
| boot-test: | |
| name: Boot test kvm_x86_64 | |
| needs: build | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Checkout | |
| uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | |
| - name: Enable KVM (grant /dev/kvm access) | |
| run: | | |
| echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' \ | |
| | sudo tee /etc/udev/rules.d/99-kvm4all.rules | |
| sudo udevadm control --reload-rules | |
| sudo udevadm trigger --name-match=kvm | |
| - name: Install QEMU + OVMF | |
| run: | | |
| sudo apt-get update | |
| sudo apt-get install -y --no-install-recommends \ | |
| qemu-system-x86 qemu-utils ovmf python3-virt-firmware | |
| - name: Download kvm recovery image | |
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 | |
| with: | |
| name: onie-kvm-recovery-iso | |
| path: build/images | |
| - name: Download secure-boot demo keys | |
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 | |
| with: | |
| name: onie-kvm-sb-keys | |
| path: encryption/machines/kvm_x86_64/keys | |
| # SB-relaxed smoke: plain OVMF on the default 'pc' machine -- proves the | |
| # image boots past GRUB into the OS independent of the SB signature chain. | |
| - name: Boot ONIE (SB-relaxed) and assert it reaches the OS | |
| run: | | |
| SERIAL_LOG="$PWD/onie-boot-serial.log" BOOT_MODE=relaxed \ | |
| emulation/ci-boot-test.sh \ | |
| build/images/onie-recovery-x86_64-kvm_x86_64-r0.iso 300 | |
| # SB-enforced: enroll the demo PK/KEK/db, boot the secboot OVMF on | |
| # q35,smm=on, and assert the shim/grub/kernel chain verifies (plus a | |
| # negative control with db omitted that MUST be rejected). This is the | |
| # signal that catches a broken signing chain in the grub/shim/kernel | |
| # modernization PRs even when the image otherwise builds and boots. | |
| - name: Boot ONIE (SB-enforced) and assert the signing chain verifies | |
| run: | | |
| SERIAL_LOG="$PWD/onie-boot-serial-secureboot.log" BOOT_MODE=secureboot \ | |
| KEYS_DIR="$PWD/encryption/machines/kvm_x86_64/keys" \ | |
| emulation/ci-boot-test.sh \ | |
| build/images/onie-recovery-x86_64-kvm_x86_64-r0.iso 300 | |
| - name: Upload serial logs | |
| if: always() | |
| uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 | |
| with: | |
| name: onie-boot-serial-log | |
| path: onie-boot-serial*.log | |
| retention-days: 7 | |
| # Functional discovery-install: prove ONIE actually installs an OS and boots | |
| # it, not just that it reaches rescue. Embed ONIE onto a blank disk, then in | |
| # install mode point discovery (install_url=) at the locally-served demo | |
| # installer, install it, and boot the resulting demo OS -- all headless via | |
| # the serial console. This exercises the real installer path (download, | |
| # exec_installer, the NOS writing GRUB + the OS to disk, the OS booting) that | |
| # the boot-smoke jobs do not. Runs Secure-Boot-relaxed (legacy BIOS) -- the | |
| # signing chain is covered by boot-test. See emulation/ci-install-test.sh. | |
| install-test: | |
| name: Install test kvm_x86_64 | |
| # Chained after boot-test (not parallel) so only one VM-heavy job runs at a | |
| # time, keeping CI resource use low. | |
| needs: boot-test | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Checkout | |
| uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | |
| - name: Enable KVM (grant /dev/kvm access) | |
| run: | | |
| echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' \ | |
| | sudo tee /etc/udev/rules.d/99-kvm4all.rules | |
| sudo udevadm control --reload-rules | |
| sudo udevadm trigger --name-match=kvm | |
| - name: Install QEMU | |
| run: | | |
| sudo apt-get update | |
| sudo apt-get install -y --no-install-recommends qemu-system-x86 qemu-utils | |
| - name: Download install-test images | |
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 | |
| with: | |
| name: onie-kvm-install-images | |
| path: build/images | |
| - name: Install demo OS via discovery and boot it | |
| run: | | |
| SERIAL_PREFIX="$PWD/onie-install" \ | |
| emulation/ci-install-test.sh \ | |
| build/images/kvm_x86_64-r0.vmlinuz \ | |
| build/images/kvm_x86_64-r0.initrd \ | |
| build/images/demo-installer-x86_64-kvm_x86_64-r0.bin 600 | |
| - name: Upload install serial logs | |
| if: always() | |
| uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 | |
| with: | |
| name: onie-install-serial-log | |
| path: onie-install-*.log | |
| retention-days: 7 | |
| # Run ONIE's own OCE (ONIE Compliance Environment, contrib/oce) compliance | |
| # suite against the image. Where install-test hands ONIE the installer URL | |
| # directly (install_url=), OCE exercises the REAL OCP discovery path: it | |
| # stands up isc-dhcp-server advertising the ONIE VIVSO vendor option (DHCP | |
| # option 125) plus an nginx HTTP server, and ONIE must DHCP, read the VIVSO | |
| # installer URL, and fetch+install the image itself. OCE needs L2 to the DUT | |
| # (it validates the DUT shares the host interface's subnet), so the VM is on a | |
| # tap interface rather than user-mode SLIRP. contrib/oce is python3 (it was | |
| # ported in the python2-to-3 PR this is stacked on). See | |
| # emulation/ci-oce-test.sh. | |
| oce-test: | |
| name: OCE compliance kvm_x86_64 | |
| # Chained after install-test (not parallel) so only one VM-heavy job runs | |
| # at a time, keeping CI resource use low. | |
| needs: install-test | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Checkout | |
| uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | |
| - name: Enable KVM (grant /dev/kvm access) | |
| run: | | |
| echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' \ | |
| | sudo tee /etc/udev/rules.d/99-kvm4all.rules | |
| sudo udevadm control --reload-rules | |
| sudo udevadm trigger --name-match=kvm | |
| - name: Install QEMU + OCE service backends + python deps | |
| run: | | |
| sudo apt-get update | |
| sudo apt-get install -y --no-install-recommends \ | |
| qemu-system-x86 qemu-utils isc-dhcp-server nginx-light tftpd-hpa \ | |
| dnsmasq dosfstools python3-venv | |
| python3 -m venv /tmp/oce-venv | |
| /tmp/oce-venv/bin/pip install --quiet jinja2 netifaces psutil | |
| - name: Download install-test images | |
| uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 | |
| with: | |
| name: onie-kvm-install-images | |
| path: build/images | |
| - name: Run OCE compliance test (full install/update via OCP discovery) | |
| run: | | |
| # On push/PR (no dispatch input) run the fast "default" subset; a | |
| # human can manually dispatch this workflow with oce_scope=full to | |
| # run the entire OCE installer+updater sweep. | |
| SCOPE="${{ inputs.oce_scope }}" | |
| [ -z "$SCOPE" ] && SCOPE=default | |
| OCE_PYTHON=/tmp/oce-venv/bin/python SERIAL_PREFIX="$PWD/onie-oce" \ | |
| emulation/ci-oce-test.sh \ | |
| build/images/kvm_x86_64-r0.vmlinuz \ | |
| build/images/kvm_x86_64-r0.initrd \ | |
| build/images/demo-installer-x86_64-kvm_x86_64-r0.bin \ | |
| build/images/onie-updater-x86_64-kvm_x86_64-r0 \ | |
| "$SCOPE" | |
| - name: Upload OCE serial logs | |
| if: always() | |
| uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 | |
| with: | |
| name: onie-oce-serial-log | |
| path: onie-oce-*.log | |
| retention-days: 7 |