Skip to content

Commit 0943ac2

Browse files
bhouse-nexthopbhouse-nexthop
committed
ci: add GitHub Action to build the kvm_x86_64 ONIE image
Add a GitHub Actions workflow that builds the generic amd64 ONIE target (kvm_x86_64, intended to run as a VM under KVM/QEMU) on every push and pull request that touches the build. The intent is to make reviewing other PRs easier: a green check confirms the change still produces a working ONIE image, so "it builds" becomes part of acceptance testing rather than something a reviewer has to verify by hand. The build follows the upstream-recommended containerized flow (contrib/build-env and machine/kvm_x86_64/INSTALL) rather than any bespoke scripting: - .github/onie-build/Dockerfile provides a Debian 11 build environment. Debian 11 is required because build-config still pulls in python2-era tooling. The package set is derived from the legacy toolchain ONIE needs (autoconf/automake, bison/flex, gperf, stgit for the patch series, libefivar-dev for the UEFI path, xorriso, etc.); secure-boot/code-signing tooling is intentionally omitted. The build runs as a "build" user whose UID/GID match the host runner so artifacts are not left root-owned. - The workflow disables secure boot (no signing keys to manage) by swapping in the repo's shipped machine/kvm_x86_64/kernel/ config-insecure and passing SECURE_BOOT_ENABLE=no SECURE_BOOT_EXT=no SECURE_GRUB=no, then runs the standard "make MACHINE=kvm_x86_64 all recovery-iso". Caching keeps the per-PR runtime low enough to run on every PR: - The build-environment image is built with docker/build-push-action and GitHub Actions layer caching, so the apt install layer is reused across runs. - The crosstool-NG cross toolchain is the long pole of a cold build (~25 min). It is built as a discrete "xtools" make target and cached. Because ONIE drives its build with mtime-compared stamp files, the entire stamp chain is cached -- build/stamp-project, build/download, build/crosstool-ng and build/x-tools -- not just the final toolchain; a missing download or project stamp would otherwise force a full rebuild. On a cache hit the restored artifacts are touched ahead of the freshly checked-out source so "make xtools" is a genuine no-op (the cache key already encodes every toolchain input, so a hit means the toolchain is valid). The key hashes those inputs -- xtools.make, crosstool-ng.make, compiler.make, conf/crosstool, the crosstool-NG patches, and the Dockerfile (it defines the environment the toolchain is built in). No loose restore-keys, so any input change forces a clean rebuild rather than restoring a mismatched toolchain. The cache is saved only after the toolchain build succeeds and before the full build, so a broken toolchain is never cached and a later-stage failure still preserves the toolchain. Other hygiene: - apt is configured with Acquire::Retries so transient Debian mirror resets do not fail the build. - A concurrency group (workflow + ref, cancel-in-progress) ensures a newer push to the same branch cancels the prior run instead of stacking concurrent builds. Signed-off-by: Brad House <bhouse@nexthop.ai>
1 parent 48db661 commit 0943ac2

2 files changed

Lines changed: 233 additions & 0 deletions

File tree

‎.github/onie-build/Dockerfile‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# ONIE build environment for CI.
2+
#
3+
# Debian 11 is required: onie/build-config/Makefile still pulls in
4+
# python2-era tooling (python-all-dev) that is unavailable on newer
5+
# Debian releases.
6+
#
7+
# This image only provides the toolchain needed to build the generic
8+
# kvm_x86_64 target with secure boot disabled. It intentionally omits
9+
# the secure-boot / code-signing tooling (pesign, sbsigntool, ...).
10+
FROM debian:11
11+
WORKDIR /onie
12+
13+
# Tolerate transient Debian mirror hiccups (connection resets mid-fetch).
14+
RUN echo 'Acquire::Retries "5";' > /etc/apt/apt.conf.d/80-retries
15+
16+
# Build dependencies for crosstool-NG and the ONIE image.
17+
RUN apt-get update && apt-get install -y \
18+
autoconf \
19+
automake \
20+
autopoint \
21+
bc \
22+
bison \
23+
bsdmainutils \
24+
build-essential \
25+
coreutils \
26+
cpio \
27+
curl \
28+
device-tree-compiler \
29+
dosfstools \
30+
fakeroot \
31+
flex \
32+
gawk \
33+
git \
34+
gperf \
35+
help2man \
36+
libefivar-dev \
37+
libelf-dev \
38+
libexpat1 \
39+
libexpat1-dev \
40+
libncurses5 \
41+
libncurses5-dev \
42+
libpopt-dev \
43+
libssl-dev \
44+
libtool \
45+
libtool-bin \
46+
locales \
47+
mtools \
48+
pkgconf \
49+
python-all-dev \
50+
python3-all-dev \
51+
python3-sphinx \
52+
python3-venv \
53+
rst2pdf \
54+
rsync \
55+
stgit \
56+
texinfo \
57+
u-boot-tools \
58+
unzip \
59+
util-linux \
60+
uuid-dev \
61+
uuid-runtime \
62+
wget \
63+
zip \
64+
xorriso \
65+
&& rm -rf /var/lib/apt/lists/*
66+
67+
# Generate UTF-8 locale (needed for the uclibc / crosstool-NG build).
68+
RUN sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && \
69+
locale-gen
70+
71+
# Create the build user with the same UID/GID as the host invoker so
72+
# build artifacts are not left owned by root. Override at build time
73+
# with --build-arg UID=$(id -u) --build-arg GID=$(id -g).
74+
#
75+
# NOTE: -l avoids docker layer-export hangs for large UIDs.
76+
# (https://github.com/moby/moby/issues/5419#issuecomment-41478290)
77+
ARG UID=1000
78+
ARG GID=1000
79+
RUN groupadd -g $GID build && \
80+
useradd -l -m -u $UID -g $GID -s /bin/bash build && \
81+
chown -R build:build /onie
82+
83+
USER build
84+
85+
# /sbin and /usr/sbin hold tools the build invokes.
86+
RUN echo 'export PATH="/sbin:/usr/sbin:$PATH"' >> ~/.bashrc
87+
88+
# The build runs git commands; give it a default identity.
89+
RUN git config --global user.email "build@example.com" && \
90+
git config --global user.name "Build User"
91+
92+
CMD ["/bin/bash", "--login"]

‎.github/workflows/build-onie.yml‎

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
name: Build ONIE (kvm_x86_64)
2+
3+
# Builds the generic amd64 ONIE image for the kvm_x86_64 target, intended
4+
# to run as a VM under KVM/QEMU. The build follows the upstream-recommended
5+
# containerized flow (see contrib/build-env and machine/kvm_x86_64/INSTALL),
6+
# with secure boot disabled so no signing keys are required.
7+
8+
on:
9+
workflow_dispatch:
10+
push:
11+
paths:
12+
- 'build-config/**'
13+
- 'machine/kvm_x86_64/**'
14+
- '.github/workflows/build-onie.yml'
15+
- '.github/onie-build/**'
16+
pull_request:
17+
paths:
18+
- 'build-config/**'
19+
- 'machine/kvm_x86_64/**'
20+
- '.github/workflows/build-onie.yml'
21+
- '.github/onie-build/**'
22+
23+
# Cancel an in-progress run when a newer commit is pushed to the same ref,
24+
# so stacked pushes don't pile up concurrent builds.
25+
concurrency:
26+
group: ${{ github.workflow }}-${{ github.ref }}
27+
cancel-in-progress: true
28+
29+
jobs:
30+
build:
31+
name: Build kvm_x86_64
32+
runs-on: ubuntu-latest
33+
steps:
34+
- name: Checkout
35+
uses: actions/checkout@v4
36+
37+
- name: Set up Docker Buildx
38+
uses: docker/setup-buildx-action@v3
39+
40+
- name: Determine host UID/GID
41+
id: ids
42+
run: |
43+
echo "uid=$(id -u)" >> "$GITHUB_OUTPUT"
44+
echo "gid=$(id -g)" >> "$GITHUB_OUTPUT"
45+
46+
- name: Build the ONIE build-environment image
47+
uses: docker/build-push-action@v6
48+
with:
49+
context: .github/onie-build
50+
tags: onie-build-env:latest
51+
build-args: |
52+
UID=${{ steps.ids.outputs.uid }}
53+
GID=${{ steps.ids.outputs.gid }}
54+
load: true
55+
cache-from: type=gha
56+
cache-to: type=gha,mode=max
57+
58+
- name: Disable secure boot in the kernel config
59+
# The kvm_x86_64 kernel config defaults to expecting signing keys.
60+
# The repo ships a config-insecure variant for non-secure-boot builds.
61+
run: cp machine/kvm_x86_64/kernel/config-insecure machine/kvm_x86_64/kernel/config
62+
63+
# Cache the crosstool-NG cross toolchain -- the long pole of a cold
64+
# build. We must cache the entire stamp chain, not just the final
65+
# toolchain, or ONIE's make will rebuild it: a missing download stamp
66+
# (build/download) or project stamp (build/stamp-project) forces the
67+
# chain to re-run from the top.
68+
#
69+
# build/stamp-project -- top-of-chain project stamp
70+
# build/download -- toolchain source tarballs + download stamps
71+
# build/crosstool-ng -- the crosstool-NG tool + its build stamps
72+
# build/x-tools -- the installed cross toolchain + its stamps
73+
#
74+
# The key is tied to every input that can change the toolchain --
75+
# including the Dockerfile, since it defines the compiler/host
76+
# environment the toolchain is built in. No loose restore-keys: any
77+
# input change forces a clean rebuild rather than restoring a
78+
# mismatched toolchain.
79+
- name: Restore cross-toolchain cache
80+
id: xtools-cache
81+
uses: actions/cache/restore@v4
82+
with:
83+
path: |
84+
build/stamp-project
85+
build/download
86+
build/crosstool-ng
87+
build/x-tools
88+
# v2: cache the full stamp chain (added build/stamp-project +
89+
# build/download); bump to bypass v1 caches that lack them.
90+
key: onie-xtools-v2-kvm_x86_64-${{ hashFiles('build-config/make/xtools.make', 'build-config/make/crosstool-ng.make', 'build-config/make/compiler.make', 'build-config/conf/crosstool/**', 'patches/crosstool-NG/**', '.github/onie-build/Dockerfile') }}
91+
92+
# ONIE drives its build with stamp files compared by mtime. A fresh
93+
# checkout gives every repo source file a current mtime, which is newer
94+
# than the restored stamps -- so make would consider the toolchain
95+
# stale and rebuild it (e.g. build/.../.config depends on the
96+
# checked-out conf/crosstool config). A cache hit means the key
97+
# matched, i.e. every toolchain input is unchanged and the restored
98+
# toolchain is valid, so bump the restored artifacts ahead of the
99+
# checkout to make "make xtools" a genuine no-op.
100+
- name: Mark restored toolchain up to date
101+
if: steps.xtools-cache.outputs.cache-hit == 'true'
102+
run: find build/stamp-project build/download build/crosstool-ng build/x-tools -exec touch {} +
103+
104+
- name: Build cross toolchain
105+
run: |
106+
docker run --rm --privileged \
107+
-v "${PWD}:/onie" \
108+
onie-build-env \
109+
bash -lc 'cd build-config && make -j"$(nproc)" \
110+
MACHINE=kvm_x86_64 \
111+
SECURE_BOOT_ENABLE=no \
112+
SECURE_BOOT_EXT=no \
113+
SECURE_GRUB=no \
114+
xtools'
115+
116+
# Save only on a cache miss, and only after the toolchain build above
117+
# succeeded (default if: success()), so a broken toolchain is never
118+
# cached. Running before the full build means a later-stage failure
119+
# still preserves the toolchain for the next run.
120+
- name: Save cross-toolchain cache
121+
if: steps.xtools-cache.outputs.cache-hit != 'true'
122+
uses: actions/cache/save@v4
123+
with:
124+
path: |
125+
build/stamp-project
126+
build/download
127+
build/crosstool-ng
128+
build/x-tools
129+
key: ${{ steps.xtools-cache.outputs.cache-primary-key }}
130+
131+
- name: Build ONIE
132+
run: |
133+
docker run --rm --privileged \
134+
-v "${PWD}:/onie" \
135+
onie-build-env \
136+
bash -lc 'cd build-config && make -j"$(nproc)" \
137+
MACHINE=kvm_x86_64 \
138+
SECURE_BOOT_ENABLE=no \
139+
SECURE_BOOT_EXT=no \
140+
SECURE_GRUB=no \
141+
all recovery-iso'

0 commit comments

Comments
 (0)