Skip to content

Commit 7bcded6

Browse files
committed
feat(extensions/netboot): clarify kernel-only deploy contract — initramfs is build-side only
Initramfs cannot be regenerated correctly outside a full image build: its content depends on the configured rootfs (customize_image hooks, /etc/initramfs-tools tweaks, board-specific extensions, userpatches overlay), and that context survives only during the build itself. After deploy the configured rootfs lives only on the NFS server, which may be OpenWRT/embedded/etc. and cannot run chroot+update-initramfs; regenerating from the generic post-debootstrap rootfs cache would produce a *different* initramfs than the one the full build would have made — fake, not just incomplete. Therefore artifact_ready__netboot_kernel_deploy now: - drops any pre-existing uInitrd from TFTP after the kernel rsync, so U-Boot does not pair the new kernel with a stale initramfs whose modules have a wrong vermagic - replaces the previous 'run update-initramfs on the server' guidance (which was wrong — server-side regen is impossible on most servers and produces a wrong initramfs even where it is) with a clear contract: 'for a fresh initramfs do a full image rebuild' - documents the boot-time consequence in the function header: boards with built-in boot networking (mvneta, etc.) come up cleanly without initramfs; modular-NIC boards fail fast at networking and the operator knows to do a full rebuild README 'Kernel-only deploy' section reframed as a narrow optimization for built-in-NIC boards rather than a general kernel refresh path, with explicit guidance on when to use it vs the full image rebuild. Closes the long-standing P2 (NETBOOT_DEPLOY_REGENERATE_INITRD) by contract clarification, not deferred TODO: standalone initramfs regeneration is an architecturally wrong frame, removed from scope. Assisted-by: Claude:claude-opus-4.7 Signed-off-by: Igor Velkov <325961+iav@users.noreply.github.com>
1 parent 3e9d578 commit 7bcded6

2 files changed

Lines changed: 84 additions & 27 deletions

File tree

extensions/netboot/README.md

Lines changed: 41 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -179,11 +179,14 @@ is skipped — only TFTP is deployed.
179179

180180
#### Kernel-only deploy
181181

182-
The same extension also exposes an `artifact_ready` handler that fires
183-
after a stand-alone `compile.sh kernel ...` run (no image build, no
184-
rootfs archive). It unpacks the produced `linux-image-${BRANCH}-${LINUXFAMILY}_*.deb`
185-
and rsyncs vmlinuz + dtbs into `${NETBOOT_DEPLOY_TFTP_ROOT}/${NETBOOT_TFTP_PREFIX}/`
186-
and `/lib/modules/${kver}/` into the NFS rootfs at `${NETBOOT_NFS_PATH}/lib/modules/${kver}/`.
182+
A narrow optimization for **boards whose boot networking driver is built
183+
into the kernel** (e.g. helios64 mvneta, helios4 mvneta — anything where
184+
`root=/dev/nfs ip=dhcp` works without an initramfs). On such boards a
185+
fresh kernel can be deployed without rebuilding the rootfs or initramfs
186+
at all, which turns a 15-minute full image rebuild into a 10-second
187+
incremental refresh — useful for kernel debugging and config iteration.
188+
189+
Triggered by `compile.sh kernel ...`:
187190

188191
```bash
189192
./compile.sh iav kernel \
@@ -192,26 +195,46 @@ and `/lib/modules/${kver}/` into the NFS rootfs at `${NETBOOT_NFS_PATH}/lib/modu
192195
NETBOOT_DEPLOY_SSH=root@m1
193196
```
194197

198+
The `artifact_ready` handler unpacks the produced
199+
`linux-image-${BRANCH}-${LINUXFAMILY}_*.deb`, rsyncs `vmlinuz` and dtbs
200+
into `${NETBOOT_DEPLOY_TFTP_ROOT}/${NETBOOT_TFTP_PREFIX}/`, syncs
201+
`/lib/modules/${kver}/` (with `--delete`) into the NFS rootfs at
202+
`${NETBOOT_NFS_PATH}/lib/modules/${kver}/`, and **removes any
203+
pre-existing `uInitrd`** from the TFTP prefix.
204+
195205
This closes a coherence gap that otherwise produces `BPF: Invalid
196206
name_offset:N` and `failed to validate module BTF: -22` spam on boot:
197207
split-BTF references in `.ko` modules point to the BTF inside the
198208
running vmlinux; if the kernel image and the modules in `/lib/modules/`
199209
come from different build runs, every match resolves to a wrong string
200210
offset. Coherent kernel + modules deployed together fix it.
201211

202-
**Caveat — initramfs is not regenerated.** A new module ABI may need a
203-
fresh initrd. After the kernel deploy, run:
204-
205-
```bash
206-
ssh root@m1 \
207-
'chroot /srv/netboot/rootfs/shared/<board>/<branch>-<release> \
208-
update-initramfs -u -k <kver>'
209-
```
210-
211-
or rebuild the full image (`compile.sh build ROOTFS_TYPE=nfs-root ...`)
212-
when the kernel ABI moves enough to need it. Cross-architecture
213-
chroot needs `qemu-user-static` + `binfmt_misc` registered on the
214-
**server** (not the build host).
212+
**Initramfs is intentionally not regenerated, by design.** Initramfs
213+
content depends on the *configured* rootfs — `customize_image` hooks,
214+
`/etc/initramfs-tools/*` tweaks, board-specific extensions, userpatches
215+
overlay — and that context only exists during a full image build. Past
216+
that build, the configured rootfs lives only on the NFS server, which
217+
may be an OpenWRT box or anything else that cannot run
218+
chroot+update-initramfs. Regenerating from a generic post-debootstrap
219+
rootfs cache would produce a *different* initramfs than the one a full
220+
image build would have made — wrong, not just incomplete.
221+
222+
So the kernel-only deploy drops the previous build's `uInitrd` and lets
223+
the kernel boot without an initramfs:
224+
225+
- **Boards with built-in boot networking**: `root=/dev/nfs ip=dhcp` runs
226+
directly from kernel — clean boot, every time. Modules under
227+
`/lib/modules/${kver}/` get loaded later for non-boot-critical drivers
228+
(WiFi, BT, etc.).
229+
- **Boards needing modular drivers in initramfs** (USB-eth, modular NIC,
230+
modular SATA, etc.): kernel will fail fast at networking instead of
231+
silently corrupting later. The fix for those boards is **not** a kernel
232+
refresh — it is a full image rebuild
233+
(`compile.sh build ROOTFS_TYPE=nfs-root ...`), which produces a fresh
234+
initramfs alongside the kernel.
235+
236+
Use the kernel-only path when you know your board falls in the first
237+
category. If unsure, the full image rebuild is always the safe choice.
215238

216239
## Build artifacts matrix
217240

extensions/netboot/netboot-deploy.sh

Lines changed: 43 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -447,10 +447,29 @@ function netboot_artifacts_ready__deploy_to_remote_server() (
447447
# Standalone-kernel deploy: fired by core's `artifact_ready` after a
448448
# `compile.sh kernel ...` run. No image, no rootfs archive — the canonical
449449
# linux-image-${BRANCH}-${LINUXFAMILY} .deb is unpacked locally; vmlinuz +
450-
# dtbs go to TFTP, /lib/modules/<ver>/ goes to the NFS rootfs. Closes the
451-
# split-BTF coherence gap: a fresh kernel was getting paired with stale
452-
# modules in the on-NFS rootfs, producing `BPF: Invalid name_offset` spam
453-
# until a full image rebuild.
450+
# dtbs go to TFTP, /lib/modules/<ver>/ goes to the NFS rootfs.
451+
#
452+
# Scope: kernel + modules only. Initramfs is intentionally NOT regenerated
453+
# here — it depends on the rootfs that was customized at full image build
454+
# time (customize_image hooks, /etc/initramfs-tools tweaks, board-specific
455+
# extensions, userpatches overlay), and that context does not survive on
456+
# the build host past the original image deploy. The configured rootfs now
457+
# lives only on the NFS server, which may be an OpenWRT box or anything
458+
# else that cannot run chroot+update-initramfs. Regenerating from the
459+
# generic post-debootstrap rootfs cache would produce a *different*
460+
# initramfs than the one the full image build would have made — fake.
461+
#
462+
# Therefore: any pre-existing uInitrd on TFTP is removed so U-Boot does
463+
# not load an initramfs whose modules have stale vermagic against the
464+
# new kernel. Boards with built-in boot networking (mvneta, etc.) come
465+
# up cleanly without initramfs via root=/dev/nfs ip=dhcp; boards that
466+
# need modular drivers in initramfs (USB-eth, modular NICs) will fail
467+
# fast at networking — the fix for those is a full image rebuild, not
468+
# a kernel-only refresh.
469+
#
470+
# Closes the split-BTF coherence gap: a fresh kernel was getting paired
471+
# with stale modules in the on-NFS rootfs, producing `BPF: Invalid
472+
# name_offset` spam until a full image rebuild.
454473
function artifact_ready__netboot_kernel_deploy() (
455474
declare scratch_key=""
456475
declare scratch_dir=""
@@ -571,6 +590,21 @@ function artifact_ready__netboot_kernel_deploy() (
571590
"${kernel_src}" \
572591
"${NETBOOT_DEPLOY_SSH}:${NETBOOT_DEPLOY_TFTP_ROOT}/${NETBOOT_TFTP_PREFIX}/${kernel_name}"
573592

593+
# Drop any pre-existing uInitrd from TFTP. See the function's header
594+
# comment for the full rationale: kernel-only deploy intentionally does
595+
# not regenerate initramfs (cannot, without the configured rootfs context),
596+
# and leaving the previous full-image build's uInitrd next to a fresh
597+
# kernel would have U-Boot load it and the kernel hit vermagic mismatches
598+
# inside initramfs's modprobe. Removing it makes U-Boot proceed without
599+
# initramfs — clean boot on built-in-NIC boards, fail-fast on modular-NIC
600+
# ones (the operator then knows to do a full image rebuild).
601+
declare q_uinitrd_path
602+
q_uinitrd_path="'${NETBOOT_DEPLOY_TFTP_ROOT//\'/\'\\\'\'}/${NETBOOT_TFTP_PREFIX//\'/\'\\\'\'}/uInitrd'"
603+
display_alert "${EXTENSION}: drop stale uInitrd from TFTP" \
604+
"${NETBOOT_DEPLOY_SSH}:${NETBOOT_DEPLOY_TFTP_ROOT}/${NETBOOT_TFTP_PREFIX}/uInitrd" "info"
605+
run_host_command_logged_raw ssh "${ssh_opts[@]}" "${NETBOOT_DEPLOY_SSH}" \
606+
"${sudo_prefix}rm -f ${q_uinitrd_path}"
607+
574608
# DTBs: linux-image deb stages them under /usr/lib/linux-image-<ver>/.
575609
# netboot.sh on a full image rebuild flattens them into ${NETBOOT_TFTP_PREFIX}/dtb/
576610
# (e.g. dtb/rockchip/rk3399-helios64.dtb). Reproduce that layout exactly so
@@ -606,10 +640,10 @@ function artifact_ready__netboot_kernel_deploy() (
606640

607641
display_alert "${EXTENSION}: kernel deploy done" \
608642
"${NETBOOT_DEPLOY_SSH} (kver=${kver})" "info"
609-
# Initramfs is intentionally left alone: regenerating it requires a
610-
# chroot into the NFS rootfs (cross-arch needs qemu-user-static binfmt
611-
# on the *server*, not the build host). The operator does this manually
612-
# or rebuilds the full image when the module ABI changes meaningfully.
643+
# Initramfs is not produced here — see the function's header comment.
644+
# The previous uInitrd was already dropped from TFTP above; this alert
645+
# exists so the operator sees, in the build log, that initramfs is a
646+
# concern of the full image build and not of kernel-only deploy.
613647
display_alert "${EXTENSION}: skipped initramfs refresh" \
614-
"run 'chroot ${NETBOOT_NFS_PATH} update-initramfs -u -k ${kver}' on the server, or rebuild the full image" "warn"
648+
"kernel-only deploy cannot regenerate initramfs (depends on configured rootfs context); for a fresh initramfs do a full image rebuild" "warn"
615649
)

0 commit comments

Comments
 (0)