Commit ca4b442
committed
feat(rootfs): native armhf on aarch64 host via runtime-disable of qemu-arm
Continues #9284 (arm64-compat-vdso extension + custom_kernel_make_params
hook). That PR enabled COMPAT_VDSO in arm64 kernels and wired
CROSS_COMPILE_COMPAT through the build; this PR uses the resulting
32-bit-capable kernels to run 32-bit ARM ELF directly instead of through
qemu-arm emulation in the image-build chroot work.
Disable qemu-arm in /proc/sys/fs/binfmt_misc for the duration of the
image build so 32-bit ARM ELF falls through to kernel binfmt_elf and
runs natively via CONFIG_COMPAT — typically ~12x faster than qemu-arm
emulation on common ARM cores (Cortex-A53/A55/A72/A73/A76).
Empirical: helios4 BUILD_MINIMAL armhf rootfs cache-miss build on RK3568
(Cortex-A55) drops from 60:35 to 19:27 — 3.12x speedup; same build with
rootfs cache-hit drops to 6:38 (~5x).
Activation point: AFTER mmdebstrap (in
create_new_rootfs_cache_via_debootstrap, right past the bash-presence
check) and at build_rootfs_and_image entry for the rootfs-cache hit
path. Earlier activation breaks mmdebstrap because its cross-arch
machinery requires a working qemu-arm in binfmt_misc and the chroot is
not yet populated for native exec.
Out-of-scope cases (target arch != armhf, host arch != aarch64) return
without touching binfmt_misc.
NATIVE_ARMHF_ON_ARM64=no (or never/disabled) disables the entire code
path before any detection runs.
CONCURRENCY: per-host owner-flock pattern
/proc/sys/fs/binfmt_misc is per-kernel state shared by every build
process on the host (under docker --privileged also shared with the
host's init user-ns). Coordination uses long-lived flock'ed owner files
under /run/lock/armbian-native-armhf/owners/${ARMBIAN_BUILD_UUID}.lock.
The first owner saves prior qemu-arm state and disables it; the last
owner restores the saved state. Live owner count = number of owner
files whose flock is held by a live process — checked by attempting
non-blocking flock on each: success means the owner is dead and the
file is pruned. Crash recovery is automatic (kernel releases the fd on
process exit).
Refcount/pidlist explicitly avoided — breaks under PID namespaces and
PID reuse. Live-flock is the source of truth.
Per-host scope by design: shared NFS for the lockdir would be incorrect
— binfmt_misc is per-kernel-instance, no cross-host coordination
possible. The lockdir at /run/lock is local-to-host. The Armbian build
container bind-mounts /run from the host, so multiple containers on
the same host share the same lockdir.
Critical-path invariants:
- State-preserving restore: the first owner records the observed
prior state (1 / 0 / missing); the last owner restores from that
file. Never blindly writes 1.
- Stale active window recovery: if a state file exists when we
become first-owner (previous build crashed pre-cleanup),
reconcile qemu-arm back to the saved prior state and remove the
stale file before observing our own prior. A single SIGKILL'ed
build cannot leave qemu-arm permanently disabled.
- Disable-failure full abort: if the echo-0 write itself fails (no
CAP_SYS_ADMIN, etc.), setup releases its owner lock and removes
the state file under the held control lock. A failed disable does
not leave a live native-mode owner — otherwise a concurrent
builder would join the ghost owner, skip qemu binary deployment,
and fail at chroot exec.
- Join-state verification: a builder taking the join branch checks
the actual qemu-arm state and aborts to qemu fallback if it is
observably enabled. Catches an external agent (admin /
update-binfmts) re-enabling qemu-arm during the active window —
without this our pre-flight arch-test would pass as a false
positive (qemu interprets the stub).
- Critical-section ordering: the cleanup trap takes the control
lock BEFORE releasing its owner lock, so prune+count+restore is
atomic with respect to any concurrent setup. Releasing owner
before control would open a race where a fresh first-owner could
observe the just-disabled state and record it as its own prior.
Reviewed iteratively over four rounds with codex (gpt-5.5) on the
synchronisation design and the race fixes that followed; coderabbitai
flagged the same join-state check independently and self-confirmed
the implementation as addressed.
Assisted-by: Claude:claude-opus-4.7
Signed-off-by: Igor Velkov <325961+iav@users.noreply.github.com>1 parent 38c583f commit ca4b442
3 files changed
Lines changed: 372 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
16 | 21 | | |
17 | 22 | | |
18 | 23 | | |
| |||
0 commit comments