Skip to content

Commit ca4b442

Browse files
committed
feat(rootfs): native armhf on aarch64 host via runtime-disable of qemu-arm
Continues #9284 (arm64-compat-vdso extension + custom_kernel_make_params hook). That PR enabled COMPAT_VDSO in arm64 kernels and wired CROSS_COMPILE_COMPAT through the build; this PR uses the resulting 32-bit-capable kernels to run 32-bit ARM ELF directly instead of through qemu-arm emulation in the image-build chroot work. Disable qemu-arm in /proc/sys/fs/binfmt_misc for the duration of the image build so 32-bit ARM ELF falls through to kernel binfmt_elf and runs natively via CONFIG_COMPAT — typically ~12x faster than qemu-arm emulation on common ARM cores (Cortex-A53/A55/A72/A73/A76). Empirical: helios4 BUILD_MINIMAL armhf rootfs cache-miss build on RK3568 (Cortex-A55) drops from 60:35 to 19:27 — 3.12x speedup; same build with rootfs cache-hit drops to 6:38 (~5x). Activation point: AFTER mmdebstrap (in create_new_rootfs_cache_via_debootstrap, right past the bash-presence check) and at build_rootfs_and_image entry for the rootfs-cache hit path. Earlier activation breaks mmdebstrap because its cross-arch machinery requires a working qemu-arm in binfmt_misc and the chroot is not yet populated for native exec. Out-of-scope cases (target arch != armhf, host arch != aarch64) return without touching binfmt_misc. NATIVE_ARMHF_ON_ARM64=no (or never/disabled) disables the entire code path before any detection runs. CONCURRENCY: per-host owner-flock pattern /proc/sys/fs/binfmt_misc is per-kernel state shared by every build process on the host (under docker --privileged also shared with the host's init user-ns). Coordination uses long-lived flock'ed owner files under /run/lock/armbian-native-armhf/owners/${ARMBIAN_BUILD_UUID}.lock. The first owner saves prior qemu-arm state and disables it; the last owner restores the saved state. Live owner count = number of owner files whose flock is held by a live process — checked by attempting non-blocking flock on each: success means the owner is dead and the file is pruned. Crash recovery is automatic (kernel releases the fd on process exit). Refcount/pidlist explicitly avoided — breaks under PID namespaces and PID reuse. Live-flock is the source of truth. Per-host scope by design: shared NFS for the lockdir would be incorrect — binfmt_misc is per-kernel-instance, no cross-host coordination possible. The lockdir at /run/lock is local-to-host. The Armbian build container bind-mounts /run from the host, so multiple containers on the same host share the same lockdir. Critical-path invariants: - State-preserving restore: the first owner records the observed prior state (1 / 0 / missing); the last owner restores from that file. Never blindly writes 1. - Stale active window recovery: if a state file exists when we become first-owner (previous build crashed pre-cleanup), reconcile qemu-arm back to the saved prior state and remove the stale file before observing our own prior. A single SIGKILL'ed build cannot leave qemu-arm permanently disabled. - Disable-failure full abort: if the echo-0 write itself fails (no CAP_SYS_ADMIN, etc.), setup releases its owner lock and removes the state file under the held control lock. A failed disable does not leave a live native-mode owner — otherwise a concurrent builder would join the ghost owner, skip qemu binary deployment, and fail at chroot exec. - Join-state verification: a builder taking the join branch checks the actual qemu-arm state and aborts to qemu fallback if it is observably enabled. Catches an external agent (admin / update-binfmts) re-enabling qemu-arm during the active window — without this our pre-flight arch-test would pass as a false positive (qemu interprets the stub). - Critical-section ordering: the cleanup trap takes the control lock BEFORE releasing its owner lock, so prune+count+restore is atomic with respect to any concurrent setup. Releasing owner before control would open a race where a fresh first-owner could observe the just-disabled state and record it as its own prior. Reviewed iteratively over four rounds with codex (gpt-5.5) on the synchronisation design and the race fixes that followed; coderabbitai flagged the same join-state check independently and self-confirmed the implementation as addressed. Assisted-by: Claude:claude-opus-4.7 Signed-off-by: Igor Velkov <325961+iav@users.noreply.github.com>
1 parent 38c583f commit ca4b442

3 files changed

Lines changed: 372 additions & 4 deletions

File tree

lib/functions/main/rootfs-image.sh

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,11 @@ function build_rootfs_and_image() {
1313
# get a basic rootfs, either from cache or from scratch
1414
get_or_create_rootfs_cache_chroot_sdcard # only occurrence of this; has its own logging sections
1515

16+
# Cache-hit path also benefits — the extracted rootfs has libc/ld-linux, so kernel
17+
# binfmt_elf can run 32-bit ARM ELF natively. Idempotent on cache-miss path where
18+
# create_new_rootfs_cache_via_debootstrap already activated this.
19+
_native_armhf_setup_binfmt_elf || true
20+
1621
# deploy the qemu binary, no matter where the rootfs came from (built or cached)
1722
LOG_SECTION="deploy_qemu_binary_to_chroot_image" do_with_logging deploy_qemu_binary_to_chroot "${SDCARD}" "image" # undeployed at end of this function
1823

0 commit comments

Comments
 (0)