You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(rootfs): native armhf on aarch64 host via runtime-disable of qemu-arm
Continues #9284 (arm64-compat-vdso extension + custom_kernel_make_params
hook). That PR enabled COMPAT_VDSO in arm64 kernels and wired
CROSS_COMPILE_COMPAT through the build; this PR uses the resulting
32-bit-capable kernels to run 32-bit ARM ELF directly instead of through
qemu-arm emulation in the image-build chroot work.
Disable qemu-arm in /proc/sys/fs/binfmt_misc for the duration of the
image build so 32-bit ARM ELF falls through to kernel binfmt_elf and
runs natively via CONFIG_COMPAT — typically ~12x faster than qemu-arm
emulation on common ARM cores (Cortex-A53/A55/A72/A73/A76).
Empirical: helios4 BUILD_MINIMAL armhf rootfs cache-miss build on RK3568
(Cortex-A55) drops from 60:35 to 19:27 — 3.12x speedup; same build with
rootfs cache-hit drops to 6:38 (~5x).
Activation point: AFTER mmdebstrap (in
create_new_rootfs_cache_via_debootstrap, right past the bash-presence
check) and at build_rootfs_and_image entry for the rootfs-cache hit
path. Earlier activation breaks mmdebstrap because its cross-arch
machinery requires a working qemu-arm in binfmt_misc and the chroot is
not yet populated for native exec.
Out-of-scope cases (target arch != armhf, host arch != aarch64) return
without touching binfmt_misc.
NATIVE_ARMHF_ON_ARM64=no (or never/disabled) disables the entire code
path before any detection runs.
CONCURRENCY: per-host owner-flock pattern
/proc/sys/fs/binfmt_misc is per-kernel state shared by every build
process on the host (under docker --privileged also shared with the
host's init user-ns). Coordination uses long-lived flock'ed owner files
under /run/lock/armbian-native-armhf/owners/${ARMBIAN_BUILD_UUID}.lock.
The first owner saves prior qemu-arm state and disables it; the last
owner restores the saved state. Live owner count = number of owner
files whose flock is held by a live process — checked by attempting
non-blocking flock on each: success means the owner is dead and the
file is pruned. Crash recovery is automatic (kernel releases the fd on
process exit).
Refcount/pidlist explicitly avoided — breaks under PID namespaces and
PID reuse. Live-flock is the source of truth.
Per-host scope by design: shared NFS for the lockdir would be incorrect
— binfmt_misc is per-kernel-instance, no cross-host coordination
possible. The lockdir at /run/lock is local-to-host. The Armbian build
container bind-mounts /run from the host, so multiple containers on
the same host share the same lockdir.
Critical-path invariants:
- State-preserving restore: the first owner records the observed
prior state (1 / 0 / missing); the last owner restores from that
file. Never blindly writes 1.
- Stale active window recovery: if a state file exists when we
become first-owner (previous build crashed pre-cleanup),
reconcile qemu-arm back to the saved prior state and remove the
stale file before observing our own prior. A single SIGKILL'ed
build cannot leave qemu-arm permanently disabled.
- Disable-failure full abort: if the echo-0 write itself fails (no
CAP_SYS_ADMIN, etc.), setup releases its owner lock and removes
the state file under the held control lock. A failed disable does
not leave a live native-mode owner — otherwise a concurrent
builder would join the ghost owner, skip qemu binary deployment,
and fail at chroot exec.
- Join-state verification: a builder taking the join branch checks
the actual qemu-arm state and aborts to qemu fallback if it is
observably enabled. Catches an external agent (admin /
update-binfmts) re-enabling qemu-arm during the active window —
without this our pre-flight arch-test would pass as a false
positive (qemu interprets the stub).
- Critical-section ordering: the cleanup trap takes the control
lock BEFORE releasing its owner lock, so prune+count+restore is
atomic with respect to any concurrent setup. Releasing owner
before control would open a race where a fresh first-owner could
observe the just-disabled state and record it as its own prior.
Reviewed iteratively over four rounds with codex (gpt-5.5) on the
synchronisation design and the race fixes that followed; coderabbitai
flagged the same join-state check independently and self-confirmed
the implementation as addressed.
Assisted-by: Claude:claude-opus-4.7
Signed-off-by: Igor Velkov <325961+iav@users.noreply.github.com>
display_alert "Native armhf via binfmt_elf""${live_count} live owner(s) but qemu-arm is observably enabled (external agent re-enabled it); aborting join, falling back to qemu-arm-static emulation""wrn"
342
+
_native_armhf_release_owner_lock
343
+
exec {control_fd}>&-
344
+
return 1
345
+
fi
346
+
# Cleanup is still registered so this owner's release decrements
display_alert "Native armhf via binfmt_elf""joining ${live_count} existing native-armhf owner(s); shared qemu-arm state preserved""info"
350
+
fi
351
+
352
+
# Authoritative check: re-run arch-test with qemu-arm now disabled. This
353
+
# reflects what the chroot exec will actually face. Pre-disable arch-test
354
+
# can be a false positive when host kernel lacks COMPAT_VDSO (extensions/
355
+
# arm64-compat-vdso, PR #9284) — qemu-arm interprets the stub.
356
+
if! arch-test armhf > /dev/null 2>&1;then
357
+
display_alert "Native armhf via binfmt_elf""post-disable verification failed (host kernel lacks COMPAT_VDSO — see extensions/arm64-compat-vdso); restoring and falling back to emulation""wrn"
358
+
# Drop control lock so the handler can re-acquire it. Do NOT
359
+
# release the owner lock here — the handler must release it UNDER
360
+
# control so live_count reflects only OTHER owners and the
361
+
# state-preserving restore decision uses a consistent snapshot.
362
+
# Releasing owner outside the control section was the race codex
363
+
# flagged in the previous round.
364
+
exec {control_fd}>&-
365
+
trap_handler_native_armhf_restore_qemu_arm
366
+
return 1
367
+
fi
368
+
369
+
exec {control_fd}>&-
370
+
display_alert "Native armhf via binfmt_elf""kernel $(uname -r), aarch64 host with COMPAT_VDSO; qemu-arm disabled, kernel binfmt_elf takes over""info"
0 commit comments