Skip to content

Commit 78d40ef

Browse files
iaviav
committed
feat(native-armhf-on-arm64): opt-in extension + host_binfmt_ready core hook
Opt-in extension that disables qemu-arm in binfmt_misc for the duration of an armhf build on an aarch64 host. The kernel's binfmt_elf path then runs 32-bit ARM ELF binaries natively via CONFIG_COMPAT, removing the ~10× qemu-user-static overhead from mmdebstrap, chroot apt-get, dpkg --configure, and customize_image stages. Activate with ENABLE_EXTENSIONS=native-armhf-on-arm64. Core change (lib/functions/host/prepare-host.sh): Add a new extension hook `host_binfmt_ready`, fired from `prepare_host_noninteractive` immediately after `prepare_host_binfmt_qemu`. At this point the binfmt_misc registrations for the target architecture are in place, which is the right window for any extension that wants to mutate them. Five lines, no behavioral change for builds that don't use the hook. Extension (extensions/native-armhf-on-arm64.sh): - Gate: ARCH=armhf and host arch aarch64 — silent no-op otherwise. - Verify qemu-arm is registered and enabled (it always is after prepare_host_binfmt_qemu — otherwise abort with a clear message). - Disable qemu-arm via /proc/sys/fs/binfmt_misc/qemu-arm. - Probe-with-rollback: exec /usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3 --help to verify the kernel can still run armhf natively via CONFIG_COMPAT. If it can't (no COMPAT), re-enable qemu-arm and abort with a clear message. We use a direct ld-linux exec rather than `arch-test armhf` because the latter is unreliable on some aarch64 hosts (e.g. Hetzner Ampere CAX, Ubuntu Noble 6.8) — it returns failure even when CONFIG_COMPAT is fully functional. - add_cleanup_handler on success: re-enable qemu-arm on build exit (best-effort; if the build is SIGKILL'd, qemu-arm stays disabled; documented manual restore in the header). Restrictions (header): - NO CONCURRENT ARMBIAN BUILDS on the same host while this extension is active. The disable mutates global kernel state. The operator owns the host while this extension runs — no flock dance, no concurrency claims. Matches the safety level of baseline armbian (which also doesn't synchronize binfmt_misc mutations across concurrent builds). - The in-core flock-based variant at PR armbian#9769 covers concurrent-safe use; this extension is the opt-in lightweight alternative. Validated on Hetzner CAX21 (Ampere Altra, Ubuntu Noble 6.8.0-90, CONFIG_COMPAT=y, no COMPAT_VDSO): ld-linux-armhf exec succeeds with qemu-arm disabled. Assisted-by: Claude:claude-opus-4.7
1 parent 761d04e commit 78d40ef

2 files changed

Lines changed: 116 additions & 0 deletions

File tree

‎extensions/native-armhf-on-arm64.sh‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
#!/usr/bin/env bash
2+
#
3+
# SPDX-License-Identifier: GPL-2.0
4+
#
5+
# Native armhf-on-aarch64 acceleration.
6+
#
7+
# Activate with ENABLE_EXTENSIONS=native-armhf-on-arm64.
8+
#
9+
# WHAT
10+
# On an aarch64 host building an armhf target, disable qemu-arm in
11+
# binfmt_misc for the duration of the build. The kernel's binfmt_elf
12+
# path then runs armhf ELF binaries natively via CONFIG_COMPAT — ~10×
13+
# faster than qemu-user-static emulation. mmdebstrap, chroot apt-get,
14+
# dpkg --configure, post-install scripts and customize_image all
15+
# benefit.
16+
#
17+
# REQUIREMENTS
18+
# * Host architecture aarch64.
19+
# * Host kernel with CONFIG_COMPAT=y (32-bit ARM EL0 support).
20+
# CONFIG_COMPAT_VDSO is not strictly required — full mmdebstrap +
21+
# post-customize runs were validated on stock Ubuntu Noble (6.8.x)
22+
# without COMPAT_VDSO.
23+
# * Build target ARCH=armhf (a no-op for any other ARCH).
24+
# * qemu-user-static (or qemu-user-binfmt) registered with qemu-arm
25+
# enabled in /proc/sys/fs/binfmt_misc/qemu-arm at build start.
26+
# The extension verifies this and refuses to run otherwise.
27+
# * CAP_SYS_ADMIN on host (the build already runs as root).
28+
#
29+
# RESTRICTIONS
30+
# * NO CONCURRENT ARMBIAN BUILDS on the same host while this extension
31+
# is active. The extension globally disables qemu-arm in binfmt_misc
32+
# for the build window. Any other parallel armhf-cross workload on
33+
# the host (CI runners, container builds depending on qemu-arm) will
34+
# have its armhf binaries routed through the kernel's binfmt_elf
35+
# fallback instead of qemu — which works on this host, but may
36+
# surprise code that depends on qemu being in the chain.
37+
# The operator owns the host while this extension runs.
38+
# * If the build is SIGKILL'd or the box crashes before the cleanup
39+
# handler fires, qemu-arm stays disabled. Re-enable manually:
40+
# sudo update-binfmts --enable qemu-arm
41+
# or
42+
# echo 1 | sudo tee /proc/sys/fs/binfmt_misc/qemu-arm
43+
#
44+
# DESIGN
45+
# No flock dance, no userspace coordination, no concurrent-safety
46+
# claims. The acceleration is mechanically simple: write 0 to the
47+
# binfmt_misc entry, write 1 back on cleanup. The in-core flock-based
48+
# variant lives at PR #9769 if you need acceleration that coexists
49+
# with concurrent armbian builds on the same host.
50+
51+
function host_binfmt_ready__native_armhf_on_arm64() {
52+
# Self-gate: target arch must be armhf and host arch must be aarch64.
53+
# Anything else: no-op, silent — the extension only accelerates the
54+
# armhf-on-aarch64 case.
55+
[[ "${ARCH}" == "armhf" ]] || return 0
56+
57+
declare host_arch
58+
host_arch="$(arch)"
59+
[[ "${host_arch}" == "aarch64" ]] || return 0
60+
61+
if [[ ! -e /proc/sys/fs/binfmt_misc/qemu-arm ]]; then
62+
exit_with_error "${EXTENSION}: qemu-arm is not registered in binfmt_misc on the host even after prepare_host_binfmt_qemu ran — this is unexpected. Aborting."
63+
fi
64+
65+
declare qemu_arm_state
66+
qemu_arm_state="$(head -n1 /proc/sys/fs/binfmt_misc/qemu-arm 2> /dev/null || true)"
67+
if [[ "${qemu_arm_state}" != "enabled" ]]; then
68+
display_alert "${EXTENSION}: qemu-arm already not enabled" "state='${qemu_arm_state}' — nothing to do, leaving alone" "info"
69+
return 0
70+
fi
71+
72+
# Probe-with-rollback: disable qemu-arm, verify the kernel can still
73+
# run armhf binaries natively via binfmt_elf (CONFIG_COMPAT), and
74+
# re-enable + bail if it can't. We probe by directly exec'ing an
75+
# armhf binary (ld-linux-armhf from gcc-arm-linux-gnueabihf, an
76+
# armbian build dependency on aarch64 hosts). `arch-test armhf` is
77+
# unreliable here — on Hetzner Ampere CAX (Ubuntu Noble 6.8) it
78+
# returns failure even when CONFIG_COMPAT is fully functional.
79+
declare armhf_probe="/usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3"
80+
if [[ ! -x "${armhf_probe}" ]]; then
81+
exit_with_error "${EXTENSION}: armhf probe binary not found at ${armhf_probe}; install gcc-arm-linux-gnueabihf (armbian's host deps already include it) and retry."
82+
fi
83+
84+
display_alert "${EXTENSION}: disabling qemu-arm" "/proc/sys/fs/binfmt_misc/qemu-arm → 0" "info"
85+
if ! echo 0 > /proc/sys/fs/binfmt_misc/qemu-arm 2> /dev/null; then
86+
exit_with_error "${EXTENSION}: cannot write to /proc/sys/fs/binfmt_misc/qemu-arm. CAP_SYS_ADMIN missing on host?"
87+
fi
88+
89+
if ! "${armhf_probe}" --help > /dev/null 2>&1; then
90+
echo 1 > /proc/sys/fs/binfmt_misc/qemu-arm 2> /dev/null || true
91+
exit_with_error "${EXTENSION}: host kernel cannot run armhf natively after disabling qemu-arm (no CONFIG_COMPAT?). Restored qemu-arm and aborting; disable this extension on this host."
92+
fi
93+
94+
declare -g _NATIVE_ARMHF_EXTENSION_DISABLED_QEMU_ARM=yes
95+
add_cleanup_handler trap_handler_native_armhf_on_arm64_restore
96+
display_alert "${EXTENSION}: armhf chroot execs will run natively via kernel binfmt_elf" "qemu-arm disabled for this build; will be restored on exit" "info"
97+
}
98+
99+
function trap_handler_native_armhf_on_arm64_restore() {
100+
[[ "${_NATIVE_ARMHF_EXTENSION_DISABLED_QEMU_ARM:-no}" == "yes" ]] || return 0
101+
[[ -e /proc/sys/fs/binfmt_misc/qemu-arm ]] || return 0
102+
if echo 1 > /proc/sys/fs/binfmt_misc/qemu-arm 2> /dev/null; then
103+
display_alert "${EXTENSION}: qemu-arm restored" "/proc/sys/fs/binfmt_misc/qemu-arm → 1" "info"
104+
else
105+
display_alert "${EXTENSION}: failed to restore qemu-arm" "manual fix: echo 1 | sudo tee /proc/sys/fs/binfmt_misc/qemu-arm" "wrn"
106+
fi
107+
unset _NATIVE_ARMHF_EXTENSION_DISABLED_QEMU_ARM
108+
}

‎lib/functions/host/prepare-host.sh‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,14 @@ function prepare_host_noninteractive() {
9797

9898
prepare_host_binfmt_qemu # in qemu-static.sh as most binfmt/qemu logic is there now
9999

100+
call_extension_method "host_binfmt_ready" <<- 'HOST_BINFMT_READY'
101+
*run after the host's binfmt_misc registrations for the target architecture have been set up*
102+
At this point qemu-${arch} is registered and enabled (or the build is a native one).
103+
Use this hook to mutate binfmt_misc — for example, an opt-in extension can disable
104+
qemu-${arch} here so that the kernel's native binfmt_elf path handles the target
105+
architecture for the rest of the build, bypassing qemu emulation overhead.
106+
HOST_BINFMT_READY
107+
100108
# @TODO: rpardini: this does not belong here, instead with the other templates, pre-configuration.
101109
[[ ! -f "${USERPATCHES_PATH}"/customize-image.sh ]] && run_host_command_logged cp -pv "${SRC}"/config/templates/customize-image.sh.template "${USERPATCHES_PATH}"/customize-image.sh
102110
[[ ! -f "${USERPATCHES_PATH}"/config-example.conf ]] && run_host_command_logged cp -pv "${SRC}"/config/templates/config-example.conf.template "${USERPATCHES_PATH}"/config-example.conf

0 commit comments

Comments
 (0)