numpy still fails on 16 KB page devices: chaquopy-openblas and chaquopy-libgfortran are not 16 KB aligned

[ryouwatanaka-lab]

Summary

On a 16 KB page size device, import numpy fails even with Python 3.13, which the 17.0.0 release notes recommend for 16 KB compatibility.

The numpy wheels themselves are fine — numpy-1.26.2-0-cp313-cp313-android_24_x86_64.whl is built with 16 KB alignment. The problem is two of its dependencies, which are still the wheels built on 2020-01-29 and are 4 KB aligned:

• chaquopy-openblas 0.2.20
• chaquopy-libgfortran 4.9

Both are published as py3-none-android_*, so they are shared by every Python version. Selecting a newer Python cannot avoid them.

Chaquopy version

17.0.0

Device / emulator

Emulator, system-images;android-35;google_apis_playstore_ps16k;x86_64 (Android 15, x86_64, adb shell getconf PAGE_SIZE = 16384).

The same app runs correctly on system-images;android-35;google_apis_playstore;x86_64 (PAGE_SIZE = 4096).

The arm64-v8a wheels have the same 4 KB alignment (checked in the wheels), though we have not tested on an arm64 device.

Also relevant: AGP 8.5.2, Gradle 8.7, Kotlin 2.0.21, minSdk 26, compileSdk 34. Reproduced with both version = "3.11" and version = "3.13".

Code

app/build.gradle.kts:

android {
    defaultConfig {
        minSdk = 26
        ndk { abiFilters += listOf("arm64-v8a", "x86_64") }
    }
}

chaquopy {
    defaultConfig {
        version = "3.13"
        buildPython("py", "-3.13")
        pip { install("numpy") }
    }
}

Python side — this is enough to reproduce:

import numpy

Reproduce

1. New app with the configuration above.
2. Run on a 16 KB page size AVD.
3. import numpy raises:

ImportError: Importing the numpy C-extensions failed.
Original error was: dlopen failed: library "libopenblas.so" not found:
  needed by /data/data/<pkg>/files/chaquopy/AssetFinder/requirements/numpy/core/_multiarray_umath.so
  in namespace clns-7

Loading the libraries explicitly with ctypes.CDLL shows the underlying cause:

libgfortran.so.3: dlopen failed: empty/missing DT_HASH/DT_GNU_HASH in
  "/data/data/<pkg>/files/chaquopy/AssetFinder/requirements/chaquopy/lib/libgfortran.so.3"
  (new hash type from the future?)
libopenblas.so:   dlopen failed: library "libgfortran.so.3" not found

Evidence

Maximum p_align over the PT_LOAD segments of each shipped .so:

file	wheel build date	p_align	loads on 16 KB
numpy/core/_multiarray_umath.so (cp313)	2024-10-21	16384	yes
numpy/core/_multiarray_umath.so (cp311)	2023-12-07	4096	no
chaquopy/lib/libc++_shared.so (libcxx 180000)	—	16384	yes
chaquopy/lib/libgfortran.so.3 (libgfortran 4.9)	2020-01-29	4096	no
chaquopy/lib/libopenblas.so (openblas 0.2.20)	2020-01-29	4096	no

On the device, of the three libraries extracted to AssetFinder/requirements/chaquopy/lib/, only the 16 KB aligned libc++_shared.so loads. The two 4 KB aligned ones do not.

Note that the hash tables in these files are actually intact — libgfortran.so.3 has a valid DT_HASH with nbucket=1031 — so the linker's "empty/missing DT_HASH/DT_GNU_HASH" message looks like a symptom of the alignment mismatch rather than a malformed file.

The libopenblas.so installed by a Chaquopy 17.0.0 build is byte-identical to the 2020 wheel at https://chaquo.com/pypi-13.1/chaquopy-openblas/ — SHA-256 516cd69ce4023d3eae58f89379dcd1451a7e106cf2ae85dbebd4a0c2d453be0c (13,270,032 bytes, android_21_x86_64).

Why Python 3.13 does not help

numpy-1.26.2-0-cp313-cp313-android_24_x86_64.whl declares:

Requires-Dist: chaquopy-openblas (>=0.2.20)
Requires-Dist: chaquopy-libcxx (>=11000)

chaquopy-libcxx has a 16 KB build (180000), but chaquopy-openblas has only 0.2.20 from 2020, and it is a py3-none-* wheel — not selectable by Python version. So the cp313 numpy wheel, despite being 16 KB ready itself, still pulls in a 4 KB libopenblas.so and fails to import.

Downgrading to numpy==1.23.3 does not help either: it resolves to the same chaquopy-openblas 0.2.20 and chaquopy-libgfortran 4.9.

Request

Rebuild chaquopy-openblas and chaquopy-libgfortran with -Wl,-z,max-page-size=16384. That alone would make numpy work on 16 KB devices on every Python version, without needing new numpy wheels.

Workaround (for anyone finding this)

Restrict to 4 KB page size devices. adb shell getconf PAGE_SIZE returns 4096 there, and numpy works normally.