missing neuron-nightly for macOS M1/M2

[ramcdougal]

neuron-nightly does not currently have any versions for M1/M2 macs.

https://pypi.org/project/NEURON-nightly/#files

Possibly related to #2358

[pramodk]

@ramcdougal : one of the limitation/reason is that we don't have Apple M1 hardware freely available in Azure or GitHub CI.

A side note: we have apple m1 machine for testing/PR builds. But having self-hosted runner is another task and would require some work + discussion.

[nrnhines]

I know that for the pkg installers is it straightforward to cross compile arm64, x86_64, or universal on either arch.
This is done by the nrn/bldnrnmacpkgcmake.sh script. I'm guessing that by analogy the wheel builds can accomplish the same thing if all the relevant libraries/programs are universal. eg.

lipo -archs `which python3.11`      
x86_64 arm64

[nrnhines]

I started a branch (hines/universal2-wheel) that gets a long way toward creating a Mac universal2 wheel. I.e.

 build % temp.macosx-10.9-arm64-cpython-312/bin/nrniv -c 'nrnpython("import platform;print(platform.machine())")'
NEURON -- VERSION 9.0a-354-g93c7d2c22+ hines/universal2-wheel (93c7d2c22+) 2024-09-25
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2022
See http://neuron.yale.edu/neuron/credits

arm64
	1 
build % arch -x86_64 temp.macosx-10.9-arm64-cpython-312/bin/nrniv -c 'nrnpython("import platform;print(platform.machine())")'
NEURON -- VERSION 9.0a-354-g93c7d2c22+ hines/universal2-wheel (93c7d2c22+) 2024-09-25
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2022
See http://neuron.yale.edu/neuron/credits

x86_64
	1 

However the bash packaging/python/build_wheels.bash osx 3.12 "" &>3.12-output errors out at the delocate part of the build

INFO:delocate.tools:Sanitize: Deleting rpath '@loader_path' from '/private/var/folders/gf/cr95gnz92l9fyz726ld3vtk00000gn/T/tmpqru_nxb_/wheel/neuron/.data/share/nrn/demo/release/arm64/special'
INFO:delocate.tools:Sanitize: Deleting rpath '/Users/hines/neuron/univwheel/build/cmake_install/lib' from '/private/var/folders/gf/cr95gnz92l9fyz726ld3vtk00000gn/T/tmpqru_nxb_/wheel/neuron/.data/share/nrn/demo/release/arm64/special'
INFO:delocate.tools:Sanitize: Deleting rpath '/Users/hines/neuron/univwheel/build/cmake_install/lib' from '/private/var/folders/gf/cr95gnz92l9fyz726ld3vtk00000gn/T/tmpqru_nxb_/wheel/neuron/.data/share/nrn/demo/release/arm64/.libs/libnrnmech.so'
Fixing: dist/NEURON_nightly-9.0a0-cp312-cp312-macosx_10_15_arm64.whl
Traceback (most recent call last):
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/bin/delocate-wheel", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/delocate/cmd/delocate_wheel.py", line 116, in main
    copied = delocate_wheel(
             ^^^^^^^^^^^^^^^
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/delocate/delocating.py", line 1090, in delocate_wheel
    out_wheel_fixed = _check_and_update_wheel_name(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/delocate/delocating.py", line 914, in _check_and_update_wheel_name
    new_name, problematic_files = _calculate_minimum_wheel_name(
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/delocate/delocating.py", line 816, in _calculate_minimum_wheel_name
    for arch, version in _get_macos_min_version(lib):
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/delocate/delocating.py", line 619, in _get_macos_min_version
    for header in MachO(dylib_path).headers:
                  ^^^^^^^^^^^^^^^^^
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/macholib/MachO.py", line 121, in __init__
    self.load(fp)
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/macholib/MachO.py", line 131, in load
    self.load_fat(fh)
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/macholib/MachO.py", line 148, in load_fat
    self.load_header(fh, arch.offset, arch.size)
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/macholib/MachO.py", line 170, in load_header
    raise ValueError("Unknown Mach-O header: 0x%08x in %r" % (header, fh))
ValueError: Unknown Mach-O header: 0x213c6172 in <_io.BufferedReader name='/private/var/folders/gf/cr95gnz92l9fyz726ld3vtk00000gn/T/tmpqru_nxb_/wheel/neuron/.data/lib/libinterviews.a'>

[nrnhines]

@pramodk per our Skype discussion...
matthew-brett/delocate#229 "Fat binaries of static libraries are not supported in the wheel" seems relevant. The error message above mentions libinterviews.a and

lipo -archs build/lib.macosx-10.9-arm64-cpython-312/neuron/.data/lib/libinterviews.a 
x86_64 arm64

chatgpt informs me that whl files are zip files. So, though not recommended, I removed

% unzip -t dist/*.whl|grep '.a  '
    testing: neuron/.data/lib/libfmt.a   OK
    testing: neuron/.data/lib/libinterviews.a   OK
    testing: neuron/.data/lib/libunidraw.a   OK

and "successfully"? delocated with

univwheel % nrn_build_venv312_-2632554/bin/delocate-wheel -w wheelhouse dist/*.whl

which generated

 % ls wheelhouse
NEURON_nightly-9.0a1.dev355+g7fb8eff0d-cp312-cp312-macosx_11_0_arm64.whl

[nrnhines]

The hines/universal2-wheel branch now creates a wheel. E.g.

 % ls wheelhouse 
NEURON_nightly-9.0a1.dev355+g7fb8eff0d.d20241019-cp312-cp312-macosx_11_0_arm64.whl

That is installable in a python virtual environment and runnable with either a arch arm64 or ```arch x86_641 prefix. Remaining issues are

The wheel name should contain the term universal2 instead of arm64 (this was created on an Apple M1).

nrnivmodl is creating an arm64 subfolder.
nrnivmodl shouid create either arm64 or x86_64 subfolders depending on which architecture it is built for.
Or perhaps not.
Presently it is creating universal2 libnrnmech.dylib shared libraries

I'm experiencing a problem with NEURONMainMenu/Quit when launching python.

[nrnhines]

This is informational. I was trying to see if there is a name convention for universal2 wheel architectures. Python.org itself just leaves out the machine type. e.g.

https://www.python.org/ftp/python/3.12.7/python-3.12.7-macos11.pkg

I noticed a discussion at python-pillow/pillow-wheels#359 that indicates using the token "universal2", however, that discussion ended with decision to have two wheels, arm64, and x86_64.

I found it interesting that they also mention it is possible to fuse arm64 and x86_64 wheels into a single wheel. e.g.

from delocate.fuse import fuse_wheels
fuse_wheels('Pillow-9.4.0-2-cp39-cp39-macosx_10_10_x86_64.whl', 'Pillow-9.4.0-cp39-cp39-macosx_11_0_arm64.whl', 'Pillow-9.4.0-cp39-cp39-macosx_11_0_universal2.whl')

Now called delocate-merge. See https://github.com/matthew-brett/delocate Making dual-architecture binaries

[JCGoran]

Possible fix would be #3321, though the automatic uploading part (and consequently, up-to-date nightly wheels on PyPI for all platforms) hasn't been implemented yet.