Spacy on m1 pro chip doesn't work

[maroxtn]

When following the instructions of installing Spacy on the m1 pro chip, I got an error.
I tried to build it from source, install it in a new virtual environment but the same error persists.

How to reproduce the behaviour

pip install -U pip setuptools wheel
pip install -U 'spacy[apple]'
python -m spacy download en_core_web_sm

When running the third command, I get this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/homebrew/lib/python3.9/site-packages/spacy/__init__.py", line 11, in <module>
    from thinc.api import prefer_gpu, require_gpu, require_cpu  # noqa: F401
  File "/opt/homebrew/lib/python3.9/site-packages/thinc/api.py", line 2, in <module>
    from .initializers import normal_init, uniform_init, glorot_uniform_init, zero_init
  File "/opt/homebrew/lib/python3.9/site-packages/thinc/initializers.py", line 4, in <module>
    from .backends import Ops
  File "/opt/homebrew/lib/python3.9/site-packages/thinc/backends/__init__.py", line 8, in <module>
    from .cupy_ops import CupyOps, has_cupy
  File "/opt/homebrew/lib/python3.9/site-packages/thinc/backends/cupy_ops.py", line 19, in <module>
    from .numpy_ops import NumpyOps
  File "thinc/backends/numpy_ops.pyx", line 1, in init thinc.backends.numpy_ops
ImportError: dlopen(/opt/homebrew/lib/python3.9/site-packages/cymem/cymem.cpython-39-darwin.so, 0x0002): tried: '/opt/homebrew/lib/python3.9/site-packages/cymem/cymem.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/cymem.cpython-39-darwin.so' (no such file), '/usr/lib/cymem.cpython-39-darwin.so' (no such file)

Your Environment

• Operating System: MacOs 12.01
• Python Version Used: 3.9.10
• spaCy Version Used: latest
• Environment Information: m1 MacBook Pro 14 inch

[adrianeboyd]

It sounds like your python environment has accidentally ended up with a mix of x86_64 and arm64 packages. (Universal binaries can make it too easy to get things mixed up, although off the top of my head I would have expected the homebrew python and the packages in in /opt/homebrew/lib/python3.9/site-packages/ to be arm64-only.)

Double-check the platform with this command, it should be arm64:

/opt/homebrew/bin/python3.9 -c "import platform; print(platform.machine())"

Try starting from scratch in a new venv without using any previously cached wheels:

/opt/homebrew/bin/python3.9 -m venv .venv
source .venv/bin/activate
python -m pip install -U pip setuptools wheel
python -m pip install 'spacy[apple]' --no-cache-dir

Does that lead to a working python -m spacy info in this venv?

[adrianeboyd]

Let me move this to the installation section of the discussion board...