maturin failed: Failed to normalize python source path `python`

[yesint]

Bug Description

When building a package pymolar from source distribution (downloaded from pypi) maturing fails with error "Failed to normalize python source path python".
If the same package is built locally (without downloading from pypi) everything works correctly.

Your maturin version (maturin --version)

1.8.2

Your Python version (python -V)

3.12.3

Your pip version (pip -V)

24.0

What bindings you're using

pyo3

Does cargo build work?

• Yes, it works

If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?

• Yes

Steps to Reproduce

1. Run $ pip3 install pymolar --verbose. This doesn't work:

$ pip3 install pymolar --verbose
Using pip 24.0 from /home/semen/work/Projects/molar/molar_python/venv/lib/python3.12/site-packages/pip (python 3.12)
Collecting pymolar
  Downloading pymolar-0.8.1.tar.gz (3.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 8.0 MB/s eta 0:00:00
  Running command pip subprocess to install build dependencies
  Collecting maturin<2.0,>=1.5
    Using cached maturin-1.8.2-py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.musllinux_1_1_x86_64.whl.metadata (16 kB)
  Using cached maturin-1.8.2-py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.musllinux_1_1_x86_64.whl (8.2 MB)
  Installing collected packages: maturin
  Successfully installed maturin-1.8.2
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Getting requirements to build wheel ... done
  Running command Preparing metadata (pyproject.toml)
  💥 maturin failed
    Caused by: Failed to normalize python source path `python`
    Caused by: No such file or directory (os error 2)
  Error running maturin: Command '['maturin', 'pep517', 'write-dist-info', '--metadata-directory', '/tmp/pip-modern-metadata-k3a4vbsz', '--interpreter', '/home/semen/work/Projects/molar/molar_python/venv/bin/python3']' returned non-zero exit status 1.
  Checking for Rust toolchain....
  Running `maturin pep517 write-dist-info --metadata-directory /tmp/pip-modern-metadata-k3a4vbsz --interpreter /home/semen/work/Projects/molar/molar_python/venv/bin/python3`
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/semen/work/Projects/molar/molar_python/venv/bin/python3 /home/semen/work/Projects/molar/molar_python/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmprkb4gtab
  cwd: /tmp/pip-install-52ib24gw/pymolar_4793aab43205470abe09961cd7e9cfb8
  Preparing metadata (pyproject.toml) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

2. Building exactly the same package locally without downloading from pypi works fine:
   maturin build && pip install . runs without any problems:

...
Successfully installed pymolar-0.8.1

The version in pypi is generated from exactly the same source as locally compiled one. Here is the command used to upload the package to pypi:

maturin sdist
maturin upload <path reported by previous command>/pymolar-0.8.1.tar.gz -r pypi -u __token__  -p <secret>

[tiran]

The sdist of hf-xet 1.0.3 is affected by the same or at least a similar bug. I have reported the issue as huggingface/xet-core#255

The hf-xet package is in a subdirectory of the xet-core repo. The sdist has the python directory in the wrong place. The sdist has hf_xet-1.0.3/hf_xet/python, but Maturin expects hf_xet-1.0.3/python.

$ curl -OLfq https://files.pythonhosted.org/packages/95/68/4c363b2e62cb3dbe12d2257ba9b22f101384692d4b9727c5f72433472cff/hf_xet-1.0.3.tar.gz
$ tar xf hf_xet-1.0.3.tar.gz
$ cd hf_xet-1.0.3
python3 -m build -w
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - maturin>=1.7,<2.0
* Getting build dependencies for wheel...
* Building wheel...
Running `maturin pep517 build-wheel -i /tmp/build-env-hjqbu4oj/bin/python --compatibility off`
💥 maturin failed
  Caused by: Failed to normalize python source path `python`
  Caused by: No such file or directory (os error 2)
Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/tmp/build-env-hjqbu4oj/bin/python', '--compatibility', 'off'] returned non-zero exit status 1

ERROR Backend subprocess exited when trying to invoke build_wheel

The build succeeds with an empty python directory in the project directory:

$ mkdir python
$ python3 -m build -w
...
Successfully built hf_xet-1.0.3-cp37-abi3-linux_x86_64.whl

[yesint]

In my case the error is no longer present after fixing a discrepancy in module-name in pyproject.toml. However, the error message is still very weird and not helpful at all and it is still not clear why it installs normally locally but doesn't install from sdist.

[tiran]

@yesint Interesting, could you please post a link to the PR that fixed your problem?

[yesint]

@tiran this is not a PR. I've fixed it in my code base (can't find an exact commit now) and now sdist works as expected. I didn't update the PyPi package yet.