Unknown scheme key: include

[hexagonrecursion]

pip version

21.0.1

Python version

3.9.1

OS

Fedora release 32 (Thirty Two)

Additional information

No response

Description

When a wheel has an include directory in its .data directory pip install produces an error:

ERROR: For req: bug==1.0.0. Unknown scheme key used in /home/user/test/bug-1.0.0-py2.py3-none-any.whl: include (for file 'bug-1.0.0.data/include/bug.h'). .data directory contents should be in subdirectories named with a valid scheme key (data, headers, platlib, purelib, scripts)

PEP 427 https://www.python.org/dev/peps/pep-0427/#file-contents (emphasis mine)

{distribution}-{version}.data/ contains one subdirectory for each non-empty install scheme key not already covered, where the subdirectory name is an index into a dictionary of install paths (e.g. data, scripts, include, purelib, platlib).

Expected behavior

Either the PEP is wrong and should say headers instead of include or pip should accept include as valid

How to Reproduce

1. Install the attached file: pip install bug-1.0.0-py2.py3-none-any.whl
   https://drive.google.com/file/d/1fYzqyV5cBBAp-ILBceEZyFPnE2bS3ug0/view?usp=sharing

Output

$ pip install bug-1.0.0-py2.py3-none-any.whl 
Processing ./bug-1.0.0-py2.py3-none-any.whl
Installing collected packages: bug
ERROR: For req: bug==1.0.0. Unknown scheme key used in /home/user/test/bug-1.0.0-py2.py3-none-any.whl: include (for file 'bug-1.0.0.data/include/bug.h'). .data directory contents should be in subdirectories named with a valid scheme key (data, headers, platlib, purelib, scripts)

Code of Conduct

• I agree to follow the PSF Code of Conduct

[uranusjr]

Seems like there’s a bug in the spec.

In Installing a wheel 'distribution-1.0-py32-none-any.whl':

Move each subtree of distribution-1.0.data/ onto its destination path. […] distribution-1.0.data/(purelib|platlib|headers|scripts|data).

But in File contents:

{distribution}-{version}.data/ contains one subdirectory for each non-empty install scheme key […] (e.g. data, scripts, include, purelib, platlib).

Implementation-wise, distutils.commands.install.SCHEME_KEYS (which the PEP based the name list on) is ('purelib', 'platlib', 'headers', 'scripts', 'data'), but the more modern sysconfig.get_path_names() (which should eventually replace distutils) returns ('stdlib', 'platstdlib', 'purelib', 'platlib', 'include', 'scripts', 'data'). So we need to fix the PEP (and possibly pip’s implementation, according to how the PEP is corrected.

cc @dholth for thoughts.

[dholth]

@uranusjr it's a typo that we have include instead of headers in the File Contents section. It was meant to include values from SCHEME_KEYS only.

Obviously we will have to map equivalent new names to the older names probably forever.

I've always wanted to do something about unknown keys. We need to be able to add new keys over time. Installers that don't know about those keys should do something with them besides throw an exception. I might deal with them by leaving the unknown *.data/key directory and its files in site-packages next to the *.dist-info directory.

The problem with that is that the distribution may have trouble finding its files later. We could put the individual mappings of keys to install paths in the *.dist-info/ directory.

[kraj]

I have same issue but with /etc see attached .whl file when tried to install with pip results in similar error.

$ pip install unattended_upgrades-0.1-py3-none-any.whl 
Defaulting to user installation because normal site-packages is not writeable
Processing ./unattended_upgrades-0.1-py3-none-any.whl
Installing collected packages: unattended-upgrades
ERROR: For req: unattended-upgrades==0.1. Unknown scheme key used in /mnt/b/yoe/master/build/tmp/work/cortexa72-yoe-linux/unattended-upgrades/2.6-r0/git/dist/unattended_upgrades-0.1-py3-none-any.whl: etc (for file 'unattended_upgrades-0.1.data/etc/kernel/postinst.d/unattended-upgrades'). .data directory contents should be in subdirectories named with a valid scheme key (data, headers, platlib, purelib, scripts)

unattended_upgrades-0.1-py3-none-any.whl.zip