Isolated build fails with generic `StopIteration` when pip is not provisioned in venv

[d-glenz]

Thank you for providing feedback on Python packaging!

To help us help you, please fill out as much of the following as you can. If a question is not relevant, feel free to skip it.

1. What is your operating system and version?

CentOS 7.6.1810

2. What is your Python version?

Python 3.6.5

3. What version of pip do you have?

pip-21.0.1

4. If following an online tutorial or guide, please provide a link to the page or section giving you trouble:

https://packaging.python.org/tutorials/packaging-projects/

5. Could you describe your issue in as much detail as possible?

Created a sample-project according to the tutorial.
Set up a virtual environment, upgraded pip to the latest and installed build-0.3.1.post1
Ran python -m build --sdist --wheel to create both a wheel and an sdist, and then encountered this:

Traceback (most recent call last):
  File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/dominikglenz/.virtualenvs/tmp-5bea59413b8ffc04/lib/python3.6/site-packages/build/__main__.py", line 214, in <module>
    main(sys.argv[1:], 'python -m build')
  File "/home/dominikglenz/.virtualenvs/tmp-5bea59413b8ffc04/lib/python3.6/site-packages/build/__main__.py", line 206, in main
    build_package(args.srcdir, outdir, distributions, config_settings, not args.no_isolation, args.skip_dependencies)
  File "/home/dominikglenz/.virtualenvs/tmp-5bea59413b8ffc04/lib/python3.6/site-packages/build/__main__.py", line 94, in build_package
    _build_in_isolated_env(builder, outdir, distributions, config_settings)
  File "/home/dominikglenz/.virtualenvs/tmp-5bea59413b8ffc04/lib/python3.6/site-packages/build/__main__.py", line 52, in _build_in_isolated_env
    with IsolatedEnvBuilder() as env:
  File "/home/dominikglenz/.virtualenvs/tmp-5bea59413b8ffc04/lib/python3.6/site-packages/build/env.py", line 91, in __enter__
    executable, scripts_dir = _create_isolated_env_venv(self._path)
  File "/home/dominikglenz/.virtualenvs/tmp-5bea59413b8ffc04/lib/python3.6/site-packages/build/env.py", line 203, in _create_isolated_env_venv
    pip_distribution = next(iter(metadata.distributions(name='pip', path=[purelib])))
StopIteration

It looks like build did not add pip to the /tmp/build-env-*/ site-packages.

[layday]

Transferred to the build tracker for greater visibility.

[layday]

This could have one of a number of causes:

• the purelib path is incorrect
• importlib_metadata is not able to find any meta path finders which implement find_distributions
• pip was genuinely not installed (least likely)

[d-glenz]

Looked into this a bit more and found that after venv.EnvBuilder(with_pip=True).create(path), so after the isolated build environment is created, path contains:

├── bin
│   ├── activate
│   ├── activate.csh
│   ├── activate.fish
│   ├── python
│   └── python3
├── include
├── lib
│   └── python3.6
│       └── site-packages
├── lib64
│   └── python3.6
│       └── site-packages
└── pyvenv.cfg

Notice how bin does not contain either python3.6 or pip, and site-packages in both lib/python3.6 and lib64/python3.6 are empty.

[FFY00]

Looks like CentOS patches the venv module then 😕

You can install build with the virtualenv extra, build[virtualenv], which should fix this. We should make virrtualenv a hard requirement on CentOS though.

[uranusjr]

I don’t think there’s a good way to add hard dependencies based on Linux distribution variants.

I think this is less about patching venv, but that on most Linux distributions, pip is a separate package from the base Python installation, and ensurepip does not work without the user installing pip from the system package first. There has been long-running tension between Linux distribution maintainers and Python packaging tools; distributions really don’t want to put pip into the base Python package (for valid reasons), but most Python packaging tools want to be able to boostrap pip (also for valid reasons).

I think the best build can do here is to detect pip is correctly populated by venv, and error out (hinting the virtualenv extra) if venv is broken.

[layday]

Should ensurepip not error then if it can't install pip if pip is not installed system-wide? I was under the impression that it did on Debian.

[uranusjr]

That depends on how CentOS packages and patches things. IIRC Debian just outright removed ensurepip without noticing it’s used by venv, which (fortunately in a sense?) made the error very visible.

[freddrake]

I see this happening when running build under tox. The underlying Python is a clean installation from sources, not a distro-provided bastardization.

Requiring build[virtualenv] instead of build makes it work, but I only found out about the virtualenv extra from reading this issue.