Open
Description
Description
When running the pip's test suite with PyPy3.10 7.3.15 on Gentoo (and older versions), lots of tests fail. Upon investigating, I've discovered that pip operates on the test venv rather than the tempdirs, like it does for CPython.
For example:
$ nox -e test-pypy3 -- -x
nox > Running session test-pypy3
nox > Creating virtual environment (virtualenv) using pypy3 in .nox/test-pypy3
nox > python tools/protected_pip.py wheel -w tests/data/common_wheels -r tests/requirements-common_wheels.txt
nox > python -m pip install setuptools
nox > python setup.py sdist --formats=zip --dist-dir /tmp/pip-24.0/.nox/test-pypy3/sdist
nox > python tools/protected_pip.py install /tmp/pip-24.0/.nox/test-pypy3/sdist/pip-24.0.zip
nox > python tools/protected_pip.py install -r tests/requirements.txt
nox > pytest -x
========================================================= test session starts =========================================================
platform linux -- Python 3.10.13[pypy-7.3.15-final], pytest-8.0.0, pluggy-1.4.0
rootdir: /tmp/pip-24.0
configfile: setup.cfg
plugins: xdist-3.5.0, rerunfailures-13.0, cov-4.1.0
collected 2545 items
tests/functional/test_bad_url.py . [ 0%]
tests/functional/test_broken_stdout.py ... [ 0%]
tests/functional/test_build_env.py ....F
============================================================== FAILURES ===============================================================
______________________________________________________ test_build_env_isolation _______________________________________________________
script = <tests.lib.PipTestEnvironment object at 0x00007fe02d042d40>
@pytest.mark.usefixtures("enable_user_site")
def test_build_env_isolation(script: PipTestEnvironment) -> None:
# Create dummy `pkg` wheel.
pkg_whl = create_basic_wheel_for_package(script, "pkg", "1.0")
# Install it to site packages.
script.pip_install_local(pkg_whl)
# And a copy in the user site.
> script.pip_install_local("--ignore-installed", "--user", pkg_whl)
tests/functional/test_build_env.py:241:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/lib/__init__.py:745: in pip_install_local
return self.pip(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <tests.lib.PipTestEnvironment object at 0x00007fe02d042d40>
cwd = PosixPath('/tmp/pytest-of-mgorny/pytest-0/test_build_env_isolation0/workspace/scratch'), allow_stderr_error = False
allow_stderr_warning = False, allow_error = False, args = ('python', '-m', 'pip', 'install', '--no-index', '--find-links', ...)
kw = {'expect_stderr': True}, expect_error = None
def run(
self,
*args: str,
cwd: Optional[StrPath] = None,
allow_stderr_error: Optional[bool] = None,
allow_stderr_warning: Optional[bool] = None,
allow_error: bool = False,
**kw: Any,
) -> TestPipResult:
"""
:param allow_stderr_error: whether a logged error is allowed in
stderr. Passing True for this argument implies
`allow_stderr_warning` since warnings are weaker than errors.
:param allow_stderr_warning: whether a logged warning (or
deprecation message) is allowed in stderr.
:param allow_error: if True (default is False) does not raise
exception when the command exit value is non-zero. Implies
expect_error, but in contrast to expect_error will not assert
that the exit value is zero.
:param expect_error: if False (the default), asserts that the command
exits with 0. Otherwise, asserts that the command exits with a
non-zero exit code. Passing True also implies allow_stderr_error
and allow_stderr_warning.
:param expect_stderr: whether to allow warnings in stderr (equivalent
to `allow_stderr_warning`). This argument is an abbreviated
version of `allow_stderr_warning` and is also kept for backwards
compatibility.
"""
if self.verbose:
print(f">> running {args} {kw}")
cwd = cwd or self.cwd
if sys.platform == "win32":
# Partial fix for ScriptTest.run using `shell=True` on Windows.
args = tuple(re.sub("([&|<>^])", r"^\1", str(a)) for a in args)
if allow_error:
kw["expect_error"] = True
# Propagate default values.
expect_error = kw.get("expect_error")
if expect_error:
# Then default to allowing logged errors.
if allow_stderr_error is not None and not allow_stderr_error:
raise RuntimeError(
"cannot pass allow_stderr_error=False with expect_error=True"
)
allow_stderr_error = True
elif kw.get("expect_stderr"):
# Then default to allowing logged warnings.
if allow_stderr_warning is not None and not allow_stderr_warning:
raise RuntimeError(
"cannot pass allow_stderr_warning=False with expect_stderr=True"
)
allow_stderr_warning = True
if allow_stderr_error:
if allow_stderr_warning is not None and not allow_stderr_warning:
raise RuntimeError(
"cannot pass allow_stderr_warning=False with "
"allow_stderr_error=True"
)
# Default values if not set.
if allow_stderr_error is None:
allow_stderr_error = False
if allow_stderr_warning is None:
allow_stderr_warning = allow_stderr_error
# Pass expect_stderr=True to allow any stderr. We do this because
# we do our checking of stderr further on in check_stderr().
kw["expect_stderr"] = True
# Ignore linter check
# B026 Star-arg unpacking after a keyword argument is strongly discouraged
> result = super().run(cwd=cwd, *args, **kw) # noqa: B026
E AssertionError: Script returned code: 1
tests/lib/__init__.py:706: AssertionError
-------------------------------------------------------- Captured stdout call ---------------------------------------------------------
Script result: python -m pip install --no-index --find-links file:///tmp/pip-24.0/tests/data/packages --ignore-installed --user /tmp/pytest-of-mgorny/pytest-0/test_build_env_isolation0/workspace/scratch/pkg-1.0-py2.py3-none-any.whl
return code: 1
-- stderr: --------------------
ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
========================================================== warnings summary ===========================================================
.nox/test-pypy3/lib/pypy3.10/site-packages/pip/_internal/metadata/pkg_resources.py:8
/tmp/pip-24.0/.nox/test-pypy3/lib/pypy3.10/site-packages/pip/_internal/metadata/pkg_resources.py:8: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
from pip._vendor import pkg_resources
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================= short test summary info =======================================================
FAILED tests/functional/test_build_env.py::test_build_env_isolation - AssertionError: Script returned code: 1
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=============================================== 1 failed, 8 passed, 1 warning in 40.88s ===============================================
nox > Command pytest -x failed with exit code 1
nox > Session test-pypy3 failed.
$ ls .nox/test-pypy3/lib/pypy3.10/site-packages/
bar freezegun pluggy six-1.16.0.dist-info
bar-3.0.dist-info freezegun-1.4.0.dist-info pluggy-1.4.0.dist-info six.py
coverage iniconfig __pycache__ tomli
coverage-7.4.1.dist-info iniconfig-2.0.0.dist-info py.py tomli-2.0.1.dist-info
cryptography installer _pytest tomli_w
cryptography-42.0.2.dist-info installer-0.7.0.dist-info pytest tomli_w-1.0.0.dist-info
dateutil markupsafe pytest-8.0.0.dist-info virtualenv
distlib MarkupSafe-2.1.5.dist-info pytest_cov virtualenv-20.25.0.dist-info
distlib-0.3.8.dist-info other pytest_cov-4.1.0.dist-info _virtualenv.pth
_distutils_hack other-0.5.dist-info pytest-cov.pth _virtualenv.py
distutils-precedence.pth packaging pytest_rerunfailures-13.0.dist-info werkzeug
exceptiongroup packaging-23.2.dist-info pytest_rerunfailures.py werkzeug-3.0.1.dist-info
exceptiongroup-1.2.0.dist-info pip pytest_xdist-3.5.0.dist-info wheel
execnet pip-24.0.dist-info python_dateutil-2.8.2.dist-info wheel-0.42.0.dist-info
execnet-2.0.2.dist-info pkg scripttest-1.3.dist-info wheel-0.42.0.virtualenv
filelock pkg-1.0.dist-info scripttest.py xdist
filelock-3.13.1.dist-info pkg_resources setuptools
foo platformdirs setuptools-69.0.3.dist-info
foo-2.0.dist-info platformdirs-4.2.0.dist-info setuptools-69.0.3.virtualenvNote bar and foo packages. If I let the test suite run longer, soon enough pip actually gets uninstalled from there.
Now, I don't know why it happens on our install and not on CI. We aren't patching PyPy in any relevant way. The install is reasonably straightforward. It involves, roughly:
/usr/bin/pypy3 -> pypy3.10
/usr/bin/pypy3.10 -> pypy3.10-c-7.3.15
/usr/bin/pypy3.10-c-7.3.15
/usr/lib/pypy3.10/**
/usr/include/pypy3.10/**
I'd be glad to debug it if you give me some hints.
Expected behavior
Most of the tests passing, and not wreaking havoc in venv.
pip version
24.0
Python version
PyPy3.10 7.3.15
OS
Gentoo Linux amd64
How to Reproduce
- Get a Gentoo install ;-).
emerge pypy3 noxnox -e test-pypy3
Output
No response
Code of Conduct
- I agree to follow the PSF Code of Conduct.