Add tests for `--platform` with pip's new resolver

[pradyunsg]

Environment

• pip version: master
• Python version: any
• OS: any

Description

We want to add tests, for --platform's behavior with the new resolver (see #8349 (comment) for details on why this is needed).

The cases, as mentioned by @uranusjr (omitting the --unstable-feature=resolver), which will be needed in the actual tests, are:

# Install a wheel from a direct URL with non-host platform.
pip install --platform=fakeplat /wheelhouse/foo-1.0-py3-none-fakeplat.whl

# Install a wheel from a find-links page with non-host platform.
pip install --platform=fakeplat --find-links=./path/to/wheelhouse foo==1.0

# Install a wheel into a target directory from a direct URL with non-host platform.
pip install --platform=fakeplat --target=./target path/to/wheelhouse/foo-1.0-py3-none-fakeplat.whl

# Install a wheel into a target directory from a find-links page with non-host platform.
pip install --platform=fakeplat --target=./target --find-links=./path/to/wheelhouse foo==1.0

Also relevant is @pfmoore's note that:

we'd have to skip for the old resolver - presumably using something like

if not use_new_resolver:
    pytest.xfail("Old resolver is broken")

[deveshks]

Out of curiosity, I took a stab at the first two tests by trying to run the mentioned commands above with the new resolver, but none of them proceeded beyond:

pip/src/pip/_internal/commands/install.py

Line 245 in 445711c

cmdoptions.check_dist_restriction(options, check_target=True)

I tried the following.

def test_install_wheel_direct_url(script):

    wheel_builder = make_wheel("foo", "1.0")
    archive_name = "foo-1.0-py3-none-fakeplat.whl"
    archive_path = script.scratch_path / archive_name
    wheel_builder.save_to(archive_path)

    result = script.pip("install",  
                        "--unstable-feature=resolver", 
                        "--platform","fakeplat", 
                        archive_path)
    print(result)


def test_install_wheel_find_links(script):

    wheel_builder = make_wheel("foo", "1.0")
    archive_name = "foo-1.0-py3-none-fakeplat.whl"
    archive_path = script.scratch_path / archive_name
    wheel_builder.save_to(archive_path)

    result = script.pip("install",  
                        "--unstable-feature=resolver", 
                        "--platform", "fakeplat", 
                        "--find-links",
                        script.scratch_path, "foo==1.0")
    print(result)

And got the following output from the print statement

First Test

------------------------------------------------------------- Captured stdout call --------------------------------------------------------------
Script result: python -m pip install --unstable-feature=resolver --platform fakeplat /private/var/folders/xg/blp845_s0xn093dyrtgy936h0000gp/T/pytest-of-devesh/pytest-0/test_install_wheel_direct_url0/workspace/scratch/foo-1.0-py3-none-fakeplat.whl
  return code: 1
-- stderr: --------------------
ERROR: When restricting platform and interpreter constraints using --python-version, --platform, --abi, or --implementation, either --no-deps must be set, or --only-binary=:all: must be set and --no-binary must not be set (or must be set to :none:).

Second Test

------------------------------------------------------------- Captured stdout call --------------------------------------------------------------
Script result: python -m pip install --unstable-feature=resolver --platform fakeplat --find-links /private/var/folders/xg/blp845_s0xn093dyrtgy936h0000gp/T/pytest-of-devesh/pytest-0/test_install_wheel_find_links0/workspace/scratch foo==1.0
  return code: 1
-- stderr: --------------------
ERROR: When restricting platform and interpreter constraints using --python-version, --platform, --abi, or --implementation, either --no-deps must be set, or --only-binary=:all: must be set and --no-binary must not be set (or must be set to :none:).

Did I miss something here?

[uranusjr]

Oh yeah, one of those needed to be added as well, I forgot when I formatted the commands in the original issue. I’d suggest simply adding --only-binary=:all: since the tests always use wheels anyway.

[deveshks]

Oh yeah, one of those needed to be added as well, I forgot when I formatted the commands in the original issue. I’d suggest simply adding --only-binary=:all: since the tests always use wheels anyway.

Just adding --only-binary=:all: causes both tests to still complain at

pip/src/pip/_internal/cli/cmdoptions.py

Lines 125 to 130 in 445711c

	if check_target:
	if dist_restriction_set and not options.target_dir:
	raise CommandError(
	"Can not use any platform or abi specific options unless "
	"installing via '--target'"
	)

because of

pip/src/pip/_internal/commands/install.py

Line 245 in 445711c

cmdoptions.check_dist_restriction(options, check_target=True)

----------------------------------------------------- Captured stdout call ------------------------------------------------------
Script result: python -m pip install --unstable-feature=resolver --platform fakeplat --only-binary=:all: /private/var/folders/xg/blp845_s0xn093dyrtgy936h0000gp/T/pytest-of-devesh/pytest-0/test_install_wheel_direct_url0/workspace/scratch/foo-1.0-py3-none-fakeplat.whl
  return code: 1
-- stderr: --------------------
ERROR: Can not use any platform or abi specific options unless installing via '--target'

[uranusjr]

I just found a test that actually tries to ensure --target turns of compatibility checks:

@pytest.mark.fails_on_new_resolver
def test_install_nonlocal_compatible_wheel_path(script, data):
    target_dir = script.scratch_path / 'target'

    # Test a full path requirement
    result = script.pip(
        'install',
        '-t', target_dir,
        '--no-index',
        '--only-binary=:all:',
        Path(data.packages) / 'simplewheel-2.0-py3-fakeabi-fakeplat.whl'
    )
    assert result.returncode == SUCCESS

Maybe this plus the --target disabling platform flags mentioned above mean the new resolver should disable the checks after all? 🤦‍♂️

[pfmoore]

I think I looked at this test. I think it's even weirder than that, it's somehow related to --only-binary. I'll try to check the details but it might not be till tomorrow. Didn't one of my recent PRs remove the mark from this test? Maybe it might be worth you rebasing on current master, just in case?

[pfmoore]

Oh yeah, it's #8462 which I haven't merged yet because there's an (unrelated, I think) Windows failure that I can't understand :-(