Skip to content

Export a matrix of resolve+python versions #15817

Open
@cognifloyd

Description

@cognifloyd

Is your feature request related to a problem? Please describe.
I would like to export venvs for multiple python versions in each of my resolves.

I keep an one instance of every major python version (2.7 and 3.4+) available on my dev box. When resolving the interpreter, pants/pex defaults to the lowest available that meets the given constraints. That's fine, until I want to work with a particular python version.

Describe the solution you'd like
Enhance python venv exports so that they multiple python versions can be exported for each resolve.

The default behavior is fine: one venv for each resolve using the lowest python version in the interpreter constraints.
In the python backend, add two options to the export subsystem that allows a user to:

  • limit the exported venvs to a selected (set of) resolve(s), and
    • filter targets to only include the indicated resolve(s)
  • define which python interpreter to use.
    • If only one resolve is given: Error if that interpreter does not match the resolve's interpreter constraints.
    • If multiple resolves are given (like the default which does all possible resolves): Warn and ignore any resolves that don't match the given interpreter

Describe alternatives you've considered
It could create a venv for each python version that is (a) available locally, and (b) matches the interpreter constraints.
That seems a bit too shotgun. I'm happy to guide pants as to which venvs I want created.

Additional context

./pants export ::

Here is the pants.toml in the project I'm using to play with this: https://github.com/st2sandbox/st2/blob/pants/pants.toml#L112-L138
It's got multiple resolves, one of which has a different set of intepreter constraints (pants-plugins follows pants' constraints, not my project's constraints).

I don't need tool resolves to have this behavior (at this point).

So, I want to get this matrix of virtualenvs (only some can be exported today, indicated by + in the today column):

resolve python today
python-default 3.6 +
python-default 3.8
pants-plugins 3.7 +
pants-plugins 3.8
pants-plugins 3.9
pylint_plugins 3.6 +
pylint_plugins 3.8

This is where the matrix is generated today:

venvs = await MultiGet(
Get(
ExportResult,
_ExportVenvRequest(resolve if python_setup.enable_resolves else None, tuple(tgts)),
)
for resolve, tgts in resolve_to_root_targets.items()
)

I'm not sure how to add extra cli args for the export goal or how to determine the python version to use here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    • Status

      No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions