PEP 518 build requirements cannot be overriden by user

Apparently not, it seems to call pip install --ignore-installed ....
Because the build itself is not be isolated from the environment in
other respects, I'm not sure if this is actually sensible behavior by pip...

If the target computer already has a satisfactory version of numpy, then the build system should use that version. Only if the version is not already installed should pip use an isolated environment.

Related: scipy/scipy#7309

[pv]

Some reasons why the current behavior is bad for Numpy especially:

• Numpy ABI is backward compatible but not forward compatible. If the numpy version pip installs in the "isolated" environment during build is newer than the version present in the system, the built C extensions can segfault if used together with the system numpy.
• Numpy build takes time, so build-requiring it can add unnecessary 10+ min to the install time.

[njsmith]

I responded on the scipy issue before seeing this one: scipy/scipy#7309 (comment)

The most correct solution to the abi issue is to build against the lowest supported numpy version. Building against the currently installed version is a hack that will fail in a number of cases; I mentioned one of them there, but another is that due to pip's wheel caching feature, if you first install scipy into an environment that has the latest numpy, and then later install scipy into an environment that has an older numpy, pip won't even invoke scipy's build system the second time, it'll just install the cached build from last time.

[pv]

Yes, the ABI issue indeed can be handled with specifying the earliest numpy version.

[rgommers]

The lowest supported version is normally Python version dependent (now numpy 1.8.2 is lowest supported, but clearly not for Python 3.6 because 1.8.2 predated Python 3.6 by a long time).

So the specification will then have to be:

numpy=='1.8.2';python_version<='3.4'
numpy=='1.9.3';python_version=='3.5'
numpy=='1.12.1';python_version=='3.6'

I have the feeling not many projects are going to get this right ....

[rgommers]

That still leaves a question of what to do for a not-yet-released Python version. Would you do:

numpy=='1.12.1';python_version>='3.6'

or

numpy;python_version>='3.7'

I'd suspect the first one, but either way you have to guess whether or not an existing version of numpy is going to work with a future Python version. You have to think about it though, if you don't specify anything for Python 3.7 then a build in an isolated venv will break (right?). So then you'd have to cut a new release for a new Python version.

[njsmith]

I guess the not-yet-released-Python issue is sort of the same as anything else about supporting not-yet-released-Python. When developing a library like (say) scipy, you have to make a guess about how future-Python will work, in terms of language semantics, C API changes, ... and if it turns out you guess wrong then you have to cut a new release? I'm not sure there is a really great solution beyond that.

Something that came up during the PEP 518 discussions, and that would be very reasonable, was the idea of having a way for users to manually override some build-requires when building. This is one situation where that might be useful.

[rgommers]

It's a little different in this case - here we use == rather than >= (as typically done in version specifiers in setup.py), which makes it much more critical to guess right.

E.g. if Python 3.7 breaks numpy, then I now need a new numpy release and new releases of every single package that depends on numpy and went with numpy=='1.12.1'.
Normally in version specifiers, you say something like numpy>=x.y.z. Then if the same happens, you need a new numpy release but nothing else.

[pradyunsg]

Yes, the ABI issue indeed can be handled with specifying the earliest numpy version.
<and>
I have the feeling not many projects are going to get this right ....

I don't think there's any way to do "use earliest compatible version" with pip; would it be something useful in this situation?

[rgommers]

@pradyunsg I think in principle yes. Are you thinking about looking at the PyPI classifiers to determine what "earliest compatible" is?

[pradyunsg]

Are you thinking about looking at the PyPI classifiers to determine what "earliest compatible" is?

TBH, I'm not really sure how this would be done. For one, I don't think we have anything other than the PyPI Classifiers for doing something this and I'm skeptical of using those for determining if pip can install a package...

[rgommers]

Yeah that's probably not the most robust mechanism.

[njsmith]

There is a way to specify earliest compatible python version in package metadata. Not the trove classifiers – those are just informational. The IPython folks worked this out because they needed to be able to tell pip not to try to install new IPython on py2.

The problem with this though is that old numpy packages can't contain metadata saying that they don't work with newer python, because by definition we don't know that until after the new python is released. (Also I think the current metadata might just be "minimum python version", not "maximum python version".)

[dstufft]

The current metadata is not minimum or maximum, but a full version specifier which supports >=, >, ==, <``, etc. I suspect the biggest blockers here are:

1. That metadata is relatively new, so hardly anything is using it currently. What do we do if nothing has it, do we just assume everything is compatible and install the oldest version available? That seems unlikely to work but it's also confusing if we suddenly switch from installing the latest to the newest once they upload a version that has that metadata.
2. A project can't know in the future what version of Python it's going to stop working on, pip 9 currently works on Python 3.6, will it work on Python 3.12? I have no idea!

Maximum versions that don't represent a version that already exist are basically impossible to get right except by pure chance. You pretty much always end up either under or over specifying things.

I had always presumed that Numpy would change the wheel requiremnts to reflect the version that it was built against, so that the dependency solver then (theortically until #988 is solved) handles things to ensure there is no version incompatibility related segfaults.

I think the worse case here is you end up installing something new that depends on Numpy and end up having to also install a new Numpy because now you have something that has a numpy>=$LATEST requirement, but since all the old things have a numpy>=$OLDERVERSION requirement, they won't need to be reinstalled, just numpy and the new thing. Combine this with the wheel cache and the fact that Numpy is pretty good about providing wheels for the big 3 platforms, it feels like this isn't going to be a big deal in practice?

Am I missing something?