Fix crash with pip==25.1, which changed caching for find_all_candidates

[gkreitz]

In pip==25.1, the implementation of caching for find_all_candidates changed, and it no longer exposes any method in its public interface to clear its cache. This causes pip-compile --generate-hashes to crash with an AttributeError.

In this PR, I replace the cache clearing with simply constructing a new object. I'm not sufficiently familiar with pip/pip-tools internals to know if that's a bad idea (it's unclear to me how much state lies in the finder object, beyond the cache), but I felt it was potentially cleaner than the alternative approach (catch the AttributeError and do finder._all_candidates.clear()) suggested in #2176.

Fixes #2176

Contributor checklist

• Included tests for the changes.
• PR title is short, clear, and ready to be included in the user-facing changelog.

Maintainer checklist

• Verified one of these labels is present: backwards incompatible, feature, enhancement, deprecation, bug, dependency, docs or skip-changelog as they determine changelog listing.
• Assign the PR to an existing or new milestone for the target version (following Semantic Versioning).

[sirosen]

I've just been reading through PackagerFinder in pip and I think this fix will work cleanly. I've hit the CI run button and let's see if it works cleanly -- obviously you don't need to add any new tests, since this will hopefully fix already-broken tests! 😄

[sirosen]

I'm going to look into what's going on with all of these failing builds as soon as I have time. I'm pretty sure they have nothing to do with this PR, and are instead caused by some non-reproducible elements in the build envs.

We'll potentially merge this even with CI red; like I said, I'm pretty sure it's right and we can back it out if it turns out to be an issue after we unwind the other CI woes.

[AndydeCleyre]

I think we need to remove the keyword option mix_stderr here in conftest.py:

cli_runner = CliRunner(mix_stderr=False)

AFAIU that option has done nothing for a while and is no longer part of Click.

EDIT:

From Click's 8.2.0 changelog:

Keep stdout and stderr streams independent in CliRunner. Always collect stderr output and never raise an exception. Add a new output stream to simulate what the user sees in its terminal. Removes the mix_stderr parameter in CliRunner. #2522 #2523

[sirosen]

Yes, we will need to handle click 8.2+ changes. Since pip-tools still has reason to support Python 3.8, I think the appropriate short-term solution is to pin to click>=8.1,<8.2. We can potentially follow the guidance in https://click.palletsprojects.com/en/stable/support-multiple-versions/ , but right now pip-tools can be broken unless you pin it's dependencies¹ -- so the priority is to get it fixed before we start making any additional changes.

Handling will depend on how exactly things go as I wade through the space of unconstrained packages in the tests.
Based on what I was most recently looking at in CI, I don't think click is the only or even the primary issue. But pinning it does seem to be helping my local testing exploration, so this might be the right way to start tackling things.

Footnotes

1. There is some inherent humor in this, seeing as pip-tools is primarily useful for making things -- including CI -- highly reproducible... ↩

[AndydeCleyre]

I'd like to clear the way for fixing #2176 (thanks!) and #2131, so have put up #2183.

[sirosen]

This was merged as part of #2195 ! 🚀

Thanks @gkreitz for the PR; we made sure to preserve your authorship on the commit here after rebase.