[MASS CI BREAKAGE] cache: pip in actions/setup-python breaks all Windows jobs with PIP_NO_PYTHON_VERSION_WARNING=1 set @ GitHub Actions CI/CD

[webknjaz]

The problem

4 days ago, I've discovered a nasty bug that's hitting all Windows jobs where setup-python is set up to use the built-in cache control: actions/setup-python#1034 (comment).

Note

TL;DR whenever the cache input is set, it crashes on every Windows job.

There's no workaround short of disabling this feature. In addition to that, the caching mechanism is imperfect. I've been making my caches take into account Python runtime/ABI stability for the past several years, which is important in periods when Python alpha versions get updated and pip caches wheels build against earlier alphas with ABI changing slightly.

The path forward

Tip

With that above in mind, the recommendation is to implement the following in all the repos, avoding the built-in mechanism: aio-libs/frozenlist#622 + aio-libs/frozenlist#633.

The solution is common enough that I've put it into a composite action for reuse: https://github.com/re-actors/cache-python-deps. The links above show a bit more sophisticated integration but sample generic use would be

- name: Restore pip cache
  uses: re-actors/cache-python-deps@release/v1
  with:
    cache-key-for-dependency-files: >-
      ${{
        hashFiles(
          '.pre-commit-config.yaml',
          'requirements/**',
          'tox.ini',
          'tox.toml',
          'pyproject.toml'
        )
      }}

provided that it's invoked after setup-python and after a sort of checkout (or similar).

cc @achimnol @bdraco @Dreamsorcerer @hellysmile @jettify @mjpieters @Nothing4You @pohmelie @samuelcolvin @aio-libs/admins