TST: Locked CI ?

[neutrinoceros]

Given the lack of response on the mailing list, let me retry this in issue form

Reposting from https://mail.python.org/archives/list/yt-dev@python.org/thread/ALJABLQADLBEXB7MZKXI2F5OAUT4UN6Y/

Hi everyone
As you may have noticed, my personal availability to work on compatibility issues for yt has been decreasing. Meanwhile, dependency hell keeps breaking loose . I'm growing wary of this kind of "implicit upgrades" constantly breaking CI at unpredictable (and inconvenient) times, so I would like to propose a long-term solution, that will help isolating them so they:

• do not incapacitate normal CI
• can be dealt with on our on schedule
• (but are still automatically detected)

Basically the solution is to build two separate CI modes:

• stable (locked): this is the regular CI that runs on every PR. It should be reliable and only break when our code breaks.
• unstable (unlocked): this is akin to our "bleeding-edge" CI now, but instead of just running against a couple, hand-picked dependencies at >pre-release stage, it's role would be expanded to running against all the latest dependencies (including potentially broken ones), and mimic >what users get from running pip install yt at a given time.

"locking" test environments means:

• skip dynamic dependency resolution on stable CI, using a static lockfile instead (think of it as a known-good record of yt's dependency tree)
• periodically upgrade the lock's content with automated PRs. Points of failure will be isolated to these PRs (and bleeding-edge CI).

I know how to implement all of this with uv, but I also know that some may feel concerned about changing the testing strategy, so I'd like to open a discussion first. Are there any strong objections or concerns I should try to address first ?

[neutrinoceros]

related: yt-project/unyt#594

[chrishavlin]

This sounds like a reasonable change to me. Would certainly be nice (particularly for new contributors) if PRs did not suddenly start failing due to unrelated upstream issues. Only slight downside that I see is that the maintenance of the github action itself will require some uv knowledge, and beyond you @neutrinoceros I'm not sure anyone in the yt community has really taken the uv plunge (I'm picking it up slowly but haven't taken the full uv plunge yet). But overall +1 from me for doing this.

[neutrinoceros]

hopefully I can teach interested parties over the review process :)

[cphyc]

That's an important discussion to have, especially because we're heavily reliant on your contribution to fix all of these incompatibilities.

If you know how to lock dependencies with uv for our per-PR CI, and have a "rolling release" in parallel to it, I'd say to go ahead.

I gave a try at uv a while back. My then-concern was that it was trying to do everything and, as a result, was quite hard to work with. On the flipside, this gives us a mature and powerful tool for managing our CI, so I'd be happy to adopt it on the codebase.

[neutrinoceros]

I gave a try at uv a while back. My then-concern was that it was trying to do everything and, as a result, was quite hard to work with.

depends what you compare it against. I think it's easier to learn than the myriad of tools it replaced, but of course it's harder at first when you're already familiar with said other tools...
Of course I learned it piecemeal, following along early releases, so it never felt like too much to me, but I can imagine it's a lot to take on if you're learning it from scratch now.
So, to anyone willing to take a dive there, here's a step-by-step guide to your journey:

• start with replacing pip with uv pip (this should be trivial)
• then get used to uv venv (also trivial if you're used to python -m venv)
• try managing your python interpreters with uv python (migration difficulty depends on what you're used to)
• if you used pipx (or contemplated it), consider trying uv tool and uvx

At this point, you're ready to move to uv's "high level" interfaces (no manual env tinkering, no manual env activation):

• uv sync (includes uv venv)
• uv run (includes uv sync)

bonus:

• pip-tools -> uv pip
• python -m build -> uv build
• twine -> uv publish
• cookiecutter -> uv init

there's plenty more jewels in there, but I'll keep this brief for maximal impact

[neutrinoceros]

The first step, enabling uv in CI as a replacement for pip, is now ready for review at #5311

[neutrinoceros]

I've been working on this within the assumption that incremental steps towards the end goal would help making reviews easier, but maybe I should instead do it in a single PR with incremental commits.