Pants "subcmd" running use case notes

[cburroughs]

Pants "subcmd" running use case notes

The typical Pants cli invocation looks like:

pants [options] [goals] [inputs]

This allows one to express multiple actions on the same set of targets (ex: pants fix test src:: and this is among the most common use cases. However, it makes it difficult to express optionally that may be specific to only some "targets" (or not even apply to targets). This document outlines 3 use cases that are challenging with the existing pattern, the first two of which are likely more tractable. I have some idea how to implement the "meat" of the change, and how I would do it with click, but not how to hook it up to the existing goals/options system.

The intent is to provide concrete examples to illustrate the current limitations to inform possible paths forward for implementation. (There is not an obvious way to do these today with the existing goal system.) But not to completely flesh out a design for all of them at once.

Cases

cache tooling (no target)

#11167

Pants maintains several different caches, including ones both internal to Pants (lmdb) and cases where pants Pants is orchestrating "named" caches for other tools. Users have reasonable requests like "tell me how big my cache is" and "prune it so it doesn't grow without bounds". The size of these caches vary greatly so some of these are more valuable than others. For example on my workstation named_caches/pex_root is 350 GiB, lmdb_store/ is 36 GiB, and third place is 3 GiB.

Some of the tools -- notably Pex -- have their own tools for pruning or otherwise managing their cache. Unlike goals, these commands would act on the specific caches, but not targets.

reference: pex --help

$ pex3 cache --help
usage: pex cache [-h] {dir,info,purge,prune} ...

Interact with the Pex cache.

options:
  -h, --help            show this help message and exit

subcommands:
  Interact with the Pex cache via the following subcommands.

  {dir,info,purge,prune}
    dir                 Print the current Pex cache directory path.
    info                Present information about Pex cache status.
    purge               Purge the Pex cache safely.
    prune               Prune the Pex cache safely.

$ pex3 cache prune --help
usage: pex cache prune [-h] [-H] [--older-than CUTOFF] [-n] [-o PATH] [--emit-warnings] [--pex-root PEX_ROOT] [--disable-cache]
                       [--cache-dir CACHE_DIR] [--tmpdir TMPDIR] [--rcfile RC_FILE]

options:
  -h, --help            show this help message and exit
  -H, --human-readable, --bytes, --kB, --MB, --GB, --TB, --PB
                        How to display disk usage amounts; defaults to bytes. The -H / --human-readable options display amounts in
                        a human readable way and each of the other unit options displays amounts with that specified unit.
  --older-than, --last-access, --last-access-before CUTOFF
                        Prune zipapp and venv caches (amongst others) last accessed before the specified time. If the dependencies
                        of the selected zipapps and venvs (e.g.: installed wheels) are unused by other zipapps and venvs, those
                        dependencies are pruned as well. The cutoff time can be specified as a date in the format `<day
                        number>/<month number>/<4 digit year>` or as a relative time in the format `<amount>
                        [second(s)|minute(s)|hour(s)|day(s)|week(s)]`.
  -n, --dry-run         Don't actually purge cache entries; instead, perform a dry run that just prints out what actions would be
                        taken
  -o, --output PATH     A file to output the Pex purge results to; STDOUT by default or when `-` is specified.

Global options:
  --emit-warnings, --no-emit-warnings
                        Emit runtime UserWarnings on stderr. If false, only emit them when PEX_VERBOSE is set.
  --pex-root PEX_ROOT   Specify the pex root used in this invocation of pex (if unspecified, uses /home/ecsb/.cache/pex).
  --disable-cache       Disable caching in the pex tool entirely.
  --cache-dir CACHE_DIR
                        DEPRECATED: Use --pex-root instead. The local cache directory to use for speeding up requirement lookups.
  --tmpdir TMPDIR       Specify the temporary directory Pex and its subprocesses should use.
  --rcfile RC_FILE      An additional path to a pexrc file to read during configuration parsing, in addition to reading
                        `/etc/pexrc` and `~/.pexrc`. If `PEX_IGNORE_RCFILES=true`, then all rc files will be ignored.

example cli

(assume these launch as experimental-cache)

• (A) pants cache named pex prune
• (B) pants cache pex prune
• (C) pants cache prune pex,lmdb,mypy
• (D) pants cache named pex -- prune (raw cli passthru)
• (E) pants cache pex -- prune (raw cli passthru)

Questions

• Is having a top level distinction named vs lmdb helpful future proofing?
• What about pants cache tool ... vs pants cache future-all-cache-cmds?
• Do we actually gain much at this point with a bunch of "wrapping" like in (A,B)? Why not just do the thinnest possible layer like (D,E)?
• As a practical matter, have there been any requests for cache control features besides lmdb and pex?

Future Questions:

• It might be nice in the future to have some sort of "prune everything older than X in all supported cache types" command. This may be more difficult in a pure passthru implementation.
• Does "online GC" make any sense for named caches, or only lmdb?

lockfile manipulation (usually ~one resolve, or resolve ecosystem)

#15704
#15568
#12880

There are a lot of different things people would like to do with lockfiles besides generating the whole thing from scratch:

• Change the version of N package.
• Try to update version X in the existing range
• Remove N packages from the lockfile.
• Point the entire lockfile at a different index while minimizing changes.

The details of all of these are pretty specific to the given ecosystems. For example, it would not make sense to say "update pytest to x.y.z" to both java and Python resolves. It might make sense to say "update pytest to x.y.z" to multiple Python resolves. But, I wearly of trying to support identical operations similar but different ecoystehsms by creating semenatics to "update pytest to x.y.z" that apply to Pex, Poetry, and UV, but are different from each.

example cli

• (A) pants lock update pex [resolve-name] --pex-specific-flags
• (B) pants lock update pex --resolve=foo --resolve=bar --pex-specific-flags
• (C) pants lock update --resolve=foo --dynamic-flags-based-on-type-of-resolve (--resolve is mandatory?)
• (D) pants generate-lockfiles --not-actually-generate-operation=update --resolve=foo --dynamic-flags-based-on-operation (today; not a real proposal)
• (E) pants lock direct-op --tool=pex --resolve=foo -- raw-pass-thru

REMINDER: This is illustrative of things that are hard to address with the goal system today. Pex/Poetry/UV have a lot of prior art here on the details.

Unlike the cache case, these operate on things existing Pants goals know about (resolves), but are not targets.

Questions

• It is not obvious from the output of pants help lint and pants help generate-lockfiles that one operates on targets and the other does not. Should we have a name for this that shows up in help? TargetGoal?
• Are any of the "subcmd" example patterns easier to implement?

targeting

#15012

For example, consider the case where a repository has multiple "types" of tests and has tagged all the unit tests with unit one can do pants fmt lint test :: to run those goals on everything, or pants --tag=unit fmt lint test :: to run on all unit targets, but not "fmt fix lint everything and also run all the unit tests".

Hey what about help

An existing "goal" that already follows this pattern is help

• pants help tools
• pants help ruff

Awkward but more composable

The examples above were generally in the form pants goal sub subsub. This looks "natural" but would not be posible to compose with other goals. Given that neither the cache nor lockfile use case operates on targets tha thist is probably an okay tradeoff, but it might be possible to have spaces or - in the "goal name. So

• pants fmt cache-named-pex-prune test ::
• pants fmt 'cache named pex prune' test ::

I'm not sure of a real use case where one would want to do this. (You can't do this with help today.)

Other Refernces

• https://leebriggs.co.uk/blog/2023/09/28/300_percent_problem

[jsirois]

A historical note of perhaps interest. In Pants v1 when I built out better support for go for Medium, I heavily leveraged the go CLI internally (go list, go build, go test, etc). I only had two weeks contracted on the project though and knew I couldn't cover all use cases; so I made sure to add support for pants go -- ... (c.f.: 956bc31). This would set up go according to all the Pantsisms the user did not care about / know about / etc and then run that go handing off everything else to it. I got out of the users way. In the two examples given in the Python backend / Pex case, a similar approach here would be to add support for, say: pants tool pex -- .... As a user you'd then:

# Here the cache sub-command works because Pex is at the version Pants uses and its PEX_ROOT has been adjusted to match the one Pants sets up.
pants tool pex -- cache ...

# This would only work sanely and in an easy to grok way for the user if Pants stopped abusing user files to store its own state - there is a Pants issue about this.
pants tool pex3 -- lock update -Pcowsay 3rdparty/python/my-lock-hopefully-pants-no-longer-adds-invalid-header-to.json

The short of it is, even if you do want to concoct more baroque layerings for whatever reason, at the end of the day, always having an escape hatch to the underlying tool, minimally configured to work with Pants, is super useful for users, tool maintainers and Pants maintainers (who will get rung up less).

[sureshjoshi]

Quite a ways back, I had the idea of having some "helper" CLIs which could even operate interactively. I think the first could be a BuiltinGoal with manual arg parsing (I have a prototype lying around one of my computers) - but the second doesn't seem possible today without some plumbing.

% ./pants plugin init --type linter --backend cc --name clangtidy --preview
Or
% ./pants plugin init
What type of plugin do you want to create? [linter/formatter/checker/...] linter
What backend do you want to use? [cc/go/python/...] cc
What is the name of the plugin? [awesome-plugin] clangtidy
Is this a preview plugin? [y/n] y

So, everything is pants - without shelling out to an external tool. Similar to how help works, to some degree. Though, help also defines multiple goals instead of using an option syntax currently.

help                                      Display usage message.
help-advanced                             Help for advanced options.
help-all                                  Print a JSON object containing all help info.

I'd be interested in seeing where the sub-command idea goes in terms of robustness to options and whatnot. A big value add to me, would be compacting a number of goals into a hierarchy (namespace) to help new users out - so they don't see a wall of goals, but get some sort of progressive disclosure.

For the external tools, having an easy way to escape hatch down to their command lines could be helpful for whenever one of them adds/changes something - and Pants isn't a limiting factor. I did think we already had some sort of pants tool concept, but that might have just been my imagination.

[benjyw]

How would the options scoping interact with this?

Today pants test --flag ... is equivalent to pants --test-flag test ....

subcmds add a potential layer of scoping, or at least scoping confusion.

I think the sanest and most expected thing to do is get rid of the long scoped flags entirely, so that each cmd (when there are no subcmds) or each subcmd (when there are) is its own thing with its own options that have to be specified in the right position, and its own help.

This also has the advantage of avoiding the rare-in-practice-thankfully but potentially very irritating name collisions that this scheme can cause. And it will also simplify options parsing.

[cognifloyd]

I like the idea of simplifying and flattening the options scopes, but I would rather the options were not positional. There have been many times where I ran something like pants fmt :: and when there was a failure, ran pants fmt :: --keep-sandboxes=on_failure (up in history and then append the flag). That, of course, doesn't work because --keep-sandboxes is a global option not a fmt scoped option.

[sureshjoshi]

To have multiple goals (though), you'd then scope all the options. I think there is a world where the problem you've mentioned has some resolution (e.g. always allow global options, and if there is one goal, use it implicitly for future options).

Some workflow problems to think though, but it seems feasible - if not a little inelegant.

[benjyw]

I like the idea of simplifying and flattening the options scopes, but I would rather the options were not positional. There have been many times where I ran something like pants fmt :: and when there was a failure, ran pants fmt :: --keep-sandboxes=on_failure (up in history and then append the flag). That, of course, doesn't work because --keep-sandboxes is a global option not a fmt scoped option.

I don't think this is going to be practical in a world of pluggable backends that can define their own options. The only way to know which options should be handled by whom is positional.

cargo uses position to determine if a flag is global or belonging to a subcommand, as does docker, and as do other swiss-army-knife commands.

[benjyw]

That said, those tools only allow a single command/subcommand per invocation. We currently allow multiple goals per invocation. Although perhaps we shouldn't since in practice those are sequential anyway.

[cognifloyd]

I wonder if we should have an experimental mode for the pants cli. We could have multiple CLI entry points, similar to pex vs pex3 where pex has a stable interface, but pex3 can be updated.

[benjyw]

Yes, I think we should. It will let us break the CLI interface and put it back together without affecting existing users.

[benjyw]

Here is a new CLI entry point to kick around and experiment with: #22808

Right now it doesn't actually do anything, future PRs will wire it up to stuff, ideally new stuff at least at first.