cli: add snapshot.derive-tracked-from-ignores option

[valtron]

This is a proposal for a new feature (opt-in via config) to alleviate the issue mentioned in #323. I skimmed the ~200 comments and didn't see this exact idea discussed there, though some were similar, particularly #323 (comment) ("There should be one source of truth for ignored or not. I assume this to be the ignore file." by @joyously is a nice way of putting it). In a sense this PR is the opposite of #8628. Also, this PR would touch on #6157.

This PR adds config snapshot.derive-tracked-from-ignores (tentative name), which if true, treats the ignores as the "source of truth" for what should be tracked, or rather, what should not be tracked:

• if a file is currently tracked, adding it to ignores should have the same effect as deleting it: jj status should show it as deleted
• if a file is not currently tracked but in the ignores, removing it from ignores should have the same effect adding that file to a repo that didn't have it: jj status should show it as added

The new test test_derive_tracked_from_ignores checks exactly this.

Additionally, if snapshot.derive-tracked-from-ignores is set, I also made jj {un,}track error, because that conflicts with ignores being the source of truth.

Checklist

If applicable:

• I have updated CHANGELOG.md
• I have updated the documentation (README.md, docs/, demos/)
• I have updated the config schema (cli/src/config-schema.json)
• I have added/updated tests to cover my changes
• I fully understand the code that I am submitting (what it does,
  how it works, how it's organized), including any code drafted by an LLM.
• For any prose generated by an LLM, I have proof-read and copy-edited with
  an eye towards deleting anything that is irrelevant, clarifying anything
  that is confusing, and adding details that are relevant. This includes,
  for example, commit descriptions, PR descriptions, and code comments.

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[joyously]

I can't tell from my cursory look at your code, but it seems like there would be an interaction with the snapshot.auto-track config. I'm guessing that this one overrides the other one. I don't see any change to explain in a doc file like Settings or wherever "ignore files" are discussed.
Also, a hint to change the ignore file along with the error on track and untrack would be nicer (since you aren't editing it for them).

[valtron]

it seems like there would be an interaction with the snapshot.auto-track config

Yeah, if a file is ignored, it overrides it, otherwise same logic as before applies.

I updated the docs to explain it a bit more, as well hints in jj file {un,}track.

[martinvonz]

Have you seen #7237? I think I'm currently leaning towards option 3 in this comment.

[valtron]

I took at a look at that issue, but it seems orthogonal (please correct me if you don't think so) to this proposal. (This is more for solving this problem, which I also ran into and it confused me.)

Running jj edit X is hairy, and for any files in the working copy that (a) aren't in X's ignores and (b) aren't in X, I see the following options:
a) check out X, keep the file and update X to add the file (current behaviour)
b) check out X, delete the file (dangerous)
c) check out X, keep the file and don't update X; but it will be auto-added on next operation that does a snapshot (contingent on snapshot.auto-track); is this what it means for a working copy to be stale?
d) fail to check out X and print an error listing the files that are currently ignored but not ignored in X (#7237) [1]

Option (d) seems reasonable to me! I would see the error, add the file to .git/info/exclude, and continue.

In your comment, I think you're tackling a more general problem (how to handle failed working copy updates when applying an operation). I don't understand the internals well, so I don't know what's right for any operation [2]... but specifically for jj edit, (d) means nothing should be done at all, so there is no operation to apply.

[1] I'd go even further and say that jj edit X should never modify X.
[2] That caveat out of the way, I prefer 3 > 1 > 1.

[martinvonz]

Perhaps what you're missing is that jj edit does produce an operation in the operation log? Each operation keeps track of where each workspace's working copy is supposed to point, so jj edit X simply updates that record. Then when the operation has been recorded, we update the working copy (the files on disk) to match the new target commit. Does that clarify?

[valtron]

Yeah that makes sense, but what I meant with (d) is that jj edit should first check if there are ignore-related changes that would cause an issue with updating the working copy, and stop if it finds any before attempting an operation. Same as, e.g., jj edit invalidrevset doesn't add anything to the op log.

Here's a concrete example based on #5596:

$ jj git init
Initialized repo in "."
Hint: Running `git clean -xdf` will remove `.jj/`!
$ jj new
Working copy  (@) now at: wlxqukvt 8bbfeea3 (empty) a
Parent commit (@-)      : oxulxukl f459f1ee (empty) (no description set)
$ echo ignored > .gitignore
$ touch ignored
$ jj op log
@  e0356873d390 u@h now, lasted 19 milliseconds
│  snapshot working copy
│  args: jj op log
○  796b690a1a28 u@h 36 seconds ago, lasted 28 milliseconds
│  new empty commit
│  args: jj new
○  823d2926f3c7 u@h 39 seconds ago, lasted 22 milliseconds
│  add workspace 'default'
○  000000000000 root()
$ jj edit @-
Error: The following files are ignored in the working copy but not in the target commit:
  ignored
hint: delete the files or exclude them using `.git/info/exclude`
$ jj op log # unchanged
@  e0356873d390 u@h now, lasted 19 milliseconds
│  snapshot working copy
│  args: jj op log
○  796b690a1a28 u@h 36 seconds ago, lasted 28 milliseconds
│  new empty commit
│  args: jj new
○  823d2926f3c7 u@h 39 seconds ago, lasted 22 milliseconds
│  add workspace 'default'
○  000000000000 root()

[martinvonz]

There are many other commands than jj edit that can update your working copy in a similar way. For example, jj new X would have exactly the same effect. jj rebase -r @ -o X would have a very similar effect. jj abandon X..@ would have the same effect. And so on. So I think we should not implement the solution in jj edit specifically.

[valtron]

Right. I guess my confusion around handling errors when updating the working copy and what's best was because I'm mentally grouping them into two kinds:

• Those we can check ahead of time (e.g. changes in ignores): for these the command (edit, rebase, etc.) should check and not log an op if it detects an error. This would be different than what's currently happening.
• Unexpected ones (e.g. ^C while working copy is getting updated): no opinion.