Tracking text chunk diffs between locations and files

[temportalflux]

While exploring improvements to git and vc-systems, a thought experiment for the larger JJ community. Often I've found that its challenging to track a chunk of text that has moved locations within a file or has moved to a new file, but the diff is largely unchanged or changed very slightly. This impacts both being able to see the history of a chunk of code vs the history of a file, and impacts how well peers are able to review a code change and comprehend the diff.

It would be stellar if jj or a future VCS was able to either itself track (computationally difficult) or allow the author or a user to markup a commit to help describe the diff. If this can be tracked in the commits, tools which view the diff of a file overtime or when viewing the diff in a PR would be able to display the changes in a way that is more nuanced and comprehendable.

Anyway no idea if this is something jj will ever explore or be capable of exploring. But something akin to this would certainly make my life easier as a software engineer.

[joyously]

Reading this, I'm reminded of trying to find the commit that deleted something. You can use blame, but it doesn't show the deletions.
jj does have evolog to show a a change evolved, but you have to know which change to look at. I think you are talking about the bigger picture, which would help find the change to look at.
There was a recent question about storing more metadata in the commits so that reviews are easier, but it has the same problem: how does the reviewer comment that there is a change missing (like the docs explaining the code)?
There is also a tool being worked on called weave that sort of does what you are asking, for rearrangement of code, by parsing code, which means it's no good for plain text or binary files.

[temportalflux]

weave sounds interesting; sounds like it would aspire to doing AST parsing or some such to compare code across files, but I suspect that in and of itself would not be sufficient. May still be prone to similar-code-but-wrong-destination. Attaching some metadata to commits would be interesting, or even some kinda of secondary database keyed by change/commit id containing file+linechunk pair mappings. Of course assuming that the moving of logic happened in a singular atomic change; this wouldnt account for code removed in one change and reintroduced in a later change that was not squashed together. I do imagine there'd always need to be some level of human input to provide adequate human context. And then let patterns emerge from there that could open avenues for workflow improvements and automation.

[PhilipMetzger]

This all sounds like source control Weaves to me. I think @durin42 is the expert on them but we haven't really found a way to integrate them cleanly into either annotate or absorb yet.

[durin42]

Everything I know about weaves I learned from Larry McVoy and a slight perusal of the BitKeeper codebase (which is Apache-licensed now.) Larry is extremely enthusiastic about sharing his knowledge with active source control hackers, I'd encourage you to look him up if you want to try and figure out how to get weaves into the jj codebase!

Realistically I think doing something involving weaves is at least a new cache layer (like the linelog extension in Mercurial was, which was also how hg absorb worked) or going all-in and building a new repo format around weaves. That's a bigger endeavor than I can manage at the moment with my other commitments, but something I'd be excited to see happen.