Build the version-diff track

[khatchad]

The study is described as running along two tracks. Only one exists.

What Is There

The commit-message track works end to end: keyword matching over data/message_refactoring_candidates.csv surfaces 374 candidates, and human verification puts its precision at 29.7%.

analysis/inspect_commit.py has the machinery a second track would need. It resolves a commit, classifies file operations, reads column headers at both revisions for CSV, TSV, gzip and LFS-backed files, and reports a column-set delta. That is per-commit, on demand, and driven by a candidate list.

What Is Not There

Nothing runs that machinery over the corpus. There is no pipeline that walks the 44,864 mined commits, computes a structural delta for each, and emits candidates whose message says nothing. No result in the paper comes from such a pass. sections/threats.tex in the paper repo refers to "our preliminary version-diff probe"; I cannot find it in this repository.

Why It Matters More Than It Looks

Three claims currently lean on the track:

1. Recall is unmeasurable without it. Precision at 29.7% says what fraction of keyword proposals are real. How many real data refactorings the keywords never propose cannot be estimated from a candidate set the keywords built. The version-diff track is the only instrument that would answer it, because it detects structural change without consulting the message.
2. The self-selection argument depends on it. The methodology argues that roughly four in five commit messages are tooling boilerplate, so message-visible refactorings are a biased sample. That argument motivates the second track and is weakened by the second track not existing.
3. The parquet re-mine needs it. Widening the sampling frame beyond format:csv requires schema-level diffing across revisions, which is the same capability.

There is also a fourth, established this week: 45.5% of resolvable candidates are pure additions, and 90.2% of the version-marked ones have no predecessor anywhere in the repository. A filename-pairing detector would therefore recover single digits. The recall that is actually recoverable lives in in-place modifications, which is exactly what this track scans.

Shape

Not a small job, so worth scoping before starting:

• Walk commits, not candidates. The unit is every commit in a cloned repository, so the clone cache and the cost model both change.
• Emit a structural delta per commit: file operations plus column-set delta for tabular files.
• Decide what counts as a candidate on structural evidence alone, without the message.
• Report the intersection with the message track, which is the quantity that makes recall estimable: how many structurally-detected candidates the keywords also proposed.

--download cost is the obvious constraint. The header-only path needs no download for non-LFS files, so a first pass restricted to those would give a usable lower bound cheaply and is worth doing before committing to the full scan.

Blocking

ponder-lab/ML-Data-Refactoring-Study-Paper currently states the two tracks in the present tense. Either this gets built before submission, or that claim becomes future tense. Tracked on the paper side separately.