Workaround for rebasing commits in moved files

[anka-213]

Copy and rename detection isn't implemented yet for rebases: #47, but I wonder, is there any kind of workaround for when I want to rebase a commit which modifies a renamed file? For example:

o     d   Modify bar
| o   c   Modify foo
| |
o |   b   Rename foo to bar
|/
o     a   Create foo

Is there any way to rebase in either direction, either moving d after c or c after d without just getting a merge conflict that covers the entire file?

[jennings]

You can use git to do the rebase:

• Add a bookmark on the branch tip, if there isn't one already
• git switch foobar
• git rebase main --update-refs
• jj bookmark delete foobar, if you don't want the bookmark anymore.

Git handles rebasing over renames better. Adding a bookmark ensures jj abandons the original commits (you could just abandon them manually, but I think adding a temporary bookmark is easier).

[9999years]

I had to do this for a rather complex megamerge (7 branches, each with multiple commits). I had claude write a script that worked eventually, at least after spending $8 of my employer's tokens. To save you the effort, here's what we wrote.

This process keeps the change IDs of the original commits (unlike a regular git rebase) but does not preserve any merge structure; you need to recreate your megamerges afterwards. Sorry this is a total mess but hopefully it helps someone. And definitely backup your repo before running it, it worked for me but this is AI-generated code so it may explode catastrophically!!

What the script does

1. Run jj op restore ... to get a clean state (useful for multiple runs of the script while iterating on it)
2. Duplicate input commits with jj duplicate
3. Rebase each of the duplicated commits with git and resolve conflicts by spawning a claude session (lol)
4. Use jj rebase on the original commits to rebase them onto the new master (causing conflicts) and then jj restore --from ... --to ... in order to move the changes from the duplicated commits into the original commits (preserving the original change IDs)
5. Abandon duplicate commits

Input format

The input is in a format like this:

HASH1 DESCRIPTION 1 ...
HASH2 DESCRIPTION 2 ...
HASH3 DESCRIPTION 3 ...
HASH4 DESCRIPTION 4 ...

HASH5 DESCRIPTION 5 ...

HASH6 DESCRIPTION 6 ...
HASH7 DESCRIPTION 7 ...

This means that after the old master commit, first HASH1 has been applied, then HASH2, then HASH3, then HASH4. Separately, HASH5 has been applied on the old master commit. (That is, HASH1 and HASH5 are siblings.) And so on, with blank lines between branches.

Quirks to note

A couple quirks of this script:

1. I'm using claude to resolve merge conflicts (!!) which is weird and slow but actually seemed to work quite well.
2. Several values are hardcoded --- the old master commit hash, the op ID to restore to at the beginning.
3. We're using gitoxide to run gix clean which removes ignored files without trashing "precious" files (like the script itself).
4. This definitely left behind a bunch of commits with empty descriptions and conflicts. I jj abandoned those but it's weird that they were created.
5. Merges are not preserved. I didn't need this, no clue how hard it would be to implement.

The scripts themselves

Note that the files start with xxx because I have xxx* marked as gitignore and precious. (This message is hand-written, only the script is AI generated!)

xxx-rebase.py

#!/usr/bin/env python3
"""Rebase commits across dux/ -> tools/ rename, preserving jj change IDs."""

from __future__ import annotations

import os
import re
import subprocess
import sys
from dataclasses import dataclass, field
from pathlib import Path

OLD_MASTER = "ab0ceea1d75ba1a0ee108f9e14603542634c6839"
SCRIPT_DIR = Path(__file__).resolve().parent
INSTRUCTIONS = SCRIPT_DIR / "xxx-claude-resolve-conflicts-instructions.md"


def run(
    cmd: list[str],
    *,
    check: bool = True,
    env: dict[str, str] | None = None,
    stream: bool = False,
) -> subprocess.CompletedProcess[str]:
    """Run a command, printing it and returning the result."""
    # Prevent jj from snapshotting/updating the working copy, so it doesn't
    # conflict with git operations that also use the working copy.
    if cmd[0] == "jj":
        cmd = [cmd[0], "--ignore-working-copy", *cmd[1:]]
    display_cmd = []
    for i, arg in enumerate(cmd):
        # Hide long args (like system prompts) — show the flag name instead.
        if len(arg) > 200:
            display_cmd.append(f"<{len(arg)} chars>")
        else:
            display_cmd.append(arg)
    print(f"  $ {' '.join(display_cmd)}", flush=True)
    merged_env = {**os.environ, **(env or {})}
    if stream:
        # Stream stdout/stderr directly to the terminal for long-running commands.
        proc = subprocess.Popen(cmd, env=merged_env)
        proc.wait()
        result = subprocess.CompletedProcess(cmd, proc.returncode, "", "")
    else:
        result = subprocess.run(cmd, capture_output=True, text=True, env=merged_env)
        if result.stdout.strip():
            for line in result.stdout.strip().splitlines():
                print(f"    {line}", flush=True)
        if result.stderr.strip():
            for line in result.stderr.strip().splitlines():
                print(f"    [stderr] {line}", flush=True)
    if check and result.returncode != 0:
        raise subprocess.CalledProcessError(
            result.returncode, cmd, result.stdout, result.stderr
        )
    return result


def jj_change_id(rev: str) -> str:
    """Get the short (8-char) jj change ID for a revision."""
    result = run(["jj", "log", "-r", rev, "--no-graph", "-T", "change_id.short(8)"])
    return result.stdout.strip()


def jj_full_hash(rev: str) -> str:
    """Get the full git commit hash for a jj revision."""
    result = run(["jj", "log", "-r", rev, "--no-graph", "-T", "commit_id"])
    return result.stdout.strip()


@dataclass
class DupInfo:
    change_id: str
    short_hash: str


@dataclass
class GroupInfo:
    commits: list[str]  # original commit hashes
    orig_change_ids: list[str] = field(default_factory=list)
    dup_infos: list[DupInfo] = field(default_factory=list)
    rebased_hashes: list[str] = field(default_factory=list)


# ── Parse xxx-commits.txt into groups ──────────────────────────────────────────


def parse_commits_file(path: Path) -> list[list[str]]:
    """Parse the commits file into groups of hashes, separated by blank lines."""
    groups: list[list[str]] = []
    current: list[str] = []

    for raw_line in path.read_text().splitlines():
        # Strip the "     N→" prefix from Read tool format (if present)
        line = re.sub(r"^\s*\d+→", "", raw_line).strip()

        if not line:
            if current:
                groups.append(current)
                current = []
        else:
            commit_hash = line.split()[0]
            current.append(commit_hash)

    if current:
        groups.append(current)

    return groups


def parse_duplicate_output(output: str) -> DupInfo:
    """Parse jj duplicate output to extract new change ID and hash.

    Example: "Duplicated a0e96f60525e as vvuupxnm 89420af1 (conflict) DUX-4423..."
    """
    first_line = output.strip().splitlines()[0]
    m = re.match(r"Duplicated [a-f0-9]+ as ([a-z]+) ([a-f0-9]+)", first_line)
    if not m:
        print(
            f"ERROR: could not parse jj duplicate output: {first_line}", file=sys.stderr
        )
        sys.exit(1)
    return DupInfo(change_id=m.group(1), short_hash=m.group(2))


def restore_repo_state() -> None:
    print("Restoring repository state before starting attempt")
    run(["jj", "op", "restore", "270f6e4fbd15"], check=True)


# ── Step 1: Duplicate all commits with jj ──────────────────────────────────────


def step1_duplicate(groups: list[GroupInfo]) -> None:
    print("=== STEP 1: Duplicating commits with jj ===")

    for gi, group in enumerate(groups):
        prev_dup_change: str | None = None

        for commit in group.commits:
            orig_cid = jj_change_id(commit)
            group.orig_change_ids.append(orig_cid)

            parent = prev_dup_change if prev_dup_change else OLD_MASTER

            print(f"  Duplicating {commit} (change={orig_cid}) onto {parent} ...")
            result = run(["jj", "duplicate", commit, "--onto", parent], check=False)
            output = result.stdout + result.stderr
            if result.returncode != 0 and "Duplicated" not in output:
                print(f"ERROR: jj duplicate failed: {output}", file=sys.stderr)
                sys.exit(1)

            dup = parse_duplicate_output(output)
            group.dup_infos.append(dup)
            prev_dup_change = dup.change_id

        print(
            f"  Group {gi + 1}: top dup change={group.dup_infos[-1].change_id}, "
            f"orig cids={' '.join(group.orig_change_ids)}"
        )

    print()


# ── Step 2: Git rebase each group ─────────────────────────────────────────────


def rebase_with_claude(group_num: int) -> None:
    """Run git rebase, invoking Claude to resolve conflicts as needed."""
    result = run(["git", "rebase", "origin/master"], check=False)
    if result.returncode == 0:
        print(f"  Group {group_num}: rebase completed cleanly")
        return

    rebase_in_progress = True
    while rebase_in_progress:
        # Check if there are actually conflicted files, or if the commit is empty.
        status = run(["git", "diff", "--name-only", "--diff-filter=U"], check=False)
        has_conflicts = bool(status.stdout.strip())

        if has_conflicts:
            print(f"  Group {group_num}: conflicts detected, invoking Claude...")

            # Build env without CLAUDECODE to avoid nesting error
            claude_env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}

            instructions_text = INSTRUCTIONS.read_text()
            run(
                [
                    "claude",
                    "-p",
                    "--dangerously-skip-permissions",
                    "--allowedTools",
                    "Bash Edit Read Glob Grep",
                    "--system-prompt",
                    instructions_text,
                    "Resolve the current git rebase conflicts. "
                    "Run 'git status' to see conflicted files, then resolve each one. "
                    "After resolving all conflicts, run 'git add -A'. "
                    "Do NOT run 'git rebase --continue'.",
                ],
                env=claude_env,
                check=False,
                stream=True,
            )

            run(["git", "add", "-A"])

            result = run(
                ["git", "rebase", "--continue"],
                check=False,
                env={**os.environ, "GIT_EDITOR": "true"},
            )
        else:
            # No unmerged files — either rerere resolved it, or the commit is empty.
            # Try --continue first; only skip if that fails (truly empty).
            print(f"  Group {group_num}: no unmerged files, trying to continue...")
            run(["git", "add", "-A"])
            result = run(
                ["git", "rebase", "--continue"],
                check=False,
                env={**os.environ, "GIT_EDITOR": "true"},
            )
            if result.returncode != 0:
                print(f"  Group {group_num}: continue failed (empty commit), skipping...")
                result = run(["git", "rebase", "--skip"], check=False)

        if result.returncode == 0:
            rebase_in_progress = False
        elif not Path(".git/rebase-merge").exists() and not Path(".git/rebase-apply").exists():
            # Rebase ended (maybe with error) but no rebase in progress — we're done.
            rebase_in_progress = False

    print(f"  Group {group_num}: rebase completed")


def step2_rebase(groups: list[GroupInfo]) -> None:
    print("=== STEP 2: Git rebase each group ===")

    for gi, group in enumerate(groups):
        top_dup_change = group.dup_infos[-1].change_id
        top_dup_full_hash = jj_full_hash(top_dup_change)

        print(f"--- Group {gi + 1}: rebasing {len(group.commits)} commit(s) ---")
        print(f"  Top duplicate: {top_dup_change} ({top_dup_full_hash})")

        # Clean up before switching: reset tracked files and remove untracked
        # non-ignored files (e.g. from a previous group's rebase). gix clean
        # without --ignored respects .gitignore, so .jj/ and xxx-* are safe.
        run(["git", "checkout", "-f"], check=False)
        run(["gix", "clean", "--directories", "--execute"], check=False)
        run(["git", "switch", "-d", top_dup_full_hash])

        rebase_with_claude(gi + 1)

        # Capture rebased commit hashes in order (bottom to top)
        result = run(["git", "log", "--reverse", "--format=%H", "origin/master..HEAD"])
        rebased = result.stdout.strip().splitlines()
        group.rebased_hashes = rebased

        if len(rebased) != len(group.commits):
            print(
                f"ERROR: Group {gi + 1} mismatch: "
                f"expected {len(group.commits)} commits, got {len(rebased)}",
                file=sys.stderr,
            )
            sys.exit(1)

        print(f"  Rebased hashes: {' '.join(rebased)}")
        print()


# ── Step 3: Restore content into original commits ─────────────────────────────


def step3_restore(groups: list[GroupInfo]) -> None:
    print("=== STEP 3: Restoring content into original commits ===")

    run(["jj", "git", "import"], check=False)

    for gi, group in enumerate(groups):
        print(f"--- Group {gi + 1} ---")

        for i in range(len(group.commits)):
            orig_cid = group.orig_change_ids[i]
            rebased_hash = group.rebased_hashes[i]
            rebased_cid = jj_change_id(rebased_hash)

            dest = "master@origin" if i == 0 else group.orig_change_ids[i - 1]

            print(
                f"  Moving {orig_cid} onto {dest}, restoring from {rebased_cid} ({rebased_hash})"
            )

            run(["jj", "rebase", "-r", orig_cid, "-d", dest])
            run(["jj", "restore", "--from", rebased_cid, "--into", orig_cid])

    print()


# ── Step 4: Cleanup ───────────────────────────────────────────────────────────


def step4_cleanup(groups: list[GroupInfo]) -> None:
    print("=== STEP 4: Cleanup ===")

    # Abandon duplicates
    for group in groups:
        for dup in group.dup_infos:
            print(f"  Abandoning duplicate: {dup.change_id}")
            run(["jj", "abandon", dup.change_id], check=False)

    # Abandon rebased commits
    for group in groups:
        for rh in group.rebased_hashes:
            try:
                rebased_cid = jj_change_id(rh)
                print(f"  Abandoning rebased: {rebased_cid} ({rh})")
                run(["jj", "abandon", rebased_cid], check=False)
            except subprocess.CalledProcessError:
                pass  # already abandoned or unreachable

    print()


# ── Main ──────────────────────────────────────────────────────────────────────


def main() -> None:
    commit_groups = parse_commits_file(SCRIPT_DIR / "xxx-commits.txt")

    print(f"Parsed {len(commit_groups)} groups:")
    for i, g in enumerate(commit_groups):
        print(f"  Group {i + 1}: {' '.join(g)}")
    print()

    groups = [GroupInfo(commits=g) for g in commit_groups]

    restore_repo_state()
    step1_duplicate(groups)
    step2_rebase(groups)
    step3_restore(groups)
    step4_cleanup(groups)

    print("=== DONE ===")
    print("Run 'jj log' to verify the results.")


if __name__ == "__main__":
    main()

xxx-claude-resolve-conflicts-instructions.md

You are resolving git rebase conflicts caused by rebasing commits onto a version of master where directories or files involved in those commits have been renamed.

## How to Resolve Conflicts

### Step 1: Identify conflicted files

Run `git status` to see all files with conflicts.

### Step 2: Resolve each file

For each conflicted file, read it to see the conflict markers (`<<<<<<<`, `=======`, `>>>>>>>`), and resolve conflicts.

### Step 3: Stage resolved files

After resolving all conflicts, run `git add -A` to stage everything.

## Important

- Do NOT run `git rebase --continue` - the calling script handles that
- Do NOT run `git rebase --abort`
- Focus only on resolving the conflicts in the current step of the rebase