Add but worktree new with a copy-on-write fast path - #15471
Open
schacon wants to merge 1 commit into
Open
Conversation
git worktree add gives you a checkout but no build output, so the first build in a new worktree is cold. Copying target/ over is worse than useless: the copy costs more than the compile it saves, and plain cp -R restamps every mtime so cargo rebuilds anyway. With -c the worktree is created --no-checkout and populated by cloning the working directory through clonefile(2), which shares blocks, keeps metadata, and clones a whole tree per syscall. Two resets then reconcile it to the workspace base, touching only the paths that differ. Measured on a 51 GB tree: 42s and 191 MB, against 111s to rebuild from scratch and 124s / 49 GB to copy. Support is probed by cloning a real file into the destination, since the same filesystem across volumes cannot share blocks; unsupported filesystems fall back to a checkout.
github-actions
Bot
added
rust
Contributor
There was a problem hiding this comment.
Pull request overview
This PR reintroduces a scoped but worktree namespace by adding but worktree new, which creates a linked git worktree at the workspace base commit and (optionally) populates it via a macOS copy-on-write clonefile(2) fast path to preserve build outputs like target/ without paying full copy costs.
Changes:
- Add
but worktree new <path> [-c/--cow], including copy-on-write population + fallback to normal checkout when unavailable. - Wire the new subcommand into CLI parsing, dispatch, help grouping, and metrics.
- Add
libcas a dependency to callclonefile(2)on macOS.
Reviewed changes
Copilot reviewed 9 out of 10 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| crates/but/src/utils/metrics.rs | Maps the new Worktree subcommand to Unknown metrics for now. |
| crates/but/src/lib.rs | Registers the new worktree args module and dispatches but worktree new to the implementation. |
| crates/but/src/command/worktree.rs | Implements but worktree new, including the macOS clonefile copy-on-write path and fallback behavior. |
| crates/but/src/command/mod.rs | Exposes the new command::worktree module (currently with incorrect feature gating). |
| crates/but/src/command/help.rs | Places worktree in the “OtherCommands” group for help output. |
| crates/but/src/args/worktree.rs | Defines clap args/docs for the but worktree command group and its new subcommand. |
| crates/but/src/args/mod.rs | Adds Worktree to the top-level Subcommands and exports the args module. |
| crates/but/Cargo.toml | Adds libc dependency for the macOS syscall. |
| Cargo.toml | Adds workspace-level libc version. |
| Cargo.lock | Locks libc into the dependency graph. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
1
to
+4
| //! A place for each command, i.e. `but foo` as `pub mod foo` here. | ||
| #[cfg(feature = "legacy")] | ||
| pub mod legacy; | ||
| #[cfg(feature = "legacy")] | ||
| pub mod worktree; |
Comment on lines
+147
to
+158
| fn cow_supported(source: &Path, dest_parent: &Path) -> bool { | ||
| let probe = source.join(".but-cow-probe"); | ||
| let clone = dest_parent.join(".but-cow-probe-clone"); | ||
| let _ = std::fs::remove_file(&clone); | ||
| if std::fs::write(&probe, b"probe").is_err() { | ||
| return false; | ||
| } | ||
| let supported = clone_path(&probe, &clone).is_ok(); | ||
| let _ = std::fs::remove_file(&probe); | ||
| let _ = std::fs::remove_file(&clone); | ||
| supported | ||
| } |
Comment on lines
+110
to
+115
| bail!( | ||
| "git {} failed\n\n{}", | ||
| args.join(" "), | ||
| String::from_utf8_lossy(&output.stderr) | ||
| ) | ||
| } |
Comment on lines
+21
to
+25
| /// With `--cow`, the worktree is populated by cloning the current working directory | ||
| /// copy-on-write instead of checking every file out. On a filesystem that supports it | ||
| /// (APFS, btrfs, XFS with reflinks) the clone is near-instant and costs almost no disk, | ||
| /// and it carries untracked build output — `target/`, `node_modules/` — across with it, | ||
| /// so builds in the new worktree start warm. |
Comment on lines
+14
to
+16
| /// Create a worktree at `path`, checked out at the workspace's base commit. | ||
| pub fn new(ctx: &Context, out: &mut dyn WriteWithUtils, path: &Path, cow: bool) -> Result<()> { | ||
| let repo = ctx.repo.get()?; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
git worktree addgives you a checkout but no build output, so the first build in a new worktree is cold. The obvious fix — copytarget/over — turns out to be a bad trade in both time and disk.but worktree new -c <path>creates the worktree--no-checkoutand populates it by cloning the working directory throughclonefile(2): blocks are shared, metadata is preserved, and a whole directory tree clones in one syscall. Two resets then reconcile the tree to the workspace base, rewriting only the paths that actually differ and leaving untracked build output alone.Measured on this repo (51 GB tree, 48 GB of it
target/)-c)cp -Rp)cp -R)Each worktree got the same one-line edit before compiling
cargo build --release -p but.Two things worth pulling out:
cp -Rwithout-pis a trap. It restamps every mtime, and cargo's fingerprints are built on those, so 48 GB of good artifacts look newer than their sources and get rebuilt.clonefilepreserves timestamps inherently.Full writeup, with charts: https://claude.ai/code/artifact/0c44bc22-a7c8-495b-a1b1-6439a92d5c41
Notes for review
but worktreenamespace that Remove but worktree #15445 removed, deliberately scoped to creation only — nolist/integrate/destroy. Discovery already flows throughbut status, and removal throughgit worktree remove. Happy to hang it elsewhere if the namespace should stay retired.unsafe. Thebutcrate is#![deny(unsafe_code)]; there's a localized#[allow]with a SAFETY comment for theclonefilecall. No safe wrapper exists in the dependency tree, and per-file reflinking would cost one syscall per file across build output. Could move the syscall into a lower-level crate instead.Unsupportedand fall back to a normal checkout. Linux would need theFICLONEioctl plus a tree walk.target/andnode_modules/, but it also picks up anything else untracked in the source.