feat(cli): one command to update submodules to tracked branch tips - #47
Merged
Conversation
hartsock
force-pushed
the
feat/submodule-status-sync
branch
from
August 25, 2026 10:01
7e38249 to
cce4ad4
Compare
WHAT
Adds `gitxtend submodule sync` — the one-command form of "keep this repo full
of submodules up to date with the tip of the branch each one is tracking" —
plus the `update_submodules` API underneath it, on both front ends.
gitxtend submodule sync ~/src/myrepo --commit
modA 9bfc36b -> 3bb8896
modB 2211b7a -> 4ff51bd (devel)
2 submodules advanced, recorded as f14b311
WHY THE APIs ALONE WERE NOT THE FEATURE
`git submodule update --remote` moves each submodule's working tree to the tip
of its tracked branch and stops. It leaves a detached HEAD in each submodule and
a MODIFIED GITLINK in the superproject — so on its own it makes the superproject
dirty rather than up to date, and the next plain `git submodule update` snaps
everything back to the SHA the superproject still records. Recording the bumps
is a second, separate commit.
`sync_submodules` did step one and returned `(ok, stderr)`: no record of what
moved, and the caller still holding a dirty tree. `update_submodules` is the
whole loop — snapshot, update, diff the snapshots, optionally record — and the
diff is what lets the command say which submodules moved and from where.
Idempotent: a repeat run reports nothing changed and makes no empty commit.
`--commit` is opt-in. Sync-and-report is the safe default; writing history is
asked for.
TWO FRONT ENDS, ONE PROGRAM
The command ships as a standalone `[[bin]]` (roadmap M3, no Python needed) AND
as a `gitxtend` console script in the wheel. Rather than two argument parsers
that agree only by review, the whole CLI lives in `src/cli.rs` as a pure
`argv -> (code, stdout, stderr)` function; the binary and `gitxtend._cli` are
both shims over it, and `cli_main` exposes it to Python. A parity test runs the
binary and the console script over the same argv and asserts byte-identical
streams and exit codes — verified non-vacuous by pointing it at a different
binary, which fails it.
No `clap`: the crate has two runtime dependencies on purpose and this surface is
two subcommands. The parser is a pure function, so it is tested directly rather
than by spawning anything.
REPORTING THE TRACKED BRANCH, NOT GIT'S DESCRIBE OUTPUT
`git submodule status`'s trailing detail describes the checked-out COMMIT, so
after a `--remote` update it reads `(remotes/origin/HEAD)` for every submodule —
including one that actually tracks `devel`. The branch is therefore read from
`.gitmodules`, and a submodule with no `branch =` line is reported with no
branch rather than guessed at (git follows the remote default there). Pinned by
a regression test that asserts git's own detail is the misleading one.
SCOPE LIMITS, STATED
- `--commit` stages TOP-LEVEL gitlinks only. With `--recursive` a nested
submodule is `outer/inner`, which the superproject cannot stage; recording it
needs a commit inside `outer` first, left to the caller rather than done
implicitly.
- These are the crate's first mutating operations, and the one place it does not
use gix — submodule update/status stay delegated to the `git` CLI so the
semantics are Git's own. Called out in the README next to the write-side note
it qualifies.
ALSO IN HERE
- Fixes the CI failure on this branch: a stray blank line failed
`cargo fmt --all --check`.
- CI + pre-push hook parity: both now build the CLI before the Python E2E job,
because the parity test needs the binary. The test SKIPS locally when it is
missing but FAILS when `CI` is set — a silently-skipped parity test is a green
that proves nothing.
- The Python E2E helper drives `gitxtend._cli.main` (the real console-script
entry point) rather than the compiled function beneath it, so the shim's
stream forwarding and exit code are covered. Package coverage 100%.
Verified: fmt, clippy (core, --all-targets, and python) -D warnings,
`cargo test --no-default-features` (86 tests), the Python E2E suite (28 tests,
100% coverage), and the command driven by hand against a real two-submodule
superproject — advance, sync, record, re-run for idempotence.
…eal repo
WHAT
Every `git` process this crate spawns now removes the environment variables by
which an ambient git points its children at *its* repository — `GIT_DIR`,
`GIT_WORK_TREE`, `GIT_INDEX_FILE`, `GIT_OBJECT_DIRECTORY`,
`GIT_ALTERNATE_OBJECT_DIRECTORIES`, `GIT_COMMON_DIR`, `GIT_NAMESPACE` — collected
as `repo::AMBIENT_REPO_ENV` and applied in `fixtures::git`, `run_git`, the
submodule helper, the `is_git_repo` parity oracle, and the Python E2E `_ENV`.
WHY (this is not hypothetical)
`GIT_DIR` overrides both `git -C <path>` and the child's current directory. A
pre-push hook runs with it set. So `cargo test` invoked from the hook pointed
every fixture command at the developer's own checkout instead of the temp dir.
Running it did all of this to this repository:
- moved `feat/submodule-status-sync` onto a fixture commit ("child v1"),
orphaning the real commit
- created a stray `devel` branch from fixture `checkout -b`
- set `core.bare = true` in `.git/config`, after which the main checkout
answered "fatal: this operation must be run in a work tree"
- wrote `user.name = fix` / `user.email = fix@example.com` and
`protocol.file.allow = always` into the shared config
All repaired; no commit was mis-authored (the identity landed after the commit).
It also meant the mandatory pre-push hook could never pass: `cargo test` under it
reported 62 failures — `remote add origin` failing with "remote origin already
exists" because it was talking to the real repo. That is what surfaced this.
`git -C` and `current_dir` are NOT protection; only removing the variables is.
TESTS
`fixture_git_scrubs_the_ambient_repo_env` and `git_in_scrubs_the_ambient_repo_env`
assert the removals on the *built* `Command` rather than by setting `GIT_DIR` for
real: environment variables are per-process and these tests run in parallel
threads, so a test that set one would corrupt its neighbours — the very failure
mode being fixed. `test_fixture_env_scrubs_the_ambient_repo_pointers` covers the
Python fixture env.
The parity oracle in `is_git_repo` was contaminated the same way and is fixed
with it: a raw `Command::new("git")` there answered about whatever repo the
environment named, so the assertion compared gix's answer about the temp dir
against git's answer about a *different* repository. It now goes through the
scrubbed `fixtures::git_command`.
VERIFIED
Both suites run clean with `GIT_DIR`/`GIT_INDEX_FILE` set to this repo — 88 Rust
tests, 29 Python tests, 100% package coverage — with the local config, `HEAD`,
and the branch list snapshotted before and after and confirmed byte-identical.
Before this change that same run rewrote all three.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A common use case: "keep this repo full of submodules up to date with the tip of
the branch each one is tracking." This PR makes that one command.
gitxtend submodule sync ~/src/myrepo --commitIt started as two APIs (
submodule_status,sync_submodules). Wiring them upend-to-end showed the APIs alone were not the feature.
Why the raw APIs weren't enough
git submodule update --remotemoves each submodule's working tree to the tip ofits tracked branch — and stops there. It leaves behind:
So on its own it makes the superproject dirty, not up to date: the next
plain
git submodule updatesnaps every submodule back to the SHA thesuperproject still records. Recording the bumps is a second, separate commit.
sync_submodulesdid step one and returned(ok, stderr)— no record of whatmoved, and the caller still holding a dirty tree.
update_submodulesis thewhole loop:
git submodule statusgit submodule update --init [--recursive] [--remote]submodules moved and from where, which
git submodule updatenever reports--commit, stage the moved top-level gitlinks and commit themIdempotent: a repeat run reports nothing changed and makes no empty commit.
--commitis opt-in. Sync-and-report is the safe default; writing history isasked for.
Two front ends, one program
The command ships as a standalone
[[bin]](roadmap M3 — no Python needed, goodfor cron) and as a
gitxtendconsole script in the wheel. Rather than twoargument parsers that agree only by review, the whole CLI lives in
src/cli.rsas a pure
argv -> (code, stdout, stderr)function. The binary andgitxtend._cliare both shims over it;cli_mainexposes it to Python.A parity test runs the binary and the console script over the same argv and
asserts byte-identical streams and exit codes — verified non-vacuous by pointing
it at a different binary, which fails it.
python -m gitxtendworks too.No
clap: the crate has two runtime dependencies on purpose and this surface istwo subcommands. The parser is a pure function, so it's tested directly rather
than by spawning anything.
Reporting the tracked branch, not git's describe output
git submodule status's trailing detail describes the checked-out commit, soafter a
--remoteupdate it reads(remotes/origin/HEAD)for every submodule —including one that actually tracks
devel. The branch is therefore read from.gitmodules. A submodule with nobranch =line is reported with no branchrather than guessed at, since git follows the remote default there. Pinned by a
regression test asserting git's own detail is the misleading one.
Also in here: a destructive pre-existing test bug
The mandatory pre-push hook could never pass on this repo.
cargo testunder itreported 62 failures, and the reason turned out to be worse than flakiness.
GIT_DIRoverrides bothgit -C <path>and the child's current directory, and apre-push hook runs with it set. The git fixtures didn't scrub it, so
cargo testinvoked from the hook pointed every fixture command at the developer's own
checkout. Running it did all of this to this repository:
feat/submodule-status-synconto a fixture commit, orphaning the real onedevelbranch from a fixturecheckout -bcore.bare = truein.git/config, after which the main checkout answeredfatal: this operation must be run in a work treeuser.name = fix/user.email = fix@example.comandprotocol.file.allow = alwaysinto the shared configAll repaired, and no commit was mis-authored. Every
gitprocess the crate spawnsnow removes
GIT_DIRand its six siblings (repo::AMBIENT_REPO_ENV). Theis_git_repoparity oracle was contaminated the same way — a rawCommand::new("git")answered about whatever repo the environment named — and isfixed with it.
The regression tests assert the removals on the built
Commandrather than bysetting
GIT_DIRfor real: env vars are per-process and these tests run inparallel threads, so a test that set one would corrupt its neighbours — the very
failure mode being fixed.
Scope limits, stated
--commitrecords top-level gitlinks only. With--recursivea nestedsubmodule is
outer/inner, which the superproject cannot stage; recording thatneeds a commit inside
outerfirst, left to the caller rather than doneimplicitly.
not use gix — submodule update/status stay delegated to the
gitCLI so thesemantics are Git's own rather than a reimplementation. Called out in the README
next to the write-side note it qualifies.
Test plan
cargo test --no-default-features— 88 tests, including the update/commit/idempotence/initialize/no-remote matrix against real two-repo submodule fixtures.
gitxtend._cli.main(the real console-script entry point), not the compiledfunction beneath it, so the shim's stream forwarding and exit code are covered.
GIT_DIR/GIT_INDEX_FILEset to this repo, with localconfig,
HEADand the branch list snapshotted before and after and confirmedbyte-identical. Before the fix above, that same run rewrote all three.
sync → record → re-run for idempotence →
--json→ exit codes (0/1/2).the parity test needs the binary. It skips locally when absent but fails when
CIis set — a silently-skipped parity test is a green that proves nothing.cargo fmt --all --checkfailure that was red on this branch.Docs
README gains a "Keeping a repo full of submodules up to date" section (usage, why
--commitmatters, the two front ends, the Python API);docs/API.mddocumentsupdate_submodulesand the three-step rationale;docs/ROADMAP.mdmarks M3started with
gitxtend statusas the next step.