Skip to content

feat(cli): one command to update submodules to tracked branch tips - #47

Merged
hartsock merged 3 commits into
mainfrom
feat/submodule-status-sync
Aug 25, 2026
Merged

feat(cli): one command to update submodules to tracked branch tips#47
hartsock merged 3 commits into
mainfrom
feat/submodule-status-sync

Conversation

@hartsock

@hartsock hartsock commented Aug 25, 2026

Copy link
Copy Markdown
Owner

Summary

A common use case: "keep this repo full of submodules up to date with the tip of
the branch each one is tracking."
This PR makes that one command.

gitxtend submodule sync ~/src/myrepo --commit
modA  9bfc36b -> 3bb8896
modB  2211b7a -> 4ff51bd  (devel)

2 submodules advanced, recorded as f14b311

It started as two APIs (submodule_status, sync_submodules). Wiring them up
end-to-end showed the APIs alone were not the feature.

Why the raw APIs weren't enough

git submodule update --remote moves each submodule's working tree to the tip of
its tracked branch — and stops there. It leaves behind:

  • a detached HEAD inside each submodule, and
  • a modified gitlink in the superproject index.

So on its own it makes the superproject dirty, not up to date: the next
plain git submodule update snaps every submodule back to the SHA the
superproject still records. Recording the bumps is a second, separate commit.

sync_submodules did step one and returned (ok, stderr) — no record of what
moved, and the caller still holding a dirty tree. update_submodules is the
whole loop:

  1. snapshot git submodule status
  2. git submodule update --init [--recursive] [--remote]
  3. snapshot again and diff — that diff is what lets the command say which
    submodules moved and from where, which git submodule update never reports
  4. with --commit, stage the moved top-level gitlinks and commit them

Idempotent: a repeat run reports nothing changed and makes no empty commit.

--commit is opt-in. Sync-and-report is the safe default; writing history is
asked for.

Two front ends, one program

The command ships as a standalone [[bin]] (roadmap M3 — no Python needed, good
for cron) and as a gitxtend console script in the wheel. Rather than two
argument parsers that agree only by review, the whole CLI lives in src/cli.rs
as a pure argv -> (code, stdout, stderr) function. The binary and
gitxtend._cli are both shims over it; cli_main exposes it to Python.

A parity test runs the binary and the console script over the same argv and
asserts byte-identical streams and exit codes — verified non-vacuous by pointing
it at a different binary, which fails it. python -m gitxtend works too.

No clap: the crate has two runtime dependencies on purpose and this surface is
two subcommands. The parser is a pure function, so it's tested directly rather
than by spawning anything.

Reporting the tracked branch, not git's describe output

git submodule status's trailing detail describes the checked-out commit, so
after a --remote update it reads (remotes/origin/HEAD) for every submodule —
including one that actually tracks devel. The branch is therefore read from
.gitmodules. A submodule with no branch = line is reported with no branch
rather than guessed at, since git follows the remote default there. Pinned by a
regression test asserting git's own detail is the misleading one.

Also in here: a destructive pre-existing test bug

The mandatory pre-push hook could never pass on this repo. cargo test under it
reported 62 failures, and the reason turned out to be worse than flakiness.

GIT_DIR overrides both git -C <path> and the child's current directory, and a
pre-push hook runs with it set. The git fixtures didn't scrub it, so cargo test
invoked from the hook pointed every fixture command at the developer's own
checkout
. Running it did all of this to this repository:

  • moved feat/submodule-status-sync onto a fixture commit, orphaning the real one
  • created a stray devel branch from a fixture checkout -b
  • set core.bare = true in .git/config, after which the main checkout answered
    fatal: this operation must be run in a work tree
  • wrote user.name = fix / user.email = fix@example.com and
    protocol.file.allow = always into the shared config

All repaired, and no commit was mis-authored. Every git process the crate spawns
now removes GIT_DIR and its six siblings (repo::AMBIENT_REPO_ENV). The
is_git_repo parity oracle was contaminated the same way — a raw
Command::new("git") answered about whatever repo the environment named — and is
fixed with it.

The regression tests assert the removals on the built Command rather than by
setting GIT_DIR for real: env vars are per-process and these tests run in
parallel threads, so a test that set one would corrupt its neighbours — the very
failure mode being fixed.

Scope limits, stated

  • --commit records top-level gitlinks only. With --recursive a nested
    submodule is outer/inner, which the superproject cannot stage; recording that
    needs a commit inside outer first, left to the caller rather than done
    implicitly.
  • These are the crate's first mutating operations, and the one place it does
    not use gix — submodule update/status stay delegated to the git CLI so the
    semantics are Git's own rather than a reimplementation. Called out in the README
    next to the write-side note it qualifies.

Test plan

  • cargo test --no-default-features88 tests, including the update/commit/
    idempotence/initialize/no-remote matrix against real two-repo submodule fixtures.
  • Python E2E — 29 tests, 100% package coverage. The helper drives
    gitxtend._cli.main (the real console-script entry point), not the compiled
    function beneath it, so the shim's stream forwarding and exit code are covered.
  • Both suites re-run with GIT_DIR/GIT_INDEX_FILE set to this repo, with local
    config, HEAD and the branch list snapshotted before and after and confirmed
    byte-identical. Before the fix above, that same run rewrote all three.
  • Driven by hand against a real two-submodule superproject: advance children →
    sync → record → re-run for idempotence → --json → exit codes (0/1/2).
  • CI + pre-push hook parity: both now build the CLI before the Python job, because
    the parity test needs the binary. It skips locally when absent but fails when
    CI is set
    — a silently-skipped parity test is a green that proves nothing.
  • Fixes the cargo fmt --all --check failure that was red on this branch.

Docs

README gains a "Keeping a repo full of submodules up to date" section (usage, why
--commit matters, the two front ends, the Python API); docs/API.md documents
update_submodules and the three-step rationale; docs/ROADMAP.md marks M3
started with gitxtend status as the next step.

@hartsock
hartsock force-pushed the feat/submodule-status-sync branch from 7e38249 to cce4ad4 Compare August 25, 2026 10:01
WHAT

Adds `gitxtend submodule sync` — the one-command form of "keep this repo full
of submodules up to date with the tip of the branch each one is tracking" —
plus the `update_submodules` API underneath it, on both front ends.

    gitxtend submodule sync ~/src/myrepo --commit

    modA  9bfc36b -> 3bb8896
    modB  2211b7a -> 4ff51bd  (devel)

    2 submodules advanced, recorded as f14b311

WHY THE APIs ALONE WERE NOT THE FEATURE

`git submodule update --remote` moves each submodule's working tree to the tip
of its tracked branch and stops. It leaves a detached HEAD in each submodule and
a MODIFIED GITLINK in the superproject — so on its own it makes the superproject
dirty rather than up to date, and the next plain `git submodule update` snaps
everything back to the SHA the superproject still records. Recording the bumps
is a second, separate commit.

`sync_submodules` did step one and returned `(ok, stderr)`: no record of what
moved, and the caller still holding a dirty tree. `update_submodules` is the
whole loop — snapshot, update, diff the snapshots, optionally record — and the
diff is what lets the command say which submodules moved and from where.
Idempotent: a repeat run reports nothing changed and makes no empty commit.

`--commit` is opt-in. Sync-and-report is the safe default; writing history is
asked for.

TWO FRONT ENDS, ONE PROGRAM

The command ships as a standalone `[[bin]]` (roadmap M3, no Python needed) AND
as a `gitxtend` console script in the wheel. Rather than two argument parsers
that agree only by review, the whole CLI lives in `src/cli.rs` as a pure
`argv -> (code, stdout, stderr)` function; the binary and `gitxtend._cli` are
both shims over it, and `cli_main` exposes it to Python. A parity test runs the
binary and the console script over the same argv and asserts byte-identical
streams and exit codes — verified non-vacuous by pointing it at a different
binary, which fails it.

No `clap`: the crate has two runtime dependencies on purpose and this surface is
two subcommands. The parser is a pure function, so it is tested directly rather
than by spawning anything.

REPORTING THE TRACKED BRANCH, NOT GIT'S DESCRIBE OUTPUT

`git submodule status`'s trailing detail describes the checked-out COMMIT, so
after a `--remote` update it reads `(remotes/origin/HEAD)` for every submodule —
including one that actually tracks `devel`. The branch is therefore read from
`.gitmodules`, and a submodule with no `branch =` line is reported with no
branch rather than guessed at (git follows the remote default there). Pinned by
a regression test that asserts git's own detail is the misleading one.

SCOPE LIMITS, STATED

- `--commit` stages TOP-LEVEL gitlinks only. With `--recursive` a nested
  submodule is `outer/inner`, which the superproject cannot stage; recording it
  needs a commit inside `outer` first, left to the caller rather than done
  implicitly.
- These are the crate's first mutating operations, and the one place it does not
  use gix — submodule update/status stay delegated to the `git` CLI so the
  semantics are Git's own. Called out in the README next to the write-side note
  it qualifies.

ALSO IN HERE

- Fixes the CI failure on this branch: a stray blank line failed
  `cargo fmt --all --check`.
- CI + pre-push hook parity: both now build the CLI before the Python E2E job,
  because the parity test needs the binary. The test SKIPS locally when it is
  missing but FAILS when `CI` is set — a silently-skipped parity test is a green
  that proves nothing.
- The Python E2E helper drives `gitxtend._cli.main` (the real console-script
  entry point) rather than the compiled function beneath it, so the shim's
  stream forwarding and exit code are covered. Package coverage 100%.

Verified: fmt, clippy (core, --all-targets, and python) -D warnings,
`cargo test --no-default-features` (86 tests), the Python E2E suite (28 tests,
100% coverage), and the command driven by hand against a real two-submodule
superproject — advance, sync, record, re-run for idempotence.
…eal repo

WHAT

Every `git` process this crate spawns now removes the environment variables by
which an ambient git points its children at *its* repository — `GIT_DIR`,
`GIT_WORK_TREE`, `GIT_INDEX_FILE`, `GIT_OBJECT_DIRECTORY`,
`GIT_ALTERNATE_OBJECT_DIRECTORIES`, `GIT_COMMON_DIR`, `GIT_NAMESPACE` — collected
as `repo::AMBIENT_REPO_ENV` and applied in `fixtures::git`, `run_git`, the
submodule helper, the `is_git_repo` parity oracle, and the Python E2E `_ENV`.

WHY (this is not hypothetical)

`GIT_DIR` overrides both `git -C <path>` and the child's current directory. A
pre-push hook runs with it set. So `cargo test` invoked from the hook pointed
every fixture command at the developer's own checkout instead of the temp dir.
Running it did all of this to this repository:

  - moved `feat/submodule-status-sync` onto a fixture commit ("child v1"),
    orphaning the real commit
  - created a stray `devel` branch from fixture `checkout -b`
  - set `core.bare = true` in `.git/config`, after which the main checkout
    answered "fatal: this operation must be run in a work tree"
  - wrote `user.name = fix` / `user.email = fix@example.com` and
    `protocol.file.allow = always` into the shared config

All repaired; no commit was mis-authored (the identity landed after the commit).

It also meant the mandatory pre-push hook could never pass: `cargo test` under it
reported 62 failures — `remote add origin` failing with "remote origin already
exists" because it was talking to the real repo. That is what surfaced this.
`git -C` and `current_dir` are NOT protection; only removing the variables is.

TESTS

`fixture_git_scrubs_the_ambient_repo_env` and `git_in_scrubs_the_ambient_repo_env`
assert the removals on the *built* `Command` rather than by setting `GIT_DIR` for
real: environment variables are per-process and these tests run in parallel
threads, so a test that set one would corrupt its neighbours — the very failure
mode being fixed. `test_fixture_env_scrubs_the_ambient_repo_pointers` covers the
Python fixture env.

The parity oracle in `is_git_repo` was contaminated the same way and is fixed
with it: a raw `Command::new("git")` there answered about whatever repo the
environment named, so the assertion compared gix's answer about the temp dir
against git's answer about a *different* repository. It now goes through the
scrubbed `fixtures::git_command`.

VERIFIED

Both suites run clean with `GIT_DIR`/`GIT_INDEX_FILE` set to this repo — 88 Rust
tests, 29 Python tests, 100% package coverage — with the local config, `HEAD`,
and the branch list snapshotted before and after and confirmed byte-identical.
Before this change that same run rewrote all three.
@hartsock hartsock changed the title Add submodule status and sync APIs feat(cli): one command to update submodules to tracked branch tips Aug 25, 2026
@hartsock
hartsock merged commit 1a81228 into main Aug 25, 2026
2 checks passed
@hartsock
hartsock deleted the feat/submodule-status-sync branch August 25, 2026 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant