Skip to content

refactor: replace pixi_git implementation with the unified rattler_git crate - #6578

Draft
wolfv wants to merge 9 commits into
mainfrom
claude/git-cache-unification-7g1icn
Draft

refactor: replace pixi_git implementation with the unified rattler_git crate#6578
wolfv wants to merge 9 commits into
mainfrom
claude/git-cache-unification-7g1icn

Conversation

@wolfv

@wolfv wolfv commented Jul 10, 2026

Copy link
Copy Markdown
Member

Description

Companion to conda/rattler#2590, which unifies the low-level git cache implementation between pixi, rattler and rattler-build.

pixi_git and rattler_git share a common ancestor (both derived from uv-git) but drifted apart: pixi_git grew git-LFS support, a lazy HTTP client, cloneable errors and a graceful GitHub fast-path fallback, while rattler_git grew submodule handling with relative-URL resolution and corrupted-cache recovery. All of pixi_git's extras are now upstreamed into rattler_git (see the rattler PR), and this PR makes pixi consume it.

pixi_git becomes a thin shim that re-exports rattler_git and keeps only the pixi-specific pieces:

  • the PIXI_GIT_LFS environment variable (identical tri-state semantics, now parsed by rattler_git::source::lfs_enabled_from_env),
  • default_checkout_options() — submodules on, LFS following PIXI_GIT_LFS,
  • to_git_client() — bridges rattler_networking::LazyClient into rattler_git::LazyClient without losing laziness (the reqwest client is still only built if the GitHub fast path actually runs).

Because all consumers imported through pixi_git::…, no other pixi code changes except the single GitResolver::fetch call site, which now passes the checkout options explicitly.

pixi also gains the rattler_git features pixi_git lacked:

  • relative submodule URLs (../sibling.git) are resolved against the real source URL instead of the local bare database path, so they no longer fail after git clone --local,
  • protocol.file.allow=always for file-based submodules (Git ≥ 2.38.1),
  • corrupted git caches are detected (git rev-parse --git-dir validation) and re-cloned instead of erroring.

The LFS integration tests moved to rattler_git together with the code they test.

⚠️ The [patch.crates-io] entries for rattler_git, rattler_build_source_cache and rattler_build_networking are temporary — they point at the companion branches (conda/rattler#2590, prefix-dev/rattler-build#2648) and must be replaced by released versions before merging.

How Has This Been Tested?

  • cargo check --workspace --all-targets passes with the patched graph (this also compiles rattler-build's source cache against the new rattler_git).
  • cargo test -p pixi_git -p pixi_compute_sources and the full pixi_command_dispatcher integration suite (32 tests, incl. git checkout/pinning tests against local fixtures) pass.
  • The ported LFS tests pass in rattler_git itself (cargo test -p rattler_git on the rattler branch), covering PIXI_GIT_LFS parsing, pointer-vs-blob checkouts, and warm-cache LFS validation.
  • cargo clippy -p pixi_git -p pixi_compute_sources --all-targets and cargo fmt are clean.

AI Disclosure

  • This PR contains AI-generated content.
    • I have tested any AI-generated content in my PR.
    • I take responsibility for any AI-generated content in my PR.

Tools: Claude Code

We have source / git caches in rattler, rattler-build and Pixi. I think Pixi
"git cache" has drifted a bit from rattler's. Can you port everything needed
in Pixi to rattler, so that we can use one low level code base for this?
Then open a draft PR to Pixi and rattler with the changes, and possibly
rattler-build if needed. Things that might be different are related to lfs,
submodules, ..

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added sufficient tests to cover my changes.

🤖 Generated with Claude Code

https://claude.ai/code/session_013EJTWLNjUFMPPPoT8746Me


Generated by Claude Code

claude added 9 commits July 10, 2026 09:44
…t crate

pixi_git and rattler_git share a common ancestor (both derived from
uv-git) but drifted apart: pixi_git grew git-LFS support, a lazy HTTP
client, cloneable errors and a graceful GitHub fast-path fallback,
while rattler_git grew submodule handling with relative-URL resolution
and corrupted-cache recovery. All of pixi_git's extras have been
upstreamed into rattler_git, which is now the single low-level git
cache implementation shared by pixi, rattler and rattler-build.

pixi_git becomes a thin shim that re-exports rattler_git and keeps the
pixi-specific pieces:
- the `PIXI_GIT_LFS` environment variable (same tri-state semantics,
  now parsed by `rattler_git::source::lfs_enabled_from_env`),
- `default_checkout_options()` (submodules on, LFS from the env var),
- `to_git_client()` bridging `rattler_networking::LazyClient` into
  `rattler_git::LazyClient` without losing laziness.

pixi additionally gains rattler_git's submodule and cache-robustness
features that pixi_git lacked: relative submodule URLs are resolved
against the real source URL (instead of the local bare database path),
`protocol.file.allow=always` for file-based submodules, and corrupted
git caches are detected and re-cloned instead of failing.

The LFS integration tests moved to rattler_git together with the code.

The `[patch.crates-io]` entries for rattler_git,
rattler_build_source_cache and rattler_build_networking are TEMPORARY
until the unified crates are released.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EJTWLNjUFMPPPoT8746Me
rattler_prefix_guard is a path dependency of the patched rattler_git,
so it appeared both from crates.io and from the rattler git branch,
which 'cargo vendor --locked' rejects as a duplicate. Patching it from
the same branch unifies the source. Also bumps the patched revs
(clippy fix in rattler_git, py-rattler-build patch in rattler-build).

TEMPORARY, to be removed together with the other patch entries.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EJTWLNjUFMPPPoT8746Me
…d drive-letter endpoint fixes)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EJTWLNjUFMPPPoT8746Me
… remote fix)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EJTWLNjUFMPPPoT8746Me
…ed from rattler_networking upstream)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EJTWLNjUFMPPPoT8746Me
conda/rattler#2590 was squash-merged and its branch deleted, so the
branch-based [patch.crates-io] entries no longer resolve. Pin
rattler_git and rattler_prefix_guard to the merge commit on main
instead. Still TEMPORARY until the crates are released to crates.io.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EJTWLNjUFMPPPoT8746Me
conda/rattler#2590 is merged; point the temporary rattler_git and
rattler_prefix_guard [patch.crates-io] entries at the main branch
instead of the merge commit. Still TEMPORARY until the crates are
released to crates.io.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EJTWLNjUFMPPPoT8746Me
rattler_git 0.2.0 (the unified git cache, conda/rattler#2590) is
published, so drop the temporary rattler_git / rattler_prefix_guard
[patch.crates-io] git dependencies and depend on the released crate.

rattler_git 0.2.0 requires rattler_networking =0.30.2, which only exists
in the rattler 0.47.1 release train, so move the rattler stack forward:
rattler 0.46->0.47, rattler_conda_types 0.47->0.48, rattler_config
0.5->0.6, rattler_repodata_gateway 0.29->0.30, and rattler_upload
0.8->0.9 (the latter to reach rattler_digest 1.3.2, pulling sigstore
0.11). No pixi source changes were needed; the workspace compiles as-is.

rattler-build #2648 (its adoption of the unified cache) is merged to
main but not yet released, and its published crates still depend on the
pre-0.47 rattler stack. Point the whole interdependent rattler_build_*
set at rattler-build main so it stays consistent with rattler_git 0.2.
Both temporary patch groups come off once rattler-build is released.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013EJTWLNjUFMPPPoT8746Me
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants