Skip to content

Try workaround for Bundler + git lfs issue (git-lfs/git-lfs#1889)#2

Merged
trinitronx merged 1 commit into
developfrom
try-bundler-git-lfs-workaround
Jun 24, 2025
Merged

Try workaround for Bundler + git lfs issue (git-lfs/git-lfs#1889)#2
trinitronx merged 1 commit into
developfrom
try-bundler-git-lfs-workaround

Conversation

@trinitronx
Copy link
Copy Markdown
Contributor

@trinitronx trinitronx commented Jun 24, 2025

Encountered an error when using Gemfile with github: source to use soloist
gem from another project.

Error was:

Fetching https://github.com/LyraPhase/soloist.git
Downloading test/fixtures/cloud-init-seed.iso.qcow2 (459 KB)
Error downloading object: test/fixtures/cloud-init-seed.iso.qcow2 (a2643ef): Smudge error: Error downloading test/fixtures/cloud-init-seed.iso.qcow2 (a2643ef240ff6a0decafaeec8262c687c0d556000bfb6bfbbad559961b7e93be): error transferring "a2643ef240ff6a0decafaeec8262c687c0d556000bfb6bfbbad559961b7e93be": [0] remote missing object a2643ef240ff6a0decafaeec8262c687c0d556000bfb6bfbbad559961b7e93be

Errors logged to '/Users/exampleuser/src/pub/soloist/sprout-wrap/vendor/bundle/ruby/3.1.0/bundler/gems/soloist-537f1668364a/.git/lfs/logs/20250623T113632.054386.log'.
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: test/fixtures/cloud-init-seed.iso.qcow2: smudge filter lfs failed
Git error: command `git reset --hard 537f1668364a946570a384134cca85d8e7437674` in directory /Users/exampleuser/src/pub/soloist/sprout-wrap/vendor/bundle/ruby/3.1.0/bundler/gems/soloist-537f1668364a has failed.
Revision 537f1668364a946570a384134cca85d8e7437674 does not exist in the repository https://github.com/LyraPhase/soloist.git. Maybe you misspelled it?
If this error persists you could try removing the cache directory '/Users/exampleuser/src/pub/soloist/sprout-wrap/vendor/bundle/ruby/3.1.0/bundler/gems/soloist-537f1668364a'

Testing the GIT_LFS_SKIP_SMUDGE=1 bundle install command worked.

The repo using git lfs was being cloned by Bundler into a cache directory,
then re-cloned from that local filesystem location to the install location.
However, that local cached clone did not clone LFS files by default. Going into
the directory with pushd $(bundle info --path soloist) and showing the remote
with: git remote -vv shows what was going on.

$ pushd "$(bundle info --path soloist)"
$ pwd
/Users/exampleuser/src/pub/soloist/sprout-wrap/vendor/bundle/ruby/3.1.0/bundler/gems/soloist-537f1668364a
$ git remote -vv
git remote -vv
origin	/Users/exampleuser/src/pub/soloist/sprout-wrap/vendor/bundle/ruby/3.1.0/cache/bundler/git/soloist-e4905c8fdc0d00e01f5e0e7c545bde55dee890e5 (fetch)
origin	/Users/exampleuser/src/pub/soloist/sprout-wrap/vendor/bundle/ruby/3.1.0/cache/bundler/git/soloist-e4905c8fdc0d00e01f5e0e7c545bde55dee890e5 (push)
$ pushd /Users/exampleuser/src/pub/soloist/sprout-wrap/vendor/bundle/ruby/3.1.0/cache/bundler/git/soloist-e4905c8fdc0d00e01f5e0e7c545bde55dee890e5
$ git remote -vv
origin	git@github.com:LyraPhase/soloist.git (fetch)
origin	git@github.com:LyraPhase/soloist.git (push)
$ git branch -vv
  develop      537f166 Merge pull request #1 from LyraPhase/upgrade-gems
  upgrade-gems a369f81 Update README badges
* master       1adea10 Bump to 1.0.4
$ git checkout develop
fatal: this operation must be run in a work tree
$ ls
FETCH_HEAD  HEAD        config      description hooks       info        lfs         objects     packed-refs refs

So, we can see the two repo paths where the first is cloned into Bundler's cache
from GitHub as a bare clone of master branch without a work tree. Any Git LFS
files were not pulled for the develop branch which bundler was trying to pull,
(gem 'soloist', github: 'LyraPhase/soloist', ref: 'develop' in Gemfile).
The second clone into the non-cache vendor directory is cloned from the cache,
using local filesystem as origin. However, when develop is checked out by
Bundler via git reset --hard, no LFS files are able to be cloned from the
cached clone directory because they were not pulled from GitHub.

That seems to be the crux of the issue. To workaround this, either we must
always use GIT_LFS_SKIP_SMUDGE=1 when using bundler install with a
git source when that repo has LFS files, or try to force the upstream LFS
file location to be GitHub for all clones. That is what we will do here.

If that doesn't work, the only path left for convenient clones is to completely
rewrite history and remove usage of git LFS entirely from the repo. That'll be
our last resort. Otherwise, as soloist gem has been abandoned on RubyGems,
we will have to either rename & re-release on RubyGems or else always use a git
source in Gemfile + GIT_LFS_SKIP_SMUDGE=1 for dependent projects. The only
file tracked by git LFS so far is the cloud-init seed ISO qcow2 image, and it's
only used during CI for the Vagrant VM so NFSv4 shares will work properly.
We may end up deleting this file and forcing regeneration during CI another way.

@trinitronx trinitronx added bug Something isn't working enhancement New feature or request labels Jun 24, 2025
@trinitronx trinitronx self-assigned this Jun 24, 2025
@trinitronx trinitronx added bug Something isn't working enhancement New feature or request labels Jun 24, 2025
Encountered an error when using `Gemfile` with `github: ` source to use soloist
gem from another project.

Error was:

    Fetching https://github.com/LyraPhase/soloist.git
    Downloading test/fixtures/cloud-init-seed.iso.qcow2 (459 KB)
    Error downloading object: test/fixtures/cloud-init-seed.iso.qcow2 (a2643ef): Smudge error: Error downloading test/fixtures/cloud-init-seed.iso.qcow2 (a2643ef240ff6a0decafaeec8262c687c0d556000bfb6bfbbad559961b7e93be): error transferring "a2643ef240ff6a0decafaeec8262c687c0d556000bfb6bfbbad559961b7e93be": [0] remote missing object a2643ef240ff6a0decafaeec8262c687c0d556000bfb6bfbbad559961b7e93be

    Errors logged to '/Users/exampleuser/src/pub/soloist/sprout-wrap/vendor/bundle/ruby/3.1.0/bundler/gems/soloist-537f1668364a/.git/lfs/logs/20250623T113632.054386.log'.
    Use `git lfs logs last` to view the log.
    error: external filter 'git-lfs filter-process' failed
    fatal: test/fixtures/cloud-init-seed.iso.qcow2: smudge filter lfs failed
    Git error: command `git reset --hard 537f166` in directory /Users/exampleuser/src/pub/soloist/sprout-wrap/vendor/bundle/ruby/3.1.0/bundler/gems/soloist-537f1668364a has failed.
    Revision 537f166 does not exist in the repository https://github.com/LyraPhase/soloist.git. Maybe you misspelled it?
    If this error persists you could try removing the cache directory '/Users/exampleuser/src/pub/soloist/sprout-wrap/vendor/bundle/ruby/3.1.0/bundler/gems/soloist-537f1668364a'

Testing the `GIT_LFS_SKIP_SMUDGE=1 bundle install` command worked.

The repo using `git lfs` was being cloned by Bundler into a cache directory,
then re-cloned from that local filesystem location to the install location.
However, that local cached clone did not clone LFS files by default. Going into
the directory with `pushd $(bundle info --path soloist)` and showing the remote
with: `git remote -vv` shows what was going on.

    $ pushd "$(bundle info --path soloist)"
    $ pwd
    /Users/exampleuser/src/pub/soloist/sprout-wrap/vendor/bundle/ruby/3.1.0/bundler/gems/soloist-537f1668364a
    $ git remote -vv
    git remote -vv
    origin	/Users/exampleuser/src/pub/soloist/sprout-wrap/vendor/bundle/ruby/3.1.0/cache/bundler/git/soloist-e4905c8fdc0d00e01f5e0e7c545bde55dee890e5 (fetch)
    origin	/Users/exampleuser/src/pub/soloist/sprout-wrap/vendor/bundle/ruby/3.1.0/cache/bundler/git/soloist-e4905c8fdc0d00e01f5e0e7c545bde55dee890e5 (push)
    $ pushd /Users/exampleuser/src/pub/soloist/sprout-wrap/vendor/bundle/ruby/3.1.0/cache/bundler/git/soloist-e4905c8fdc0d00e01f5e0e7c545bde55dee890e5
    $ git remote -vv
    origin	git@github.com:LyraPhase/soloist.git (fetch)
    origin	git@github.com:LyraPhase/soloist.git (push)
    $ git branch -vv
      develop      537f166 Merge pull request #1 from LyraPhase/upgrade-gems
      upgrade-gems a369f81 Update README badges
    * master       1adea10 Bump to 1.0.4
    $ git checkout develop
    fatal: this operation must be run in a work tree
    $ ls
    FETCH_HEAD  HEAD        config      description hooks       info        lfs         objects     packed-refs refs

So, we can see the two repo paths where the first is cloned into Bundler's cache
from GitHub as a bare clone of `master` branch without a work tree. Any Git LFS
files were not pulled for the `develop` branch which bundler was trying to pull,
(`gem 'soloist', github: 'LyraPhase/soloist', ref: 'develop'` in `Gemfile`).
The second clone into the non-cache `vendor` directory is cloned from the cache,
using local filesystem as origin.  However, when `develop` is checked out by
Bundler via `git reset --hard`, no LFS files are able to be cloned from the
cached clone directory because they were not pulled from GitHub.

That seems to be the crux of the issue. To workaround this, either we must
always use `GIT_LFS_SKIP_SMUDGE=1` when using `bundler install` with a
git source when that repo has LFS files, or try to force the upstream LFS
file location to be GitHub for all clones.  That is what we will do here.

If that doesn't work, the only path left for convenient clones is to completely
rewrite history and remove usage of git LFS entirely from the repo. That'll be
our last resort.  Otherwise, as `soloist` gem has been abandoned on RubyGems,
we will have to either rename & re-release on RubyGems or else always use a git
source in `Gemfile` + `GIT_LFS_SKIP_SMUDGE=1` for dependent projects. The only
file tracked by git LFS so far is the cloud-init seed ISO qcow2 image, and it's
only used during CI for the Vagrant VM so NFSv4 shares will work properly.
We may end up deleting this file and forcing regeneration during CI another way.
@trinitronx trinitronx force-pushed the try-bundler-git-lfs-workaround branch from 8f20df1 to d02e2f2 Compare June 24, 2025 08:14
@trinitronx
Copy link
Copy Markdown
Contributor Author

It would seem that after testing this locally it should work. We will merge both to master and develop branches first just to be safe.

Another workaround found during testing would involve somehow hooking more commands during the initial bundle install to force it to run: git lfs fetch --all from the first bare clone:

$ git lfs fetch --all origin master develop 
fetch: 2 objects found, done.

Then the lfs directory will contain the fetched LFS files from GitHub in the bare repo. Any subsequent local filesystem clones will be able to pull those normally.

However, Bundler doesn't have a simple method to hook into this to integrate with git or github sources using git LFS easily. So, we'll opt to configure all cloned repos to point at GitHub in .lfsconfig.

@trinitronx trinitronx merged commit bb822cf into develop Jun 24, 2025
20 checks passed
@trinitronx trinitronx deleted the try-bundler-git-lfs-workaround branch June 24, 2025 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant