Crash with empty repo (repo does not have a commit checked out)

[Gara-Dorta]

Describe the problem

Copier crashes when trying to initialize an empty git repo inside the template.

Template

https://gitlab.ewi.tudelft.nl/reit/python-package-template/-/tree/clone-first?ref_type=heads

To Reproduce

1. Create a template
2. Create a empty repo inside the template. The file structure is

• template
  • .git
  • template files and folders
  • empty_repo
    • .git

4. Run copier copier copy --trust -r HEAD ./ ./empty_repo/

Logs

Traceback (most recent call last):
  File "/home/user/.local/bin/copier", line 10, in <module>
    sys.exit(CopierApp.run())
             ~~~~~~~~~~~~~^^
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/plumbum/cli/application.py", line 631, in run
    inst, retcode = subapp.run(argv, exit=False)
                    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/plumbum/cli/application.py", line 626, in run
    retcode = inst.main(*tailargs)
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/copier/_cli.py", line 275, in main
    return _handle_exceptions(inner)
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/copier/_cli.py", line 80, in _handle_exceptions
    exit_code = method()
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/copier/_cli.py", line 257, in inner
    run_copy(
    ~~~~~~~~^
        template_src,
        ^^^^^^^^^^^^^
    ...<13 lines>...
        skip_tasks=self.skip_tasks,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/copier/_main.py", line 1686, in run_copy
    with Worker(
         ~~~~~~^
        src_path=src_path,
        ^^^^^^^^^^^^^^^^^^
    ...<23 lines>...
        skip_tasks=skip_tasks,
        ^^^^^^^^^^^^^^^^^^^^^^
    ) as worker:
    ^
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/copier/_main.py", line 282, in __exit__
    raise value
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/copier/_main.py", line 1713, in run_copy
    worker.run_copy()
    ~~~~~~~~~~~~~~~^^
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/copier/_main.py", line 104, in _wrapper
    return func(*args, **kwargs)
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/copier/_main.py", line 1239, in run_copy
    self._print_message(self.template.message_before_copy)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/uv/python/cpython-3.13.5-linux-x86_64-gnu/lib/python3.13/functools.py", line 1026, in __get__
    val = self.func(instance)
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/copier/_template.py", line 363, in message_before_copy
    return self.config_data.get("message_before_copy", "")
           ^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/uv/python/cpython-3.13.5-linux-x86_64-gnu/lib/python3.13/functools.py", line 1026, in __get__
    val = self.func(instance)
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/copier/_template.py", line 303, in config_data
    result = filter_config(self._raw_config)[0]
                           ^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/uv/python/cpython-3.13.5-linux-x86_64-gnu/lib/python3.13/functools.py", line 1026, in __get__
    val = self.func(instance)
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/copier/_template.py", line 260, in _raw_config
    for p in self.local_abspath.glob("copier.*")
             ^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/uv/python/cpython-3.13.5-linux-x86_64-gnu/lib/python3.13/functools.py", line 1026, in __get__
    val = self.func(instance)
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/copier/_template.py", line 571, in local_abspath
    clone(
    ~~~~~^
        self.url_expanded,
        ^^^^^^^^^^^^^^^^^^
        self.ref or get_latest_tag(self.url_expanded, self.use_prereleases),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        location=str(self._temp_clone_path),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/copier/_vcs.py", line 237, in clone
    git("--git-dir=.git", f"--work-tree={url_abspath}", "add", "-A")
    ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/plumbum/commands/base.py", line 110, in __call__
    return self.run(args, **kwargs)[1]
           ~~~~~~~~^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/plumbum/commands/base.py", line 249, in run
    return p.run()
           ~~~~~^^
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/plumbum/commands/base.py", line 212, in runner
    return run_proc(p, retcode, timeout)
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/plumbum/commands/processes.py", line 330, in run_proc
    return _check_process(proc, retcode, timeout, stdout, stderr)
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/plumbum/commands/processes.py", line 19, in _check_process
    proc.verify(retcode, timeout, stdout, stderr)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/uv/tools/copier/lib/python3.13/site-packages/plumbum/machines/base.py", line 29, in verify
    raise ProcessExecutionError(
        getattr(self, "argv", None), self.returncode, stdout, stderr
    )
plumbum.commands.processes.ProcessExecutionError: Unexpected exit code: 128
Command line: | /usr/bin/git --git-dir=.git --work-tree=/home/user/code/reit/python-package-template add -A
Stderr:       | error: 'empty-repo/' does not have a commit checked out
              | fatal: adding files failed

Expected behavior

Copier would no crash in this situation.

Screenshots/screencasts/logs

No response

Operating system

Linux

Operating system distribution and version

Ubuntu 24.04

Copier version

9.15.1

Python version

3.13

Installation method

uvx+pypi

Additional context

This does not happen when running repo from the repo, e.g. copier copy --trust -r HEAD https://gitlab.ewi.tudelft.nl/reit/python-package-template ./empty-repo. It also doesn't matter if I run this inside the template repo, the empty repo or one level out, the error in the same.

[sisp]

Why do you want to nest Git repositories? And in particular, why do you want to generate a project into a Git repository nested inside your template repository? 🤔 I'm just trying to understand whether this is a reasonable use case that we should strive to support, and I currently lack the imagination why this is useful. 🤷

[Gara-Dorta]

Thanks for looking into this.

For the TLDR, my template use workflow is:

1. Create an empty repo in github
2. Clone the repo
3. Run my template to populate the repo

I do that instead of creating an empty repo with copier because it's fewer steps for my users.

My template lives in gitlab. I have some test CI jobs that creates a test repo and do checks on it. By default the gitlab CI clones the template repo in a folder and expects job work to be done in that folder. That leads to:

template
    .git
    template files and folders
    empty_repo
        .git

So if I work outside the template folder, I need to manually clean up that folder.

Additionally, when doing template repo development and locally creating repos with something like copier copy --trust -r HEAD path-to-template path-to-test-project. It is not obvious to new collaborators why this would fail if path-to-test-project happens to be inside path-to-template.

Here are a few links to the relevant parts of my template repo:

• Step by step guide of template setup https://gitlab.ewi.tudelft.nl/reit/python-package-template/-/tree/clone-first?ref_type=heads#setting-up-a-new-project
• CI job with the workarounds due to this issue https://gitlab.ewi.tudelft.nl/reit/python-package-template/-/blob/clone-first/.gitlab/workflows/test-project-generation.yml?ref_type=heads#L28-54
• Example failing CI job that shows that gitlab does not clean up outside of the repo folder https://gitlab.ewi.tudelft.nl/reit/python-package-template/-/jobs/4932120

[sisp]

Thanks for elaborating on your use case, @Gara-Dorta! 👍

I see two scenarios:

1. Your usage instructions recommend cloning an empty Git repo first and generating the project using the remote template repo. This works fine according to my reproduction. ✅

2. Your CI job that tests generating a project seems unnecessarily complicated (the way I see it) where you use the local template clone with the nested empty repo as destination. Instead of cleaning up the CI working directory, I think you can simply set

    .test_project_generation_base:
      ...
      variables:
        ...
   +    GIT_STRATEGY: none
      ...

   which disables repo cloning and initialization, then initialize the repo and generate the project using a few predefined CI variables:

     script:
       ...
       - git init
       - copier copy ... --vcs-ref $CI_COMMIT_BRANCH $CI_REPOSITORY_URL .
       ...

   Alternatively, I think you could use a destination inside /tmp/ and switch the current working directory there after the project generation has completed. I don't see a need to create a nested repo TBH.

My personal recommendation is creating a proper test suite using, e.g., pytest and generating test projects in temporary directories using, e.g., pytest's tmp_path_factory: pytest.TempPathFactory fixture, similar to how Copier's own test suite works. This allows you to test different scenarios in isolated temporary destinations and make granular assertions about the generated projects.

[Gara-Dorta]

Thanks for thinking along on how to solve this.

I agree with your suggestion, it's a cleaner way to go about it. However, I still think that the ginormous error from the first step is not very user friendly. Would you consider adding a shorter more meaningful error, e.g. copier does not support nested repos?

[Gara-Dorta]

In case it is useful to anybody, I ended up using mktemp -d because pulling the repo like it was suggested on every copier invocation made the CI almost twice as slow (from one and a half minutes to two and half).

Here is the relevant PR https://gitlab.ewi.tudelft.nl/reit/python-package-template/-/merge_requests/64/diffs