updating.md

Updating a project

Copier enables the code lifecycle management for generated projects. There are several common use cases for that, including but not limited to

1. Update the answers to previous questions

   Questions can be reanswered to fit the latest requirements of the generated projects. This is helpful, especially when the template includes optional tools that fit into different phases of a project. In that case, template consumers are able to activate the optional tools gradually when the project matures.

2. Sync updates from evolved templates

   The template creators might update the template to add new functionalities or bug fixes. The generated project can get updated if the template consumers want to keep it to the latest version.

The best way to update a project from its template is when all of these conditions are true:

1. The destination folder includes a valid .copier-answers.yml file.
2. The template is versioned with Git (with tags).
3. The destination folder is versioned with Git.

If that's your case, then just enter the destination folder, make sure git status shows it clean, and run:

copier update

This will read all available Git tags, will compare them using PEP 440, and will check out the latest one before updating. To update to the latest commit, add --vcs-ref=HEAD. You can use any other Git ref you want.

When updating, Copier will do its best to respect your project evolution by using the answers you provided when copied last time. However, sometimes it's impossible for Copier to know how to merge the changes from the evolved template into the evolved project. In those cases, Copier updates a conflicting file with conflict markers in the same ways as a git merge command encounters conflicts; in fact, Copier uses git merge internally. For more information, see the "Checking Out Conflicts" section of the git documentation.

If the update results in conflicts, you should review those manually before committing.

You probably don't want to lose important changes or to include merge conflicts in your Git history, but if you aren't careful, it's easy to make mistakes.

That's why the recommended way to prevent these mistakes is to add a pre-commit (or equivalent) hook that forbids committing conflict markers.

Preventing Commit of Merge Conflicts

You need to check for conflict markers in your files:

repos:
    - repo: https://github.com/pre-commit/pre-commit-hooks
      rev: v4.3.0
      hooks:
          # Prevent committing inline conflict markers
          - id: check-merge-conflict
            args: [--assume-in-merge]

Never change the answers file manually

!!! important

**Never** update `.copier-answers.yml` manually.

This will trick Copier, making it believe that those modified answers produced the
current subproject, while it was produced by a different answers set. This will
produce unpredictable behavior of the smart diff algorithm used for updates, which
may work under certain circumstances, but not always.

**This is an unsupported way to update**. Please do not open issues if you updated
this way.

The correct process to update a subproject is:

1. Run copier update.
2. Answer the questions. They'll default to your answers on your last update.

If you want to just reuse all previous answers:

copier update --defaults

If you want to change just one question, and leave all others untouched, and don't want to go through the whole questionnaire again:

copier update --defaults --data updated_question="my new answer"

You can achieve the same using a data file:

echo "updated_question: my new answer" > /tmp/data-file.yaml
copier update --defaults --data-file /tmp/data-file.yaml

If you want to update the answers to all questions, but not the template:

copier update --vcs-ref=:current:

How the update works

To understand how the updating process works, take a look at this diagram:

graph TD

%% nodes ----------------------------------------------------------
template_repo("template repository")
template_current("/tmp/template-old<br>(current tag)")
template_latest("/tmp/template-new<br>(latest tag)")

project_regen_current("/tmp/project-old<br>(fresh, current version)")
project_regen_latest("/tmp/project-new<br>(fresh, latest version)")
project_current("current project")
project_half("half migrated<br>project")
project_updated("updated project")
project_full("fully updated<br>and migrated project")

update["3-way merge"]
regen_current["generate and run tasks<br>& apply pre-migrations"]
regen_latest["generate and run tasks"]

%% edges ----------------------------------------------------------
        template_repo --> |git clone| template_current
        template_repo --> |git clone| template_latest

     template_current --> regen_current
      project_current .-> |use answers| regen_current
        regen_current --> project_regen_current
      template_latest --> regen_latest
         regen_latest --> project_regen_latest
      project_current --> |apply pre-migrations| project_half
         project_half .-> |use answers| regen_latest
         project_half --> update
project_regen_current --> update
 project_regen_latest --> update
               update --> project_updated
      project_updated --> |apply post-migrations| project_full

%% style ----------------------------------------------------------
classDef blackborder stroke:#000;
class regen_current,regen_latest,update blackborder;

Loading

As you can see here, copier does several things:

• Regenerate the project fresh from the current template version, using the project's existing answers (with pre-migrations applied afterwards) – this becomes the merge-base.
• Regenerate the project fresh from the latest template version, using the same answers (with pre-migrations applied to the project beforehand).
• Build a synthetic Git commit graph from these three states: the current-version regeneration (common ancestor), the latest-version regeneration, and the actual current project.
• Perform a Git 3-way merge (using git merge) of the latest-version regeneration into the current project, using the current-version regeneration as their common ancestor – conflicts are marked like any normal git merge conflict.
• Run post-migrations on the merged result to produce the fully updated project.

Handling of deleted paths

Template-based files/directories that were deleted in the generated project are automatically excluded from updates. If you want to recover such a file later on, you can run copier recopy and recommit it to your repository. Subsequent updates for the path will then be respected again.

An exception to this behavior applies to paths that are matched by skip_if_exists. Their presence is always ensured, even during an update operation.

Recover from a broken update

Usually Copier will replay the last project generation without problems. However, sometimes that process can break. Examples:

• When the last update was relying on some external resources that are no longer available.
• When the old and new versions of the template depend on different incompatible versions of the same Jinja extension, but Copier can only use one.
• When the old version of the template was built for an older version of Copier.

Generally, you should keep your templates as pure and simple as possible to avoid those situations. But it can still happen.

To overcome this, use the copier recopy command, which will discard all the smart update algorithm explained above. It will behave just like if you were applying the template for the first time, but it will keep your answers from the last update.

Of course, the experience will be less satisfactory. The new template will override any changes found in your local project. But you can use a Git diff tool to overcome that. After doing this, further updates generally keep on working as usual.

Aborting an update

When you're not happy with the result of a copier update run or unsure about adding the introduced changes to your code base, specifically when you have unpleasant conflicts, it's not 100% obvious how to get back to the previously clean copy of your branch. The following strategies won't work:

• git checkout <branch> – error: you need to resolve your current index first
• git checkout . – error: path '<filename>' is unmerged

Here is what you can do using Git in the terminal to throw away all changes:

git merge --abort

or

git reset           # throw away merge conflict information
git checkout .      # restore modified files
git clean -d -i     # remove untracked files and folders

If you want fine-grained control to restore files selectively, read the output of the git status command attentively. It shows all the commands you may need as hints!

Checking for updates

Copier provides a subcommand copier check-update that can be used to check if there are updates to the template used to generate a project. Two workflows are recommended, one for manual checking, and one for checking as part of a script or other automation.

Manual Checking

To manually check if the template used to generate your project has been updated, simply run copier check-update in your project directory. Sample output is provided for different scenarios:

# No update available
$ copier check-update
Project is up-to-date!

# Update available
$ copier check-update
New template version available.
Current version is 1.0.0, latest version is 2.0.0.

# Prerelease update available
$ copier check-update --prereleases
New template version available.
Current version is 1.0.0, latest version is 2.0.0a0.

Automated Checking

To facilitate automated checking for updates, copier check-update provides two options:

1. JSON output via the flag --output-format json
2. Exit code output via the flag --quiet

Sample output is provided for different scenarios:

# No update available
$ copier check-update --output-format json
{"update_available": false, "current_version": "1.0.0", "latest_version": "1.0.0"}
$ copier check-update --quiet
[No output, exits 0]

# Update available
$ copier check-update --output-format json
{"update_available": true, "current_version": "1.0.0", "latest_version": "2.0.0"}
$ copier check-update --quiet
[No output, exits 2]

# Prerelease update available
$ copier check-update --output-format json --prereleases
New template version available.
{"update_available": true, "current_version": "1.0.0", "latest_version": "2.0.0a0"}
$ copier check-update --quiet --prereleases
[No output, exits 2]