9.15.2 appears to have breaking change

[oc-ben-ellis]

Describe the problem

Copier 9.15.2 renders a non-root {{ _copier_conf.answers_file }}.jinja template to the destination root as .copier-answers.yml.

Copier 9.15.1 rendered the same answers-file template into the same nested destination directory where the template file lived. This changes the generated output path for existing templates that include .copier-answers.yml.jinja below the template root.

Template

This minimal template can be created locally with:

rm -rf /tmp/copier-answers-repro
mkdir -p /tmp/copier-answers-repro/template/nested/{{ project_name }}
cd /tmp/copier-answers-repro

cat > template/copier.yml <<'YAML'
project_name:
  type: str
  default: example-project
YAML

cat > 'template/nested/{{ project_name }}/.copier-answers.yml.jinja' <<'JINJA'
_commit: ''
_src_path: template
project_name: {{ project_name }}
JINJA

cat > 'template/nested/{{ project_name }}/README.md.jinja' <<'JINJA'
# {{ project_name }}
JINJA

To Reproduce

1. Create the template above.
2. Run Copier 9.15.1:

rm -rf /tmp/copier-answers-repro/output-9151
mkdir -p /tmp/copier-answers-repro/output-9151
cd /tmp/copier-answers-repro/output-9151
git init -q

uvx --from "copier==9.15.1" copier copy --trust --defaults --force \
  --data project_name=example-project \
  ../template .

3. Check where the answers file was created:

test -f nested/example-project/.copier-answers.yml && echo "nested answers exists"
test ! -f .copier-answers.yml && echo "root answers absent"

Observed with 9.15.1:

nested answers exists
root answers absent

4. Run Copier 9.15.2:

rm -rf /tmp/copier-answers-repro/output-9152
mkdir -p /tmp/copier-answers-repro/output-9152
cd /tmp/copier-answers-repro/output-9152
git init -q

uvx --from "copier==9.15.2" copier copy --trust --defaults --force \
  --data project_name=example-project \
  ../template .

5. Check where the answers file was created:

test ! -f nested/example-project/.copier-answers.yml && echo "nested answers missing"
test -f .copier-answers.yml && echo "root answers exists"

Observed with 9.15.2:

nested answers missing
root answers exists

Logs

Expected behavior

The answers file should be rendered consistently with the previous Copier behavior, or the new behavior should require an explicit opt-in.

For the template above, I expected:

nested/example-project/.copier-answers.yml

I did not expect:

.copier-answers.yml

Screenshots/screencasts/logs

No response

Operating system

Linux

Operating system distribution and version

linux 6.12.91-1-MANJARO

Copier version

copier 9.15.2

Python version

Python 3.14.5

Installation method

uvx+pypi

Additional context

This appears related to the 9.15.2 answers-file path changes. The same template behaves differently across Copier versions:

• Copier 9.15.1 renders the nested .copier-answers.yml.jinja file into the nested destination directory.
• Copier 9.15.2 renders .copier-answers.yml at the destination root instead.

The exact behavioral change
Removed _adjust_rendered_part(), which used to turn a full rendered answers path into just the filename so the file could land in the template’s subdirectory:

[Commit d76195a — deletion of _adjust_rendered_part](d76195a#diff-8b8e8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8f8The breaking change is in copier-org/copier#2186, specifically commit d76195a in copier/_main.py.

The exact behavioral change
Removed _adjust_rendered_part(), which used to strip a full rendered answers path down to just the filename so nested templates could place the file under a subdirectory:

commit d76195a — deletion of _adjust_rendered_part

The diff hunk is around lines 1040–1057 in that commit view.

Replaced with an early yield of self.answers_relpath (default .copier-answers.yml at repo root) whenever the rendered path matches the answers file:

if str(self.answers_relpath) == rendered_part:
yield self.answers_relpath, new_context
continue
and:

if str(self.answers_relpath) == rendered_part:
yield (self.answers_relpath, extra_context)
return

[sisp]

As mentioned in #2186, Copier's documentation states that the answers file template must be in the template's root directory:

The file must be called exactly {{ _copier_conf.answers_file }}.jinja [...] in your template's root folder [...].

https://copier.readthedocs.io/en/stable/configuring/#the-copier-answersyml-file

I've traced this information back to at least October 2020. Placing the answers file template elsewhere was broken and not intended, hence I don't consider #2186 a breaking change.

[oc-ben-ellis]

It still supports the -f flag as well which is what we've done to work around the issue.

Happy to close if you're happy. I've had to refactor out code base to address this but all sorted.

[sisp]

Sorry for the inconvenience. I'm glad you sorted it out.

What do you mean regarding the -f flag?

[oc-ben-ellis]

Sorry I mean-a, --answers-file VALUE:str

[oc-ben-ellis]

or _answers_file in the config

[sisp]

The flag and setting remain available as before, only the answers file template must be in the repo's root directory to correctly support configuring a path relative to the subproject root where the answers file will be rendered. This was broken prior to v9.15.2.

[pSpitzner]

Hi all,

thanks for all the hard work around the (sticky) issue around the answers file.

I have been following this and the previous issues and PRs regarding the answers file, to wrap my head around how to use it correctly (while aiming for a clean and minimal template repo). So far, I could not find a detailed write-up.

As I understand it, the currently intended behvaiour that was fixed in 9.15.2 is:

• we differntiate between the template file {{ _copier_conf.answers_file }}.jinja, and the answers file. The latter gets placed during copy and re-used during update and reflects the filled-in template.
• the template file is optional, but has to sit in the root directory.
• if no template file is found, no answers file will be produced.
• in the copier.yml file (or via -a flag) one can specify a custom path to specify where the answers will be rendered at (if a template file is present). This does not affect — or depend on — the location of the template file.
• in order to enable convenient updates, an answers file is mandatory. This in turn requires the template file in the repo root.
• There is no way to place the template file in a subdirectory.

Is this summary complete and correct?

If so, I think it poses the following inconveniences and inconsistencies:

• There is no way to keep the root repo clean. In most cases, it needs to contain the copier.yml (fine) but also the template file.
• template and answers file locations become inconsistent once a template-creator uses _answers_file setting. I think this is what caused a lot of the confusion. For all other tempalte files, their location matches rendered locations 1:1.
• Given the content of the template file is in many cases the same (docs: "The file must have this content: ..."), and we cannot change its location, why do we need a file for it to begin with? Wouldnt the simple _answers_file setting in copier.yml suffice to achieve the same effect?

Any comments and further insights would be very welcome!

thanks again,
p

[sisp]

Thanks for the warm introductory words, @pSpitzner. 🙏

As I understand it, the currently intended behvaiour that was fixed in 9.15.2 is:

[...]

Is this summary complete and correct?

Yes, that's correct.

If so, I think it poses the following inconveniences and inconsistencies:

• There is no way to keep the root repo clean. [...]

That's correct if you don't use the _subdirectory setting in copier.yml, which I personally recommend for most if not all templates in order to separate the template files from repo boilerplate and tests. For example, I use pytest to implement a (rather comprehensive) test suite for my templates, similar to how Copier's own test suite works but aimed at asserting correctly rendered files especially with feature questions like "Enable Renovate integration?" or "Enable devcontainer integration?".

• template and answers file locations become inconsistent once a template-creator uses _answers_file setting. I think this is what caused a lot of the confusion. For all other tempalte files, their location matches rendered locations 1:1.

That's correct. While the template file location is static (must be located at the _subdirectory path, which is . by default, i.e. the repo's root directory), a template creator can choose a different file name and/or location via the _answers_file setting. This alternative location must be a relative path anchored at the _subdirectory path. The crux about this decision is: If we allowed the template file location to be an arbitrary subpath under the _subdirectory path, then

• the _answers_file setting could be a relative path anchored at this file's location, which would work, but
• the --answers-file flag overrides the _answers_file setting with the same semantics, so template users would need to know the template file location to override the path as intended.

Alternatively, the _answers_file setting could be a relative path anchored at the _subdirectory path, even when the template file is located in an arbitrary subpath, but I don't think this behavior is intuitive, thus it would need to be documented and understood by Copier users, too. Anchoring only the user-facing --answers-file flag at the _subdirectory path while anchoring the template-creator-facing _answers_file path at the template file's location would be even more confusing IMHO. Last but not least, I think #983 is a good idea for most templates that support updating, it just hasn't made it to the top of my priorities list.

• Given the content of the template file is in many cases the same (docs: "The file must have this content: ..."), and we cannot change its location, why do we need a file for it to begin with? Wouldnt the simple _answers_file setting in copier.yml suffice to achieve the same effect?

You're quite right, but there are a few edge cases like locking a computed value which is, e.g., useful for locking the first copyright year at the time of the initial copier copy ... run. I think the proposal of #983 can be extended by another setting like _answers_file_template which defaults to {{ _copier_answers|to_nice_yaml -}} and can be overridden if needed – which is likely rather rare. When we implement #983, the default (and IMO intuitive) template file location will be the _subdirectory path, so migrating from the template file to the _updatable: true setting will be seamless.

I'm curious about your opinion.

[pSpitzner]

Thanks for the very detailed explanation, @sisp!
This already clarifies a lot.

I wasnt aware of #983, but thats exactly what I had in mind. Extending it with _answers_file_template in the copier.yml seems sensible and consistent, to me, too. 👍
(Coming up with a solid upgrade path from the previous convention might be a bit tricky, but making the new copier.yml convention take precedence, while printing a warning when a legacy template file is found, is probably a good start.)

I have to admit, I had not given the subdirectory much thought so far. TLDR is that I felt like I needed the inverse of what it offers. That is, _subdirectory allows us to put everything that should be deployed into a folder, while I had planned to put everything that should not be deployed, meta data etc. into a folder. But maybe I misunderstood how _subdirectory works.

The main way I am using copier is like an app store, where a repo contains a dbt project. The aim is usually to deploy the repo as is (therefore the _subdirectory is not desriable, as the dbt code is the focus, and everything installation-related is only one part of it). The main task during deployment that goes beyond git cloning (or smart updating!) is to ask questions which then end up in a .env, and to selectively copy a few extra files, depending on the given answers.

Hope this clarifies a bit where I am coming from 😃