Skip to content

refactor!: replace jinja2-ansible-filters by jinja2-copier-extension #2039

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions copier/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -627,10 +627,7 @@ def jinja_env(self) -> YieldEnvironment:
"""
paths = [str(self.template.local_abspath)]
loader = FileSystemLoader(paths)
default_extensions = [
"jinja2_ansible_filters.AnsibleCoreFiltersExtension",
YieldExtension,
]
default_extensions = ["jinja2_copier_extension.CopierExtension", YieldExtension]
extensions = default_extensions + list(self.template.jinja_extensions)
try:
env = YieldEnvironment(
Expand Down
28 changes: 2 additions & 26 deletions copier/tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
import re
import stat
import sys
from contextlib import suppress
from decimal import Decimal
from enum import Enum
from importlib.metadata import version
Expand All @@ -17,6 +16,7 @@
from typing import Any, Callable, Iterator, Literal, TextIO, cast

import colorama
from jinja2_copier_extension._filters.types import do_bool
from packaging.version import Version
from pathspec.patterns.gitwildmatch import GitWildMatchPattern
from pydantic import StrictBool
Expand Down Expand Up @@ -110,31 +110,7 @@ def cast_to_str(value: Any) -> str:
raise ValueError(f"Could not convert {value} to string")


def cast_to_bool(value: Any) -> bool:
"""Parse anything to bool.

Params:
value:
Anything to be casted to a bool. Tries to be as smart as possible.

1. Cast to number. Then: 0 = False; anything else = True.
1. Find [YAML booleans](https://yaml.org/type/bool.html),
[YAML nulls](https://yaml.org/type/null.html) or `none` in it
and use it appropriately.
1. Cast to boolean using standard python `bool(value)`.
"""
# Assume it's a number
with suppress(TypeError, ValueError):
return bool(float(value))
# Assume it's a string
with suppress(AttributeError):
lower = value.lower()
if lower in {"y", "yes", "t", "true", "on"}:
return True
elif lower in {"n", "no", "f", "false", "off", "~", "null", "none"}:
return False
# Assume nothing
return bool(value)
cast_to_bool = do_bool


def force_str_end(original_str: str, end: str = "\n") -> str:
Expand Down
2 changes: 1 addition & 1 deletion docs/configuring.md
Original file line number Diff line number Diff line change
Expand Up @@ -1071,7 +1071,7 @@ filters, global variables and functions, or tags to the environment.

The following extensions are _always_ loaded:

- [`jinja2_ansible_filters.AnsibleCoreFiltersExtension`](https://gitlab.com/dreamer-labs/libraries/jinja2-ansible-filters/):
- [`jinja2_copier_extension.CopierExtension`](https://github.com/copier-org/jinja2-copier-extension):
this extension adds most of the
[Ansible filters](https://docs.ansible.com/ansible/2.3/playbooks_filters.html) to
the environment.
Expand Down
9 changes: 9 additions & 0 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,15 @@
name = "copier-${version}";
projectDir = ./.;

# HACK: https://github.com/nix-community/poetry2nix/issues/568#issuecomment-2563254176
overrides =
pkgs.poetry2nix.defaultPoetryOverrides.extend
(final: prev: {
jinja2-copier-extension = prev.jinja2-copier-extension.overridePythonAttrs (old: {
buildInputs = (old.buildInputs or []) ++ [pkgs.python311Packages.hatchling];
});
});

# Trick poetry-dynamic-versioning into using our version
POETRY_DYNAMIC_VERSIONING_BYPASS = version;

Expand Down
19 changes: 9 additions & 10 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ colorama = ">=0.4.6"
dunamai = ">=1.7.0"
funcy = ">=1.17"
jinja2 = ">=3.1.5"
jinja2-ansible-filters = ">=1.3.1"
jinja2-copier-extension = "0.1.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO we should put lower bounds only.

This way we leave downstream packagers to do their work in a more easy way. On our side, we still supply the nix packages for whoever wants officially supported pinnings. Or they can reuse our poetry.lock in any other way.

Copy link
Member Author

@sisp sisp Mar 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, but we don't test the behavior of Jinja filters from jinja2-copier-extension in Copier, so if we change a filter in an incompatible way without pinning in our manifest, the identical Copier version will behave differently when installed before and after that release of jinja2-copier-extension.

Concretely, I'm planning to revise some API inconsistencies, that I retained from jinja2-ansible-filters for backwards compatibility, in jinja2-copier-extension in a future release. For example:

 def do_regex_findall(
     string: str,
-    # TODO(sisp): Rename argument to `pattern` for consistency with other filters.
-    regex: str,
+    pattern: str,
-    # TODO(sisp): Require flags to be keyword-only arguments.
-    multiline: bool = False,  # noqa: FBT001 FBT002
-    ignorecase: bool = False,  # noqa: FBT001 FBT002
+    *,
+    multiline: bool = False,
+    ignorecase: bool = False,
 ) -> list[str] | list[tuple[str, ...]]:

Renaming regex to pattern may be a breaking change in case somebody passes the pattern as a keyword argument. And making multiline and ignorecase keyword-only arguments may be breaking in case somebody passes values as positional arguments. If we release these changes as v0.2.0 and use a lower bound in our pyproject.toml

jinja2-copier-extension = ">=0.1.0"

then next time somebody re-installs Copier (e.g., even just pipx run copier with an expired package cache suffices) a previously working Copier template may suddenly fail to work.

Isn't the difference in this case that the Jinja filters registered by jinja2-copier-extensions.CopierExtension are part of Copier's public (Jinja) API, whereas the public API of other dependencies is used merely internally in Copier?

packaging = ">=23.0"
pathspec = ">=0.9.0"
plumbum = ">=1.6.9"
Expand Down
6 changes: 6 additions & 0 deletions renovate.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,12 @@
{
"matchManagers": ["github-actions"],
"addLabels": ["github_actions"]
},
{
"matchCategories": ["python"],
"matchDepNames": ["jinja2-copier-extension"],
"rangeStrategy": "replace",
"semanticCommitType": "fix"
}
]
}
Loading