Add ability to specify constraints for WheelhouseTactic by himax16 · Pull Request #693 · juju/charm-tools

himax16 · 2026-02-27T12:07:17Z

Add ability to specify wheel build constraints using wheelhouse-constraints.txt file.

This is done through modifying WheelhouseTactic class.

Solves #690

Checklist

Are all your commits [logically] grouped and squashed appropriately?
Does this patch have code coverage?
Does your code pass make test?

freyes

when trying to build charm-octavia from master, I get this error:

:: charmtools.build.tactics: Processing constraints:
:: utils: b"ERROR: Invalid requirement: '/tmp/tmpw34ntq_4/wheelhouse-constraints.txt'\nHint: It looks like a path. The path does exist. The argument you provided (/tmp/tmpw34ntq_4/wheelhouse-constraints.txt) appears to be a requirements file. If that is the case, use the '-r' flag to install the packages specified within it.\n"
:: utils: process: bash -c . /tmp/tmp4cz62vuq/bin/activate ; pip3 wheel --no-binary :all: -w /tmp/tmps7uuv4bk -r /tmp/tmpw34ntq_4/wheelhouse.txt /tmp/tmpw34ntq_4/wheelhouse-constraints.txt (1)
:: charmtools.build.tactics: Build failed. If you are building on Focal and have Jinja2 or MarkupSafe as part of your dependencies, try passing the `--upgrade-buildvenv-core-deps` argument.
:: build: Traceback (most recent call last):
::   File "/snap/charm/x1/usr/local/lib/python3.10/dist-packages/charmtools/build/builder.py", line 1251, in main
::     build()
::   File "/snap/charm/x1/usr/local/lib/python3.10/dist-packages/charmtools/build/builder.py", line 854, in __call__
::     self.generate()
::   File "/snap/charm/x1/usr/local/lib/python3.10/dist-packages/charmtools/build/builder.py", line 793, in generate
::     self.exec_plan(self.plan, self.layers)
::   File "/snap/charm/x1/usr/local/lib/python3.10/dist-packages/charmtools/build/builder.py", line 651, in exec_plan
::     tactic()
::   File "/snap/charm/x1/usr/local/lib/python3.10/dist-packages/charmtools/build/tactics.py", line 1380, in __call__
::     self._process_combined(wheelhouse)
::   File "/snap/charm/x1/usr/local/lib/python3.10/dist-packages/charmtools/build/tactics.py", line 1413, in _process_combined
::     self._add(
::   File "/snap/charm/x1/usr/local/lib/python3.10/dist-packages/charmtools/build/tactics.py", line 1226, in _add
::     self._pip('wheel',
::   File "/snap/charm/x1/usr/local/lib/python3.10/dist-packages/charmtools/build/tactics.py", line 1350, in _pip
::     return self._run_in_venv('pip3', *args, env=env)
::   File "/snap/charm/x1/usr/local/lib/python3.10/dist-packages/charmtools/build/tactics.py", line 1346, in _run_in_venv
::     raise BuildError(res.output)
:: charmtools.build.errors.BuildError: ERROR: Invalid requirement: '/tmp/tmpw34ntq_4/wheelhouse-constraints.txt'
:: Hint: It looks like a path. The path does exist. The argument you provided (/tmp/tmpw34ntq_4/wheelhouse-constraints.txt) appears to be a requirements file. If that is the case, use the '-r' flag to install the packages specified within it.
::
:: build: ERROR: Invalid requirement: '/tmp/tmpw34ntq_4/wheelhouse-constraints.txt'
:: Hint: It looks like a path. The path does exist. The argument you provided (/tmp/tmpw34ntq_4/wheelhouse-constraints.txt) appears to be a requirements file. If that is the case, use the '-r' flag to install the packages specified within it.
:: charm tool execution command=['charm', 'proof']
:: charm tool execution SUCCESS: returncode=0
:: charm tool execution command=['charm', 'build', '--verbose', '--binary-wheels-from-source', '-o', '/root/charm-octavia/parts/charm/build']
Failed to run the build script for part 'charm'.
Detailed information: 
:: + /snap/charmcraft/7286/bin/python3 -I /snap/charmcraft/7286/lib/python3.10/site-packages/charmcraft/parts/plugins/_reactive.py octavia /root/charm-octavia/parts/charm/build /root/charm-octavia/parts/charm/install --binary-wheels-from-source
:: /snap/charm/x1/usr/local/lib/python3.10/dist-packages/requirements/requirement.py:24: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
::   from pkg_resources import Requirement as Req
:: WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
Recommended resolution: Check the build output and verify the project can work with the 'reactive' plugin.
For more information, check out: https://canonical-charmcraft.readthedocs-hosted.com/en/3.5.3/reference/plugins/
Full execution log: '/root/.local/state/charmcraft/log/charmcraft-20260305-144709.015315.log'
root@charm-tools-noble:~/charm-octavia# cat /tmp/tmpw34ntq_4/wheelhouse-constraints.txt
root@charm-tools-noble:~/charm-octavia# wc -l /tmp/tmpw34ntq_4/wheelhouse-constraints.txt
0 /tmp/tmpw34ntq_4/wheelhouse-constraints.txt

freyes · 2026-03-05T14:56:07Z

When adding a wheelhouse-constraints.txt file to charm-octavia, I get the same error:

:: charmtools.build.tactics: Processing constraints:
:: utils: b"ERROR: Invalid requirement: '/tmp/tmpszyh8z0d/wheelhouse-constraints.txt'\nHint: It looks like a path. The path does exist. The argument you provided (/tmp/tmpszyh8z0d/wheelhouse-constraints.txt) appears to be a requirements file. If that is the case, use the '-r' flag to install the packages specified within it.\n"
:: utils: process: bash -c . /tmp/tmp84l1ed2q/bin/activate ; pip3 wheel --no-binary :all: -w /tmp/tmpq9ffqwio -r /tmp/tmpszyh8z0d/wheelhouse.txt /tmp/tmpszyh8z0d/wheelhouse-constraints.txt (1)
[...]
::   from pkg_resources import Requirement as Req
:: WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
Recommended resolution: Check the build output and verify the project can work with the 'reactive' plugin.
For more information, check out: https://canonical-charmcraft.readthedocs-hosted.com/en/3.5.3/reference/plugins/
Full execution log: '/root/.local/state/charmcraft/log/charmcraft-20260305-145138.593290.log'
root@charm-tools-noble:~/charm-octavia# cat src/wheelhouse-constraints.txt 
maturin<1.11.0
root@charm-tools-noble:~/charm-octavia# wc -l /tmp/tmpszyh8z0d/wheelhouse-constraints.txt
0 /tmp/tmpszyh8z0d/wheelhouse-constraints.txt

freyes · 2026-03-05T14:58:59Z

Test case:

lxc launch ubuntu-daily:noble charm-tools-noble-dev
lxc shell charm-tools-noble-dev
# inside the container's shell:
# install charm-tools snap, it can be a locally built or the one built by this CI process
snap install --dangerous --classic ./charm_0.0.0_amd64.snap
snap install --classic --channel 3.x charmcraft
git clone https://opendev.org/openstack/charm-octavia
cd charm-octavia
charmcraft pack -v --destructive-mode

freyes

I tested the last revision and it's not picking up the constraints:

: charmtools.build.tactics:
:: charmtools.build.tactics: Processing constraints:
:: utils: b'Ignoring pip: markers \'python_version < "3.8"\' don\'t match your environment\nIgnoring Jinja2: m

root@charm-tools-noble:~/charm-octavia# cat src/wheelhouse-constraints.txt 
maturin<1.11.0
root@charm-tools-noble:~/charm-octavia#

shouldn't this have printed "maturing<1.11.0" right after "Processing constraints"?

himax16 · 2026-03-06T13:55:19Z

Fixed some indentation error in charmtools/build/tactics.py, should be working now.

425de15#diff-1bfa2c0d1ca74177308d77537041b4913d528a572ca16d399e41a084aba14a86R1206-R1207

freyes

The patch works as intended,

:: charmtools.build.tactics: Processing constraints:
:: charmtools.build.tactics:   # octavia
:: charmtools.build.tactics:   maturin<1.11.0
:: charmtools.build.tactics:
:: charmtools.build.tactics:   # Default constraints
:: charmtools.build.tactics:   setuptools<82
:: charmtools.build.tactics:

I was able to build charm-octavia using this:

diff --git a/charmcraft.yaml b/charmcraft.yaml
index b8bb4fe..292c569 100644
--- a/charmcraft.yaml
+++ b/charmcraft.yaml
@@ -13,8 +13,9 @@ parts:
       - libssl-dev
       - libxml2-dev
       - libxslt1-dev
-      - rustc
-      - cargo
+      - rustc-1.77
+      - cargo-1.77
+      - build-essential
       - pkg-config
     source: src/
     build-snaps:
@@ -22,6 +23,13 @@ parts:
     build-environment:
       - CHARM_INTERFACES_DIR: $CRAFT_PROJECT_DIR/interfaces/
       - CHARM_LAYERS_DIR: $CRAFT_PROJECT_DIR/layers/
+    override-build: |
+      rm -f /usr/bin/rustc
+      ln -s /usr/bin/rustc-1.77 /usr/bin/rustc
+      rm -f /usr/bin/cargo
+      ln -s /usr/bin/cargo-1.77 /usr/bin/cargo
+      craftctl default
+
 base: ubuntu@24.04
 platforms:
   amd64:
diff --git a/src/wheelhouse-constraints.txt b/src/wheelhouse-constraints.txt
new file mode 100644
index 0000000..99c22e7
--- /dev/null
+++ b/src/wheelhouse-constraints.txt
@@ -0,0 +1 @@
+maturin<1.11.0

Although this is breaking one of our downstream consumers ( https://github.com/charmed-kubernetes/charm-openstack-integrator/ )

 main ✗= ~/Projects/charms/k8s/charm-openstack-integrator $ tox -e validate-wheelhouse --recreate
validate-wheelhouse: remove tox env folder /home/freyes/Projects/charms/k8s/charm-openstack-integrator/.tox/validate-wheelhouse
validate-wheelhouse: install_deps> python -I -m pip install 'path<17' git+https://github.com/himax16/charm-tools.git@master
validate-wheelhouse: commands[0]> tests/validate-wheelhouse.sh
++ mktemp -d --tmpdir=/home/freyes/Projects/charms/k8s/charm-openstack-integrator/.tox/validate-wheelhouse/tmp
+ build_dir=/home/freyes/Projects/charms/k8s/charm-openstack-integrator/.tox/validate-wheelhouse/tmp/tmp.3pb4793W6o
++ egrep '^name\S*:' ./metadata.yaml
++ awk '{ print $2 }'
+ charm=openstack-integrator
+ trap cleanup EXIT
+ charm-build src --build-dir /home/freyes/Projects/charms/k8s/charm-openstack-integrator/.tox/validate-wheelhouse/tmp/tmp.3pb4793W6o --debug
Traceback (most recent call last):
  File "/home/freyes/Projects/charms/k8s/charm-openstack-integrator/.tox/validate-wheelhouse/bin/charm-build", line 5, in <module>
    from charmtools.build.builder import main
  File "/home/freyes/Projects/charms/k8s/charm-openstack-integrator/.tox/validate-wheelhouse/lib/python3.12/site-packages/charmtools/build/__init__.py", line 6, in <module>
    import charmtools.build.tactics
  File "/home/freyes/Projects/charms/k8s/charm-openstack-integrator/.tox/validate-wheelhouse/lib/python3.12/site-packages/charmtools/build/tactics.py", line 13, in <module>
    import requirements
  File "/home/freyes/Projects/charms/k8s/charm-openstack-integrator/.tox/validate-wheelhouse/lib/python3.12/site-packages/requirements/__init__.py", line 19, in <module>
    from .parser import parse
  File "/home/freyes/Projects/charms/k8s/charm-openstack-integrator/.tox/validate-wheelhouse/lib/python3.12/site-packages/requirements/parser.py", line 23, in <module>
    from .requirement import Requirement
  File "/home/freyes/Projects/charms/k8s/charm-openstack-integrator/.tox/validate-wheelhouse/lib/python3.12/site-packages/requirements/requirement.py", line 24, in <module>
    from pkg_resources import Requirement as Req
ModuleNotFoundError: No module named 'pkg_resources'
+ cleanup
+ rm -rf /home/freyes/Projects/charms/k8s/charm-openstack-integrator/.tox/validate-wheelhouse/tmp/tmp.3pb4793W6o
validate-wheelhouse: exit 1 (0.13 seconds) /home/freyes/Projects/charms/k8s/charm-openstack-integrator> tests/validate-wheelhouse.sh pid=1662328
  validate-wheelhouse: FAIL code 1 (8.55=setup[8.43]+cmd[0.13] seconds)
  evaluation failed :( (8.63 seconds)

The proposed PR is missing these commits - himax16/charm-tools@master...juju:charm-tools:master - it needs to be rebased on top of master

himax16 · 2026-03-09T12:36:06Z

Done, still need permission to rerun CI

charmtools/templates/bash/template.py

charmtools/build/tactics.py

freyes · 2026-03-16T13:42:28Z

charmtools/build/tactics.py

-        self._layer_refs = {}
+
+        # Dictionary to store original layer references for each packages
+        self.layer_refs = {}


why are you dropping the _ prefix?

It was from older implementation that accesses the layer_refs from builder side, should I make it "private" again?

yes, please

charmtools/build/tactics.py

Signed-off-by: Himawan Winarto <himawan.winarto@canonical.com>

himax16 force-pushed the master branch from cfb543c to 6fd4aeb Compare March 2, 2026 14:02

himax16 marked this pull request as ready for review March 2, 2026 14:59

himax16 force-pushed the master branch from 4d1fde8 to 6b4f8bb Compare March 2, 2026 15:38

freyes requested changes Mar 5, 2026

View reviewed changes

himax16 force-pushed the master branch 2 times, most recently from 379abc6 to ca27b60 Compare March 5, 2026 15:45

freyes requested changes Mar 5, 2026

View reviewed changes

himax16 force-pushed the master branch from ca27b60 to be034bc Compare March 6, 2026 13:52

freyes requested changes Mar 6, 2026

View reviewed changes

himax16 force-pushed the master branch from be034bc to 250bb5e Compare March 9, 2026 12:34

freyes self-requested a review March 16, 2026 13:32

freyes requested changes Mar 20, 2026

View reviewed changes

himax16 added 4 commits March 23, 2026 12:56

Modify WheelhouseTactic to allow constraints

79f98ce

Signed-off-by: Himawan Winarto <himawan.winarto@canonical.com>

Fix tests to solve pkg_resources issues

8ae4e7f

Signed-off-by: Himawan Winarto <himawan.winarto@canonical.com>

Add tests for wheelhouse constraints

47b80e3

Signed-off-by: Himawan Winarto <himawan.winarto@canonical.com>

Update version retrieval in setup.py to handle JSON format

e1b612e

Signed-off-by: Himawan Winarto <himawan.winarto@canonical.com>

himax16 force-pushed the master branch from 250bb5e to e1b612e Compare March 23, 2026 17:24

himax16 requested a review from freyes March 23, 2026 17:25

Update changelog and reproducible charms documentations

41a5b4b

Signed-off-by: Himawan Winarto <himawan.winarto@canonical.com>

freyes requested changes Mar 24, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ability to specify constraints for WheelhouseTactic#693

Add ability to specify constraints for WheelhouseTactic#693
himax16 wants to merge 5 commits intojuju:masterfrom
himax16:master

himax16 commented Feb 27, 2026 •

edited

Loading

Uh oh!

freyes left a comment

Uh oh!

freyes commented Mar 5, 2026

Uh oh!

freyes commented Mar 5, 2026

Uh oh!

freyes left a comment

Uh oh!

himax16 commented Mar 6, 2026

Uh oh!

freyes left a comment

Uh oh!

himax16 commented Mar 9, 2026

Uh oh!

Uh oh!

Uh oh!

freyes Mar 16, 2026

Uh oh!

himax16 Mar 23, 2026

Uh oh!

freyes Mar 24, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

himax16 commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Uh oh!

freyes left a comment

Choose a reason for hiding this comment

Uh oh!

freyes commented Mar 5, 2026

Uh oh!

freyes commented Mar 5, 2026

Uh oh!

freyes left a comment

Choose a reason for hiding this comment

Uh oh!

himax16 commented Mar 6, 2026

Uh oh!

freyes left a comment

Choose a reason for hiding this comment

Uh oh!

himax16 commented Mar 9, 2026

Uh oh!

Uh oh!

Uh oh!

freyes Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

himax16 Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

freyes Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

himax16 commented Feb 27, 2026 •

edited

Loading