Skip to content

Commit 02cf1ef

Browse files
authored
CI: auto-derive addon pip deps from requires_mod in .gpr.py (#14)
The Dockerfile bakes in only `dbf`, but addons declare a wider set of Python deps in their .gpr.py `requires_mod` lists (networkx, psycopg2, pygraphviz, lxml, svgwrite, boto3, litellm, life_line_chart, psycopg). Without these installed, per-addon unit tests and the plugin- registration subprocess load fail with ImportError/NameError. Add a pre-test step to unit-test-linux, unit-test-windows, and integration-test that globs every *.gpr.py, extracts the requires_mod union via ast.literal_eval, and pip-installs each package one at a time. Per-package install (not batched) keeps a single build failure (pygraphviz without graphviz-dev, psycopg2 without libpq-dev) from aborting the rest — the affected addon's tests will skip or fail in isolation without blocking others. Mirrors Gramps' Addon Manager install path (gramps/gui/plug/_windows.py __on_install_clicked → req.install → gen/utils/requirements.py), keeping .gpr.py files as the single source of truth for addon deps. New addon deps do not need a parallel update to the Dockerfile or this workflow.
1 parent 7abb603 commit 02cf1ef

1 file changed

Lines changed: 100 additions & 0 deletions

File tree

.github/workflows/ci.yml

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,44 @@ jobs:
100100
steps:
101101
- uses: actions/checkout@v4
102102

103+
- name: Install addon runtime deps (derived from requires_mod)
104+
# Auto-derive the union of requires_mod across every .gpr.py in
105+
# the repo. Mirrors Gramps' Addon Manager install path
106+
# (gramps/gui/plug/_windows.py __on_install_clicked → req.install →
107+
# gen/utils/requirements.py). Keeps .gpr.py files as the single
108+
# source of truth for addon deps — no parallel list to maintain
109+
# in the image or workflow. Best-effort: a package needing exotic
110+
# system deps (pygraphviz → graphviz-dev, psycopg2 → libpq-dev)
111+
# may fail here; the affected addon's tests will skip or fail in
112+
# isolation without blocking the rest.
113+
shell: bash
114+
run: |
115+
addon_mods=$(python3 - <<'PY'
116+
import ast, glob, re
117+
pat = re.compile(r"requires_mod\s*=\s*(\[[^\]]*\])")
118+
mods = set()
119+
for f in glob.glob("*/*.gpr.py"):
120+
try:
121+
text = open(f, encoding="utf-8").read()
122+
except OSError:
123+
continue
124+
for m in pat.finditer(text):
125+
try:
126+
mods.update(ast.literal_eval(m.group(1)))
127+
except (ValueError, SyntaxError):
128+
pass
129+
print(" ".join(sorted(mods)))
130+
PY
131+
)
132+
if [ -n "$addon_mods" ]; then
133+
echo "→ addon deps: $addon_mods"
134+
for mod in $addon_mods; do
135+
pip install "$mod" || echo "× $mod failed to install (continuing)"
136+
done
137+
else
138+
echo "no requires_mod declarations found"
139+
fi
140+
103141
- name: Run per-addon unit tests
104142
# Filename convention (all OSes):
105143
# test_*.py — general (any OS)
@@ -166,6 +204,36 @@ jobs:
166204
mamba list | head -30
167205
python -c "import gramps, gi; print('deps OK')"
168206
207+
- name: Install addon runtime deps (derived from requires_mod)
208+
# See unit-test-linux for rationale. Uses `python` (conda-forge
209+
# env) to match the surrounding Windows job style.
210+
run: |
211+
addon_mods=$(python - <<'PY'
212+
import ast, glob, re
213+
pat = re.compile(r"requires_mod\s*=\s*(\[[^\]]*\])")
214+
mods = set()
215+
for f in glob.glob("*/*.gpr.py"):
216+
try:
217+
text = open(f, encoding="utf-8").read()
218+
except OSError:
219+
continue
220+
for m in pat.finditer(text):
221+
try:
222+
mods.update(ast.literal_eval(m.group(1)))
223+
except (ValueError, SyntaxError):
224+
pass
225+
print(" ".join(sorted(mods)))
226+
PY
227+
)
228+
if [ -n "$addon_mods" ]; then
229+
echo "→ addon deps: $addon_mods"
230+
for mod in $addon_mods; do
231+
pip install "$mod" || echo "× $mod failed to install (continuing)"
232+
done
233+
else
234+
echo "no requires_mod declarations found"
235+
fi
236+
169237
- name: Run per-addon unit tests
170238
# See filename-convention note in unit-test-linux. The Windows
171239
# job runs test_*.py except test_linux_* and test_integration_*.
@@ -206,6 +274,38 @@ jobs:
206274
steps:
207275
- uses: actions/checkout@v4
208276

277+
- name: Install addon runtime deps (derived from requires_mod)
278+
# See unit-test-linux for rationale. The plugin registration test
279+
# subprocess-loads each addon's module, which imports its
280+
# requires_mod packages.
281+
shell: bash
282+
run: |
283+
addon_mods=$(python3 - <<'PY'
284+
import ast, glob, re
285+
pat = re.compile(r"requires_mod\s*=\s*(\[[^\]]*\])")
286+
mods = set()
287+
for f in glob.glob("*/*.gpr.py"):
288+
try:
289+
text = open(f, encoding="utf-8").read()
290+
except OSError:
291+
continue
292+
for m in pat.finditer(text):
293+
try:
294+
mods.update(ast.literal_eval(m.group(1)))
295+
except (ValueError, SyntaxError):
296+
pass
297+
print(" ".join(sorted(mods)))
298+
PY
299+
)
300+
if [ -n "$addon_mods" ]; then
301+
echo "→ addon deps: $addon_mods"
302+
for mod in $addon_mods; do
303+
pip install "$mod" || echo "× $mod failed to install (continuing)"
304+
done
305+
else
306+
echo "no requires_mod declarations found"
307+
fi
308+
209309
- name: Run plugin registration tests
210310
# shell: bash for consistency with the surrounding steps; the
211311
# current command uses no bashisms, but keeps this block safe

0 commit comments

Comments
 (0)