Skip to content

Commit d5c09b7

Browse files
committed
ci: per-job scoped dependency sync + working cache + drop redundant prereqs
- setup-uv-cached: add `group` input; on cache miss run `uv sync --frozen --no-default-groups --group <group>` so each job installs only its slice (~2-15 packages) not all 300. Legacy callers (no group) fall back to --all-groups. Always sets up Python 3.12. - cache-management: add `group` input; key becomes `deps-py3.12-<group>-<uv.lock hash>`. Add a populate step so the cache prime actually installs the group's packages before actions/cache saves (it previously saved an empty .venv on every miss). Each job's own setup-uv-cached syncs-on-miss too, so correctness never depends on the prime winning the race. - Per-job group mapping: lint-ruff/auto-format -> lint; lint-pyright -> typecheck; arch-validation/version-drift -> arch; test legs -> test; build-and-package -> build; bandit/safety -> security; docs -> docs. - Cache key base unified to `deps` across all workflows (was quality-deps, test-deps, docs-deps, ...) so per-group keys are globally deduplicated. - version-drift step no longer does `pip install pyyaml`; it uses the venv populated by setup-uv-cached (pyyaml is a project dependency). - Drop redundant dev-install prereqs from ci-quality-ruff, ci-arch-imports, ci-arch-lint-imports (ci.mk) and format-fix (quality.mk); CI jobs receive a pre-populated venv, local fresh-checkout still works via `make dev-install`.
1 parent 4377b42 commit d5c09b7

14 files changed

Lines changed: 195 additions & 49 deletions

File tree

.github/actions/setup-uv-cached/action.yml

Lines changed: 29 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,19 @@
11
name: 'Setup UV with Cache'
2-
description: 'Install UV and restore cached environment'
2+
description: 'Install UV and restore cached environment, syncing only the requested dependency group'
33
inputs:
44
cache-key:
5-
description: 'Cache key for UV environment'
5+
description: 'Cache key for UV environment (caller-supplied; keyed per group)'
66
required: true
7+
group:
8+
description: >-
9+
PEP 735 dependency-group to sync (e.g. lint, typecheck, arch, test, docs,
10+
security, build, ci, dev). When supplied the action runs
11+
`uv sync --frozen --no-default-groups --group <group>` on a cache miss so
12+
each job installs only the packages it needs. When empty the action falls
13+
back to `uv sync --frozen --all-groups` (preserves legacy callers that
14+
have not yet been updated to pass a group).
15+
required: false
16+
default: ''
717
fail-on-cache-miss:
818
description: 'Whether to fail if cache is not found'
919
required: false
@@ -15,6 +25,11 @@ runs:
1525
- name: Install UV
1626
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
1727

28+
- name: Setup Python 3.12
29+
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
30+
with:
31+
python-version: '3.12'
32+
1833
- name: Restore UV environment
1934
id: restore-cache
2035
uses: actions/cache/restore@a7833574556fa59680c1b7cb190c1735db73ebf0 # v5.0.0
@@ -27,7 +42,17 @@ runs:
2742
key: ${{ inputs.cache-key }}
2843
fail-on-cache-miss: ${{ inputs.fail-on-cache-miss }}
2944

30-
- name: Install dependencies if cache miss
45+
- name: Install dependencies (scoped, on cache miss)
3146
if: steps.restore-cache.outputs.cache-hit != 'true'
3247
shell: bash
33-
run: make dev-install
48+
env:
49+
ORB_SKIP_UI_BUILD: "1"
50+
UV_GROUP: ${{ inputs.group }}
51+
run: |
52+
if [ -n "$UV_GROUP" ]; then
53+
# Scoped install: only the packages this job needs (~2-15 pkgs, fast even cold)
54+
uv sync --frozen --no-default-groups --group "$UV_GROUP"
55+
else
56+
# Legacy fallback: full install (used by callers that predate group scoping)
57+
uv sync --frozen --all-groups
58+
fi

.github/workflows/cache-management.yml

Lines changed: 44 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,15 @@ on:
1919
description: 'Python version for cache key'
2020
required: true
2121
type: string
22+
group:
23+
description: >-
24+
PEP 735 dependency-group to prime (e.g. lint, typecheck, test).
25+
When provided the prime job runs `uv sync --frozen --no-default-groups
26+
--group <group>` so only the relevant slice is cached. When empty the
27+
prime installs --all-groups (legacy / umbrella callers).
28+
required: false
29+
type: string
30+
default: ''
2231
outputs:
2332
cache-key:
2433
description: 'Generated cache key for use in setup-uv-cached'
@@ -36,12 +45,27 @@ jobs:
3645
- name: Checkout code
3746
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
3847

48+
- name: Install UV
49+
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
50+
51+
- name: Setup Python 3.12
52+
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
53+
with:
54+
python-version: '3.12'
55+
3956
- name: Generate cache key
4057
id: cache-key
4158
run: |
4259
case "${{ inputs.cache-type }}" in
4360
dependencies)
44-
KEY="${{ inputs.cache-key-base }}-py${{ inputs.python-version }}-${{ hashFiles('.project.yml', 'pyproject.toml', 'uv.lock') }}"
61+
# Key encodes the group so per-group caches are independent;
62+
# a uv.lock bump invalidates only that group's slice.
63+
GROUP="${{ inputs.group }}"
64+
if [ -n "$GROUP" ]; then
65+
KEY="${{ inputs.cache-key-base }}-py3.12-${GROUP}-${{ hashFiles('.project.yml', 'pyproject.toml', 'uv.lock') }}"
66+
else
67+
KEY="${{ inputs.cache-key-base }}-py3.12-${{ hashFiles('.project.yml', 'pyproject.toml', 'uv.lock') }}"
68+
fi
4569
;;
4670
*)
4771
KEY="${{ inputs.cache-key-base }}-${{ github.sha }}"
@@ -50,7 +74,7 @@ jobs:
5074
echo "key=$KEY" >> "$GITHUB_OUTPUT"
5175
echo "Generated cache key: $KEY"
5276
53-
- name: Restore cache
77+
- name: Restore cache (check for existing warm cache)
5478
id: cache
5579
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v5
5680
with:
@@ -61,5 +85,22 @@ jobs:
6185
.pytest_cache
6286
key: ${{ steps.cache-key.outputs.key }}
6387
restore-keys: |
64-
${{ inputs.cache-key-base }}-py${{ inputs.python-version }}-
88+
${{ inputs.cache-key-base }}-py3.12-${{ inputs.group != '' && format('{0}-', inputs.group) || '' }}
6589
${{ inputs.cache-key-base }}-
90+
91+
- name: Prime cache (populate on miss)
92+
# Only runs when there is no existing warm cache for this key.
93+
# Installs the scoped group so the post-step cache save writes a populated
94+
# .venv rather than an empty one. Each job's setup-uv-cached composite
95+
# also syncs-on-miss, so correctness never depends on the prime winning
96+
# the race; this is best-effort warming.
97+
if: steps.cache.outputs.cache-hit != 'true'
98+
env:
99+
ORB_SKIP_UI_BUILD: "1"
100+
UV_GROUP: ${{ inputs.group }}
101+
run: |
102+
if [ -n "$UV_GROUP" ]; then
103+
uv sync --frozen --no-default-groups --group "$UV_GROUP"
104+
else
105+
uv sync --frozen --all-groups
106+
fi

.github/workflows/changelog-validation.yml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,9 @@ jobs:
3232
uses: ./.github/workflows/cache-management.yml
3333
with:
3434
cache-type: dependencies
35-
cache-key-base: changelog-validation
35+
cache-key-base: deps
3636
python-version: ${{ needs.config.outputs.default-python-version }}
37+
group: build
3738

3839
validate-changelog:
3940
name: Validate Changelog Format
@@ -53,6 +54,7 @@ jobs:
5354
uses: ./.github/actions/setup-uv-cached
5455
with:
5556
cache-key: ${{ needs.setup-cache.outputs.cache-key }}
57+
group: build
5658
fail-on-cache-miss: false
5759

5860
- name: Install changelog dependencies

.github/workflows/ci-quality.yml

Lines changed: 54 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -45,14 +45,37 @@ jobs:
4545
name: Configuration
4646
uses: ./.github/workflows/shared-config.yml
4747

48-
setup-cache:
49-
name: Setup Cache
48+
# One setup-cache job per group that needs priming.
49+
# Each installs only its slice so cache misses are cheap even cold.
50+
setup-cache-lint:
51+
name: Setup Cache (lint)
5052
needs: config
5153
uses: ./.github/workflows/cache-management.yml
5254
with:
5355
cache-type: dependencies
54-
cache-key-base: quality-deps
56+
cache-key-base: deps
5557
python-version: ${{ needs.config.outputs.default-python-version }}
58+
group: lint
59+
60+
setup-cache-typecheck:
61+
name: Setup Cache (typecheck)
62+
needs: config
63+
uses: ./.github/workflows/cache-management.yml
64+
with:
65+
cache-type: dependencies
66+
cache-key-base: deps
67+
python-version: ${{ needs.config.outputs.default-python-version }}
68+
group: typecheck
69+
70+
setup-cache-arch:
71+
name: Setup Cache (arch)
72+
needs: config
73+
uses: ./.github/workflows/cache-management.yml
74+
with:
75+
cache-type: dependencies
76+
cache-key-base: deps
77+
python-version: ${{ needs.config.outputs.default-python-version }}
78+
group: arch
5679

5780
auto-format:
5881
name: Auto-Format Code
@@ -64,7 +87,7 @@ jobs:
6487
&& github.actor != 'open-resource-broker-cicd[bot]'
6588
runs-on: ubuntu-latest
6689
timeout-minutes: 15
67-
needs: [config, setup-cache]
90+
needs: [config, setup-cache-lint]
6891
# Override the workflow-level concurrency so this job is never cancelled
6992
# mid-run. If it were interrupted after `ruff` rewrites files but before
7093
# `git push`, the PR branch would be left in a half-formatted state.
@@ -94,7 +117,8 @@ jobs:
94117
- name: Setup UV with cache
95118
uses: ./.github/actions/setup-uv-cached
96119
with:
97-
cache-key: ${{ needs.setup-cache.outputs.cache-key }}
120+
cache-key: ${{ needs.setup-cache-lint.outputs.cache-key }}
121+
group: lint
98122
fail-on-cache-miss: false
99123

100124
- name: Auto-format code
@@ -128,7 +152,7 @@ jobs:
128152
&& github.event.pull_request.head.repo.full_name != github.repository
129153
runs-on: ubuntu-latest
130154
timeout-minutes: 15
131-
needs: [config, setup-cache]
155+
needs: [config, setup-cache-lint]
132156
permissions:
133157
contents: read
134158
pull-requests: write
@@ -142,7 +166,8 @@ jobs:
142166
- name: Setup UV with cache
143167
uses: ./.github/actions/setup-uv-cached
144168
with:
145-
cache-key: ${{ needs.setup-cache.outputs.cache-key }}
169+
cache-key: ${{ needs.setup-cache-lint.outputs.cache-key }}
170+
group: lint
146171
fail-on-cache-miss: false
147172

148173
- name: Run formatter
@@ -159,7 +184,7 @@ jobs:
159184
name: Ruff (Code Quality)
160185
runs-on: ubuntu-latest
161186
timeout-minutes: 15
162-
needs: [config, setup-cache]
187+
needs: [config, setup-cache-lint]
163188
permissions:
164189
contents: read
165190
steps:
@@ -169,7 +194,8 @@ jobs:
169194
- name: Setup UV with cache
170195
uses: ./.github/actions/setup-uv-cached
171196
with:
172-
cache-key: ${{ needs.setup-cache.outputs.cache-key }}
197+
cache-key: ${{ needs.setup-cache-lint.outputs.cache-key }}
198+
group: lint
173199
fail-on-cache-miss: false
174200

175201
- name: Check code quality
@@ -179,7 +205,7 @@ jobs:
179205
name: Ruff (Extended Checks)
180206
runs-on: ubuntu-latest
181207
timeout-minutes: 15
182-
needs: [config, setup-cache, lint-ruff]
208+
needs: [config, setup-cache-lint, lint-ruff]
183209
permissions:
184210
contents: read
185211
steps:
@@ -189,7 +215,8 @@ jobs:
189215
- name: Setup UV with cache
190216
uses: ./.github/actions/setup-uv-cached
191217
with:
192-
cache-key: ${{ needs.setup-cache.outputs.cache-key }}
218+
cache-key: ${{ needs.setup-cache-lint.outputs.cache-key }}
219+
group: lint
193220
fail-on-cache-miss: false
194221

195222
- name: Extended linting (warnings)
@@ -200,7 +227,7 @@ jobs:
200227
name: Type Checking (pyright)
201228
runs-on: ubuntu-latest
202229
timeout-minutes: 15
203-
needs: [config, setup-cache, lint-ruff]
230+
needs: [config, setup-cache-typecheck, lint-ruff]
204231
permissions:
205232
contents: read
206233
steps:
@@ -210,7 +237,8 @@ jobs:
210237
- name: Setup UV with cache
211238
uses: ./.github/actions/setup-uv-cached
212239
with:
213-
cache-key: ${{ needs.setup-cache.outputs.cache-key }}
240+
cache-key: ${{ needs.setup-cache-typecheck.outputs.cache-key }}
241+
group: typecheck
214242
fail-on-cache-miss: false
215243

216244
- name: Run pyright type check
@@ -221,7 +249,7 @@ jobs:
221249
name: Architecture Validation
222250
runs-on: ubuntu-latest
223251
timeout-minutes: 15
224-
needs: [config, setup-cache, lint-ruff]
252+
needs: [config, setup-cache-arch, lint-ruff]
225253
permissions:
226254
contents: read
227255
strategy:
@@ -243,7 +271,8 @@ jobs:
243271
- name: Setup UV with cache
244272
uses: ./.github/actions/setup-uv-cached
245273
with:
246-
cache-key: ${{ needs.setup-cache.outputs.cache-key }}
274+
cache-key: ${{ needs.setup-cache-arch.outputs.cache-key }}
275+
group: arch
247276
fail-on-cache-miss: false
248277

249278
- name: Run architecture validation (${{ matrix.description }})
@@ -254,16 +283,22 @@ jobs:
254283
runs-on: ubuntu-latest
255284
timeout-minutes: 15
256285
# Runs independently of lint to give early signal on version drift.
257-
# Only needs a checkout + pyyaml — no UV venv required.
258-
needs: config
286+
# pyyaml is available via the project's core [dependencies] so any
287+
# group that installs the project also provides it. Use the arch
288+
# group (smallest group that triggers a uv sync with the project).
289+
needs: [config, setup-cache-arch]
259290
permissions:
260291
contents: read
261292
steps:
262293
- name: Checkout code
263294
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
264295

265-
- name: Install PyYAML
266-
run: pip install --quiet pyyaml
296+
- name: Setup UV with cache
297+
uses: ./.github/actions/setup-uv-cached
298+
with:
299+
cache-key: ${{ needs.setup-cache-arch.outputs.cache-key }}
300+
group: arch
301+
fail-on-cache-miss: false
267302

268303
- name: Check Python version drift
269304
run: make ci-check-python-version-drift

0 commit comments

Comments
 (0)