Skip to content

chore(ci): merge-gated per-service test batteries (ADR-027) - #40

Merged
tcballard merged 4 commits into
mainfrom
ci/per-service-test-batteries
Jun 9, 2026
Merged

chore(ci): merge-gated per-service test batteries (ADR-027)#40
tcballard merged 4 commits into
mainfrom
ci/per-service-test-batteries

Conversation

@tcballard

Copy link
Copy Markdown
Collaborator

Summary

Reshapes RAC's CI test pipeline and records the policy as ADR-027. Bundles the merged ADR filename/heading hygiene fixes from #39.

Changes:

  • tests.yml: the reusable suite is split into a per-service battery × Python-version matrix — 11 batteries (8 .py services + grouped core/cli/artifacts) × 3.11/3.12/3.13 = 33 named jobs (e.g. relationships (py3.11)), replacing the single pytest (3.x) matrix.
  • ci.yml: tests run only on merge to main (push: [main]) plus workflow_dispatch; the pull_request trigger is removed; cancel-in-progress: false.
  • rac/decisions/adr-027-ci-test-topology.md: records the trigger policy, release gate, and battery topology.
  • ADR hygiene (via fix(decisions): correct ADR filenames and heading numbers #39): adr-024 file rename + four heading-number corrections.

Roadmap / ADR Trace

No roadmap item — CI infrastructure plus repository hygiene.

  • New: rac/decisions/adr-027-ci-test-topology.md
  • Related: adr-008-agent-ready-architecture (service architecture the batteries mirror), adr-005-cli-first + adr-007-json-contract-stability (public contracts the release gate protects), adr-003-structured-outputs-first.

Scope

Included

  • Per-service battery matrix with service+version job names for at-a-glance triage.
  • Merge-only trigger (push: [main]) + workflow_dispatch; cancel-in-progress: false so every merge is fully tested.
  • ADR-027 recording the decision.
  • Bundled ADR hygiene fixes (4 files) merged from fix(decisions): correct ADR filenames and heading numbers #39.

Excluded

  • No change to python-publish.yml — the release gate (release-build: needs: test) was already present; it now automatically enforces all 33 batteries because it calls the same reusable workflow.
  • No pre-merge PR testing — deliberate (ADR-027). PRs get no automated test run; workflow_dispatch is the manual escape hatch.
  • No dynamic matrix discovery — the battery list is an explicit static enumeration (deterministic, readable).
  • No pytest markers or source changes — batteries select tests by path.
  • No ADR body edits beyond the H1 heading lines (e.g. Category not added to adr-024).

Product / Architecture Decisions

  • A battery = one .py service, plus grouped core / cli / artifacts; tests are selected by path, so no markers or source changes are required.
  • Job naming name: <battery> (py<version>) surfaces the responsible service in the Actions UI (the original motivation).
  • Trigger policy: test on merge to main, not on PRs (ADR-027). The post-merge-feedback tradeoff is accepted and documented; a regression is caught on the main run and blocks the next release via the gate.
  • cancel-in-progress: false on main because those runs gate releases and each merge should be tested in full.
  • Explicit static matrix chosen over dynamic discovery for determinism (a RAC core principle); orphan risk is guarded by a coverage check (every tests/test_*.py maps to exactly one battery).

User-Facing Contract

CLI / Human / JSON

None — no application behavior, schema, or exit-code changes. This PR is CI configuration plus documentation.

CI (developer-facing)

  • On merge to main or manual dispatch: 33 named checks (core (py3.11), relationships (py3.12), …).
  • On pull requests: no automated test run (by design).
  • On release: build/publish gated on all 33 batteries passing.

Verification

Ran

.venv/bin/python -m pytest -q            # 398 passed
# workflow YAML parsed: matrix = 11 batteries x [3.11, 3.12, 3.13] = 33 jobs, fail-fast: false
# ci.yml triggers = push:[main] + workflow_dispatch (no pull_request); cancel-in-progress: false
# battery coverage: 19/19 test files mapped, 0 orphans
# ADR corpus: 27/27 filename == heading number

Covered

  • Full suite green on the integrated branch (398 passed).
  • Representative batteries run in isolation exactly as CI invokes them (core 41, artifacts 134, relationships 72, index 17).
  • Every tests/test_*.py belongs to exactly one battery — no test silently unrun.
  • ci.yml no longer triggers on pull_request; workflow_dispatch present for manual runs.
  • All 27 ADRs have matching filename ↔ heading numbers.
  • Note: the live 33-job run is first observable on merge to main or via workflow_dispatch, since tests no longer run on PRs.

Review Path

  1. .github/workflows/tests.yml — the battery × version matrix and per-battery pytest invocation.
  2. .github/workflows/ci.yml — trigger policy (push: [main] + workflow_dispatch, no pull_request).
  3. rac/decisions/adr-027-ci-test-topology.md — the recorded decision and its tradeoffs.
  4. rac/decisions/adr-019 / adr-024 / adr-025 / adr-026 — bundled hygiene fixes (from fix(decisions): correct ADR filenames and heading numbers #39).

Notes For Reviewer

  • This PR can't show a green test run on the PR itself — tests are merge-gated by design (ADR-027). To see the 33-job grid before merging, run the CI workflow via “Run workflow” (workflow_dispatch) on this branch.
  • python-publish.yml is intentionally untouched; it already consumes the reusable workflow, so the release gate now requires every battery.
  • Deferred, not in scope: adr-024 missing Category section; adr-019 title ("Asset References") vs filename slug ("asset-management").

tcballard added 4 commits June 9, 2026 11:26
Tests now run only on push to main (a merged PR or direct push) plus workflow_dispatch; the pull_request trigger is removed so in-flight branches no longer re-run the full suite on every push.

tests.yml is restructured into a battery x Python-version matrix — one battery per .py service plus grouped core/cli/artifacts (11 batteries x 3.11/3.12/3.13 = 33 jobs) — so a failed check names the service and version. Releases stay gated on the same reusable workflow. Recorded in ADR-027.
Document the CI test topology decided on this branch: the test-on-merge trigger policy and its post-merge-feedback tradeoff, release gating, and the per-service battery matrix. Status Accepted; category Process.
Rename adr024-rac-not-content-store.md -> adr-024-rac-not-content-store.md to match the adr-NNN-slug.md convention every other ADR uses, and correct the heading, which read 'ADR-016' (a number owned by adr-016-relationships-as-structural-references.md) to 'ADR-024'.
Three ADR bodies carried wrong headings: adr-019-asset-management read 'ADR-017' (a number owned by adr-017-rac-managed-knowledge-not-work), and adr-025/adr-026 still had '# ADR-XXX' placeholders. Align each heading with its filename number and the '# ADR-NNN: Title' convention.
@tcballard

Copy link
Copy Markdown
Collaborator Author

Verification — manual workflow_dispatch run ✅

Tests are merge-gated (ADR-027), so this PR cannot run them on its own. Triggered the CI workflow manually on ci/per-service-test-batteries to prove the battery grid before merge.

All 11 batteries × {3.11, 3.12, 3.13}, each job named tests / <service> (py<version>):

core · cli · artifacts · diff · improve · index · ingest · inspect · portfolio · relationships · stats — green on every supported Python version.

A failure now names the exact service and version (e.g. tests / relationships (py3.12)) instead of a generic pytest (3.11).

@tcballard
tcballard merged commit fc7349f into main Jun 9, 2026
66 checks passed
@tcballard
tcballard deleted the ci/per-service-test-batteries branch June 9, 2026 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant