ci: split build matrix on physics/api marker axis (PR B for #1300) (#1311)

* ci: split build matrix on physics/api marker axis (#1300 PR B)

Follow-up to #1305 (PR A). Now that every test file declares `physics`
or `api`, the CI matrix can stop running the full suite on every Python
and instead run each nature axis where its coverage genuinely matters.

- build-suews/action.yml: CIBW_TEST_COMMAND runs `-m "physics and
  <tier>"` on the build Python only. Physics tests are binary-
  determined (compiled artefact + CPU floating-point), so one run per
  (OS, arch) suffices post-abi3 (#1299).
- determine-matrix.sh: rename output `test_python` -> `api_python` to
  reflect that this matrix now drives the api-tier cross-Python tests,
  not a generic bridge-loading subset.
- test-api-cross-python-reusable.yml (renamed from test-bridge-reusable
  .yml): installs the single abi3 wheel into each test CPython and runs
  `-m "api and <tier>"`. Composes the tier expression inline so the
  caller doesn't need to.
- build-publish_to_pypi.yml: rename job `test_bridge_loading` ->
  `test_api_cross_python`, pass `test_tier` into the reusable workflow,
  and add a new standalone `check_test_markers` job that runs a static
  AST-based lint (no supy build required) ensuring every test file
  declares physics or api. The lint is gated into `pr-gate` so a
  missing marker blocks the merge.
- scripts/lint/check_test_markers.py: the static lint. Skips files
  under `test/umep/` which pick up `api` via the conftest hook.
- pr-gate.sh: surface check_test_markers result alongside build and
  api-test results.

Net effect:
  - PR / merge queue: physics runs on N_platforms x 1 Python, api runs
    on N_platforms x 2 bookend Pythons. Full-tree marker lint runs
    once (single-platform, fast).
  - Nightly / tag: physics x 4 platforms + api x (4 platforms x 6
    Pythons). Total CPU similar to pre-split, but each cell now
    carries orthogonal information.

Docs: RELEASE_MANUAL.md and prep-release skill reference the renamed
job; test/README.md notes that `smoke_bridge` is now legacy (coverage
subsumed by `api and smoke`). Tests tagged `@pytest.mark.smoke_bridge`
continue to work; the marker is still registered.

* test: tag data_model yaml fixture tests with api marker (gh#1300)

Three tests landed on master between #1305 and this PR and need
the nature-axis marker. All three exercise pydantic/yaml config
validation and the suews-convert yaml-upgrade CLI, so they fall
cleanly into the api tier.

Fixes the Check pytest marker axis CI job on this PR.

* docs(rules): require physics/api marker on every new test file

Update the tests/patterns.md rule (auto-loaded when editing test files)
so contributors and AI assistants see the nature-axis requirement
up-front when designing new tests. The lint in
scripts/lint/check_test_markers.py is the hard gate; this rule is the
soft gate that prevents the lint from firing in the first place.

gh#1300

Author: sunt05

.claude/rules/tests/patterns.md

@@ -62,6 +62,47 @@ with pytest.raises(ValueError, match="must be positive"):
 
 ## Pytest Markers
 
+Markers sit on two orthogonal axes (gh#1300). Every new test file **must**
+declare the nature axis at module level; tier markers compose on top as
+per-test decorators.
+
+### Nature axis — REQUIRED on every file
+
+Pick exactly one (or both, rarely) and declare at module level:
+
+```python
+import pytest
+
+pytestmark = pytest.mark.api        # Python wrapper surface
+# or:
+pytestmark = pytest.mark.physics    # numerical / binary correctness
+# or (file straddles both):
+pytestmark = [pytest.mark.physics, pytest.mark.api]
+```
+
+- `physics` — outputs determined by the compiled artefact + CPU
+  floating-point. CI runs these once per `(OS, arch)` on the build
+  Python. Typical: mass/energy balance, DailyState accumulation, Fortran
+  state persistence.
+- `api` — exercises the pandas / numpy / pydantic / click surface. CI
+  runs these across `(platform x Python)` because the dependency
+  surface varies per interpreter. Typical: config validation, CLI, YAML
+  round-trip, `SUEWSSimulation` methods.
+
+When unsure, pick `api` — it's the broader coverage axis and safer if
+the test is genuinely mixed.
+
+UMEP tests (`test/umep/*.py`) pick up `api` automatically via
+`test/umep/conftest.py`; no file-level declaration needed there.
+
+**A static CI lint (`scripts/lint/check_test_markers.py`) and a
+`pytest_collection_finish` hook in `test/conftest.py` both fail any PR
+that introduces a test file without a nature marker.** If you see the
+lint fire, add a `pytestmark = pytest.mark.api` (or physics) line — do
+not bypass.
+
+### Tier axis — per-test decorators
+
 ```python
 @pytest.mark.smoke   # Critical, fast tests (~60s total)
 @pytest.mark.core    # Core physics/logic tests
@@ -70,6 +111,10 @@ with pytest.raises(ValueError, match="must be positive"):
 @pytest.mark.cfg     # Config/schema validation tests
 ```
 
+The tier axis composes with the nature axis. CI expressions like
+`-m "api and smoke"` and `-m "physics and not slow"` select the right
+subset per matrix cell.
+
 ---
 
 ## Test File Locations

.claude/skills/prep-release/references/release-steps.md

@@ -303,7 +303,7 @@ git push origin "$VERSION"
 ## Step 7: Post-Release Verification
 
 **Monitor (~20 min):**
-- GitHub Actions: build_wheels, test_bridge_loading, publish
+- GitHub Actions: build_wheels, test_api_cross_python, publish
 - PyPI: supy `YYYY.M.D` appears (one cp39-abi3 wheel per platform)
 - GitHub Release: created automatically after successful publish
 - Zenodo DOI appears on the dashboard

.github/actions/build-suews/action.yml

@@ -157,26 +157,24 @@ runs:
           pip install delvewheel
         CIBW_REPAIR_WHEEL_COMMAND_WINDOWS: "delvewheel repair -w {dest_dir} {wheel}"
         CIBW_TEST_REQUIRES: pytest
-        # Test tiers: choose subset based on path detection and trigger
-        # Always include --durations=10 to show slowest tests in output
+        # Test tiers: choose subset based on path detection and trigger.
+        # gh#1300: physics tests are binary-determined, so one run per
+        # (OS, arch) on the build Python is sufficient. API tests (Python
+        # wrapper surface) run separately per (platform x Python) via
+        # test-api-cross-python-reusable.yml.
+        # Always include --durations=10 to show slowest tests in output.
         # After pytest, enforce bridge-manifest coverage of public Fortran types.
         # Uses run_ci_tests.py wrapper to avoid shell-level && chaining,
         # which breaks on Windows cmd.exe with quoted marker expressions.
-        # gh#1300: the expressions below compose tier markers only. A
-        # follow-up PR splits the matrix on the `physics` vs `api` nature
-        # axis — `physics` runs once per (OS, arch) on the canonical
-        # Python while `api` runs per (platform x Python). No behaviour
-        # change here yet; the new markers are registered in
-        # pyproject.toml and applied to every test file, ready to be
-        # consumed when the matrix split lands.
         CIBW_TEST_COMMAND: >-
           python {project}/scripts/suews/run_ci_tests.py {project}
           -v --tb=short --durations=10
-          ${{ inputs.test_tier == 'smoke' && '-m smoke' ||
-              inputs.test_tier == 'core' && '-m "smoke or core"' ||
-              inputs.test_tier == 'cfg' && '-m "smoke or cfg"' ||
-              inputs.test_tier == 'standard' && '-m "not slow"' ||
-              '' }}
+          ${{ inputs.test_tier == 'smoke' && '-m "physics and smoke"' ||
+              inputs.test_tier == 'core' && '-m "physics and (smoke or core)"' ||
+              inputs.test_tier == 'cfg' && '-m "physics and (smoke or cfg)"' ||
+              inputs.test_tier == 'standard' && '-m "physics and not slow"' ||
+              inputs.test_tier == 'all' && '-m physics' ||
+              '-m physics' }}
         MACOSX_DEPLOYMENT_TARGET: '15.0'
 
     - name: Upload wheels

.github/scripts/determine-matrix.sh

@@ -2,11 +2,15 @@
 # Determine the cibuildwheel build matrix based on trigger type and change detection.
 #
 # Called from build-publish_to_pypi.yml determine_matrix job.
-# Writes buildplat, python, test_python, test_tier to GITHUB_OUTPUT.
+# Writes buildplat, python, api_python, test_tier to GITHUB_OUTPUT.
 #
 # "python" is the cibuildwheel build matrix (always cp39 — emits one abi3
-# wheel per platform). "test_python" is the cross-version bridge-loading
-# matrix (BOOKEND for PRs, ALL for nightly/tag).
+# wheel per platform). Physics tests run during the build step on the
+# build Python only, since they're binary-determined (gh#1300).
+# "api_python" is the cross-version matrix used by test-api-cross-python;
+# the api marker covers the Python wrapper surface (pandas/numpy/pydantic,
+# CLI, SUEWSSimulation) and needs full (platform x Python) coverage
+# (BOOKEND for PRs, ALL for nightly/tag).
 #
 # Required environment variables:
 #   EVENT_NAME           -- github.event_name
@@ -55,7 +59,7 @@ if [[ "${FORTRAN_CHANGED}" == "true" ]] || [[ "${RUST_CHANGED}" == "true" ]] ||
   NEEDS_MULTIPLATFORM=true
 fi
 
-TEST_PYTHON="$BOOKEND_PYTHON"
+API_PYTHON="$BOOKEND_PYTHON"
 
 if [[ "${EVENT_NAME}" == "pull_request" ]] && [[ "${IS_DRAFT}" == "true" ]]; then
   if [[ "${FORTRAN_CHANGED}" == "true" ]] || [[ "${RUST_CHANGED}" == "true" ]]; then
@@ -100,15 +104,15 @@ elif [[ "${EVENT_NAME}" == "schedule" ]]; then
   echo "buildplat=$FULL_PLATFORMS" >> "$GITHUB_OUTPUT"
   echo "python=$BUILD_PYTHON" >> "$GITHUB_OUTPUT"
   echo "test_tier=all" >> "$GITHUB_OUTPUT"
-  TEST_PYTHON="$ALL_PYTHON"
+  API_PYTHON="$ALL_PYTHON"
 
 elif [[ "${EVENT_NAME}" == "workflow_dispatch" ]]; then
   case "${INPUT_MATRIX_CONFIG}" in
     full)
       echo "Manual dispatch: full matrix"
       echo "buildplat=$FULL_PLATFORMS" >> "$GITHUB_OUTPUT"
       echo "python=$BUILD_PYTHON" >> "$GITHUB_OUTPUT"
-      TEST_PYTHON="$ALL_PYTHON"
+      API_PYTHON="$ALL_PYTHON"
       ;;
     pr)
       echo "Manual dispatch: PR-style reduced matrix"
@@ -122,7 +126,7 @@ elif [[ "${EVENT_NAME}" == "workflow_dispatch" ]]; then
       ;;
     custom)
       echo "Manual dispatch: custom platform matrix"
-      echo "  Build is always cp39-abi3; py3X toggles select the bridge-test matrix"
+      echo "  Build is always cp39-abi3; py3X toggles select the api cross-CPython matrix"
 
       PLATFORMS="["
       [[ "${INPUT_PLAT_LINUX}" == "true" ]] && PLATFORMS+='["ubuntu-latest", "manylinux", "x86_64"],'
@@ -147,13 +151,13 @@ elif [[ "${EVENT_NAME}" == "workflow_dispatch" ]]; then
       TEST_PYS="${TEST_PYS%,}]"
 
       if [[ "$TEST_PYS" == "[]" ]]; then
-        echo "::error::Custom matrix requires at least one Python version for bridge tests"
+        echo "::error::Custom matrix requires at least one Python version for api tests"
         exit 1
       fi
 
       echo "buildplat=$PLATFORMS" >> "$GITHUB_OUTPUT"
       echo "python=$BUILD_PYTHON" >> "$GITHUB_OUTPUT"
-      TEST_PYTHON="$TEST_PYS"
+      API_PYTHON="$TEST_PYS"
       ;;
   esac
   echo "test_tier=${INPUT_TEST_TIER}" >> "$GITHUB_OUTPUT"
@@ -164,11 +168,12 @@ else
   echo "buildplat=$FULL_PLATFORMS" >> "$GITHUB_OUTPUT"
   echo "python=$BUILD_PYTHON" >> "$GITHUB_OUTPUT"
   echo "test_tier=all" >> "$GITHUB_OUTPUT"
-  TEST_PYTHON="$ALL_PYTHON"
+  API_PYTHON="$ALL_PYTHON"
 fi
 
-# Cross-version bridge-loading matrix: BOOKEND for PRs/merge queue, ALL for
-# nightly/tag/dispatch-full. The same single abi3 wheel is installed into
-# each Python version and exercised with -m smoke_bridge.
-echo "test_python=$TEST_PYTHON" >> "$GITHUB_OUTPUT"
+# Cross-version api-test matrix (gh#1300): BOOKEND for PRs/merge queue,
+# ALL for nightly/tag/dispatch-full. The same single abi3 wheel is
+# installed into each Python version and exercised with
+# `-m "api and <tier>"` by test-api-cross-python-reusable.yml.
+echo "api_python=$API_PYTHON" >> "$GITHUB_OUTPUT"
 

.github/scripts/pr-gate.sh

@@ -7,15 +7,19 @@
 # Required environment variables:
 #   DETECT_CHANGES_RESULT -- needs.detect-changes.result
 #   BUILD_WHEELS_RESULT   -- needs.build_wheels.result
-#   TEST_BRIDGE_RESULT    -- needs.test_bridge_loading.result
+#   TEST_BRIDGE_RESULT    -- needs.test_api_cross_python.result
+#   CHECK_MARKERS_RESULT  -- needs.check_test_markers.result
 #   NEEDS_BUILD           -- needs.detect-changes.outputs.needs-build
 
 set -euo pipefail
 
+CHECK_MARKERS_RESULT="${CHECK_MARKERS_RESULT:-skipped}"
+
 echo "=== Job Results ==="
 echo "detect-changes: ${DETECT_CHANGES_RESULT}"
 echo "build_wheels: ${BUILD_WHEELS_RESULT}"
-echo "test_bridge_loading: ${TEST_BRIDGE_RESULT}"
+echo "test_api_cross_python: ${TEST_BRIDGE_RESULT}"
+echo "check_test_markers: ${CHECK_MARKERS_RESULT}"
 echo "needs-build: ${NEEDS_BUILD}"
 echo ""
 
@@ -35,6 +39,18 @@ fi
 
 VALIDATION_PASSED=true
 
+# Marker-axis lint (gh#1300) runs on every PR regardless of needs-build.
+echo "Validating test marker axis (gh#1300)..."
+if [[ "${CHECK_MARKERS_RESULT}" == "success" ]]; then
+  echo "[OK] Every test file declares physics or api"
+elif [[ "${CHECK_MARKERS_RESULT}" == "skipped" ]]; then
+  echo "[OK] Marker lint skipped (not a PR/merge-queue/dispatch run)"
+else
+  echo "[X] Marker lint failed - some test file is missing physics/api marker"
+  echo "  check_test_markers: ${CHECK_MARKERS_RESULT}"
+  VALIDATION_PASSED=false
+fi
+
 if [[ "${NEEDS_BUILD}" == "true" ]]; then
   echo "Code changes detected - validating standard build..."
   if [[ "${BUILD_WHEELS_RESULT}" == "success" ]]; then
@@ -45,12 +61,12 @@ if [[ "${NEEDS_BUILD}" == "true" ]]; then
     VALIDATION_PASSED=false
   fi
 
-  echo "Validating bridge-load smoke (cross-CPython)..."
+  echo "Validating api cross-CPython tests..."
   if [[ "${TEST_BRIDGE_RESULT}" == "success" ]]; then
-    echo "[OK] Bridge-load smoke passed on all matrix cells"
+    echo "[OK] API tests passed on all matrix cells"
   else
-    echo "[X] Bridge-load smoke failed"
-    echo "  test_bridge_loading: ${TEST_BRIDGE_RESULT}"
+    echo "[X] API cross-CPython tests failed"
+    echo "  test_api_cross_python: ${TEST_BRIDGE_RESULT}"
     VALIDATION_PASSED=false
   fi
 
@@ -63,7 +79,7 @@ else
   else
     echo "Note: Unexpected build activity for non-code PR"
     echo "  build_wheels: ${BUILD_WHEELS_RESULT}"
-    echo "  test_bridge_loading: ${TEST_BRIDGE_RESULT}"
+    echo "  test_api_cross_python: ${TEST_BRIDGE_RESULT}"
     # Still pass if builds succeeded (conservative)
     for result in "${BUILD_WHEELS_RESULT}" "${TEST_BRIDGE_RESULT}"; do
       if [[ "$result" != "success" ]] && [[ "$result" != "skipped" ]]; then

.github/workflows/build-publish_to_pypi.yml

@@ -65,10 +65,10 @@ on:
         type: boolean
         default: true
 
-      # Python toggles drive the cross-CPython bridge-loading test matrix.
+      # Python toggles drive the cross-CPython api-test matrix (gh#1300).
       # The build is always cp39-abi3 (one wheel covers all supported CPythons),
-      # so these toggles only affect which interpreters install and import the
-      # wheel under -m smoke_bridge.
+      # so these toggles only affect which interpreters install the wheel and
+      # run `-m "api and <tier>"` in test_api_cross_python.
       py39:
         description: '[custom] Test on Python 3.9 (build is always cp39-abi3)'
         type: boolean
@@ -280,7 +280,7 @@ jobs:
     outputs:
       buildplat: ${{ steps.set-matrix.outputs.buildplat }}
       python: ${{ steps.set-matrix.outputs.python }}
-      test_python: ${{ steps.set-matrix.outputs.test_python }}
+      api_python: ${{ steps.set-matrix.outputs.api_python }}
       test_tier: ${{ steps.set-matrix.outputs.test_tier }}
     steps:
       - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
@@ -380,21 +380,50 @@ jobs:
       test_tier: ${{ needs.determine_matrix.outputs.test_tier }}
       is_testpypi_build: ${{ github.event_name == 'schedule' || (startsWith(github.ref, 'refs/tags/') && contains(github.ref, 'dev')) }}
 
-  test_bridge_loading:
-    name: Cross-CPython bridge tests
+  test_api_cross_python:
+    name: API cross-CPython tests
     needs: [determine_matrix, build_wheels]
     if: always() && needs.build_wheels.result == 'success'
-    uses: ./.github/workflows/test-bridge-reusable.yml
+    uses: ./.github/workflows/test-api-cross-python-reusable.yml
     with:
       buildplat_json: ${{ needs.determine_matrix.outputs.buildplat }}
-      test_python_json: ${{ needs.determine_matrix.outputs.test_python }}
+      api_python_json: ${{ needs.determine_matrix.outputs.api_python }}
+      test_tier: ${{ needs.determine_matrix.outputs.test_tier }}
+
+  # Standalone lint: every test file must declare `physics` or `api`
+  # (gh#1300). Uses a static AST-based script so CI can verify marker
+  # coverage without building supy — keeps the check cheap for
+  # test-only PRs. The runtime counterpart is a pytest_collection_finish
+  # hook in test/conftest.py, which catches the same thing during normal
+  # full-tree pytest runs.
+  check_test_markers:
+    name: Check pytest marker axis
+    runs-on: ubuntu-latest
+    if: >-
+      always() &&
+      (github.event_name == 'pull_request' ||
+       github.event_name == 'merge_group' ||
+       github.event_name == 'schedule' ||
+       github.event_name == 'workflow_dispatch')
+    steps:
+      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
+        with:
+          persist-credentials: false
+
+      - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
+        with:
+          python-version: '3.12'
+
+      - name: Lint physics/api marker coverage
+        shell: bash
+        run: python scripts/lint/check_test_markers.py
 
   # Single consolidated check for branch protection (always runs for PRs/merge queue)
   pr-gate:
     name: PR build validation
     runs-on: ubuntu-latest
     if: always() && (github.event_name == 'pull_request' || github.event_name == 'merge_group')
-    needs: [build_wheels, test_bridge_loading, detect-changes]
+    needs: [build_wheels, test_api_cross_python, check_test_markers, detect-changes]
     steps:
       - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
         with:
@@ -406,7 +435,8 @@ jobs:
         env:
           DETECT_CHANGES_RESULT: ${{ needs.detect-changes.result }}
           BUILD_WHEELS_RESULT: ${{ needs.build_wheels.result }}
-          TEST_BRIDGE_RESULT: ${{ needs.test_bridge_loading.result }}
+          TEST_BRIDGE_RESULT: ${{ needs.test_api_cross_python.result }}
+          CHECK_MARKERS_RESULT: ${{ needs.check_test_markers.result }}
           NEEDS_BUILD: ${{ needs.detect-changes.outputs.needs-build }}
         run: bash .github/scripts/pr-gate.sh
 
@@ -416,17 +446,17 @@ jobs:
     needs:
       - determine_matrix
       - build_wheels
-      - test_bridge_loading
+      - test_api_cross_python
       - create_nightly_tag
       - verify-release-tag
     # Publish only when every platform wheel built AND the cross-CPython
-    # bridge smoke passed on every matrix cell. A partial build (one leg
-    # fails) would leave test_bridge_loading skipped; allowing that would
-    # ship un-validated wheels, so both must be explicit successes.
+    # api tests passed on every matrix cell. A partial build (one leg
+    # fails) would leave test_api_cross_python skipped; allowing that
+    # would ship un-validated wheels, so both must be explicit successes.
     if: |
       always() &&
       needs.build_wheels.result == 'success' &&
-      needs.test_bridge_loading.result == 'success' &&
+      needs.test_api_cross_python.result == 'success' &&
       (needs.create_nightly_tag.result == 'success' || needs.create_nightly_tag.result == 'skipped') &&
       (needs.verify-release-tag.result == 'skipped' || needs.verify-release-tag.outputs.on-master == 'true') &&
       (
@@ -459,18 +489,18 @@ jobs:
     needs:
       - determine_matrix
       - build_wheels
-      - test_bridge_loading
+      - test_api_cross_python
       - create_nightly_tag
       - verify-release-tag
-    # Production tags only: all platform wheels must succeed AND the cross-CPython
-    # bridge-loading smoke must pass (the only signal the cp39-abi3 wheel imports
-    # on cp310..cp314 after the build matrix collapse).
+    # Production tags only: all platform wheels must succeed AND the
+    # cross-CPython api tests must pass (the only signal the cp39-abi3
+    # wheel imports on cp310..cp314 after the build matrix collapse).
     if: |
       always() &&
       startsWith(github.ref, 'refs/tags') &&
       !contains(github.ref, 'dev') &&
       needs.build_wheels.result == 'success' &&
-      needs.test_bridge_loading.result == 'success' &&
+      needs.test_api_cross_python.result == 'success' &&
       (needs.create_nightly_tag.result == 'success' || needs.create_nightly_tag.result == 'skipped') &&
       (needs.verify-release-tag.result == 'skipped' || needs.verify-release-tag.outputs.on-master == 'true')