Merge pull request #12 from FormingWorlds/tl/interior-refactor

This branch is a rewrite of Aragog around a pressure-entropy formulation. Specific entropy at staggered nodes becomes the prognostic variable; T, ρ, φ, c_p, α, and (∂T/∂P)_S are read from SPIDER-format P-S tables generated by Zalmoxis from PALEOS. The earlier P-T path is gone, the mush zone is now resolved continuously across the solidus and liquidus crossings, and the energy budget is re-derived around the lever-rule blender (latent-augmented c_p, harmonic-mean ρ).

Production integration. SUNDIALS CVODE drives a JAX-derived analytic Jacobian as the default; numpy and JAX paths agree to float-64 machine precision on the production mid-solidification and near-solid regimes. Four core-BC modes with SPIDER bit-parity at energy_balance, a per-call mass-weighted ΔΦ cap as a SUNDIALS root function for the mush regime, scipy Radau / BDF as fall-backs. AragogRunner makes Aragog a first-class citizen in the PROTEUS coupled loop, with a documented hf_row contract, a Zalmoxis mesh hand-off, and Newton-solve mass coordinates.

Energy bookkeeping rewritten end to end. Per-call J integrals replace a state-mass residual that grew with mantle cooling, and a solver-residual diagnostic closes at ~1e-7 of total cooling on production trajectories. 

Configuration, tests, docs. Parameters.from_file dispatches on file suffix (tomllib for .toml, typed_configparser for .cfg) so quote and inline-comment leakage on TOML inputs no longer corrupts strings; [scalings] is hard-rejected on both paths. The test surface is rebuilt to PROTEUS Ecosystem Testing Standard v1: 4-marker scheme with --strict-markers, marker-validation gate, 95% coverage gate with one-way ratchet, Codecov nightly upload, auto-refreshing test-count badges. Currently 575 tests at 96.81% nightly coverage. Documentation restructured per Diátaxis, verification figure suite under docs/figures/vv/, HPC-cluster guidance in installation, tutorial that runs end-to-end against the bundled EOS-tables tarball.

CI. ci_tests.yml runs the unit tier on ubuntu + macos × py3.12 / 3.13, scoped to push:main + pull_request:main with concurrency cancellation; nightly.yml runs the full marker set with the 95% gate enforced and uploads coverage to Codecov on schedule and on every push to main. Python floor tightened to >=3.12 to match the matrix.

Aragog is now the production interior-energetics backend of PROTEUS, with the regression infrastructure to keep it that way.

Author: timlichtenberg

.claude/rules/proteus-tests.md

@@ -0,0 +1 @@
+../../.claude/rules/proteus-tests.md

.codecov.yml

@@ -0,0 +1,14 @@
+coverage:
+  status:
+    project:
+      default:
+        target: auto
+        threshold: 1%
+    patch:
+      default:
+        target: auto
+        threshold: 1%
+
+comment:
+  layout: "reach, diff, flags, files"
+  require_changes: true

.github/badges/tests-integration.json

@@ -0,0 +1,6 @@
+{
+  "schemaVersion": 1,
+  "label": "integration tests",
+  "message": "65",
+  "color": "blue"
+}

.github/badges/tests-total.json

@@ -0,0 +1,6 @@
+{
+  "schemaVersion": 1,
+  "label": "tests",
+  "message": "575",
+  "color": "blue"
+}

.github/badges/tests-unit.json

@@ -0,0 +1,6 @@
+{
+  "schemaVersion": 1,
+  "label": "unit tests",
+  "message": "514",
+  "color": "blue"
+}

.github/copilot-instructions.md

@@ -0,0 +1 @@
+../CLAUDE.md

.github/workflows/ci_tests.yml

@@ -1,7 +1,4 @@
-# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
-# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
-
-name: Tests for Aragog
+name: Aragog CI Test Suite
 
 on:
   push:
@@ -17,41 +14,177 @@ on:
       - ready_for_review
   workflow_dispatch:
 
+# Push and PR CI: unit tests only, on ubuntu + macos x py 3.12 / 3.13.
+# Targets <5 min wall. Slow tests (full multi-Myr CVODE solves) live in
+# the scheduled nightly workflow `nightly.yml` instead, alongside coverage
+# enforcement against the 95% floor. Running the slow suite or coverage
+# instrumentation on every push burns ~10 min CI time per commit and gives
+# no bug-finding signal that the unit suite doesn't already cover.
+#
+# Trigger scoping: push fires only on main (release-merge sanity check),
+# pull_request fires on PRs into main (feature-branch validation). With
+# both, every commit on a PR runs exactly one matrix sweep instead of two
+# (push + pull_request would otherwise both fire on a feature-branch
+# push that has an open PR).
+#
+# Uses micromamba + conda-forge for the env so SUNDIALS, gfortran, and the
+# scikits-odes binary stack come prebuilt instead of being compiled from
+# source against stock apt/brew (which lag the SUNDIALS major version
+# scikits-odes-sundials 3.x requires).
+
 permissions:
-  actions: write
-  contents: write
+  contents: read
 
-jobs:
-  build:
+# Cancel superseded runs on the same ref so a stack of feature-branch
+# pushes does not pile up macOS runners (free-tier concurrency is tight).
+# The base ref keeps push to main and pull_request to main in separate
+# groups so the latter cannot cancel a release sanity check.
+concurrency:
+  group: ci-tests-${{ github.workflow }}-${{ github.event_name }}-${{ github.ref }}
+  cancel-in-progress: true
 
+jobs:
+  test-fast:
+    name: Aragog-check (${{ matrix.os }}, py${{ matrix.python-version }})
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.11", "3.12","3.13"]
         os: [ubuntu-latest, macos-latest]
+        python-version: ["3.12", "3.13"]
 
     runs-on: ${{ matrix.os }}
 
+    defaults:
+      run:
+        shell: bash -el {0}
+
     steps:
-    - uses: actions/checkout@v5
-
-    - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v5
-      with:
-        python-version: ${{ matrix.python-version }}
-
-    - name: Install dependencies
-      run: |
-        python -m pip install --upgrade pip
-        python -m pip install flake8 pytest pytest-cov pytest-dependency
-        if [ -f pyproject.toml ]; then pip install .; fi
-
-    - name: Lint with flake8
-      run: |
-        # stop the build if there are Python syntax errors or undefined names
-        flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
-        # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
-        flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
-
-    - name: Test with pytest
-      run: pytest --cov-report term --cov-report markdown-append:$GITHUB_STEP_SUMMARY --cov=aragog tests/
+      - uses: actions/checkout@v5
+        with:
+          fetch-depth: 0
+
+      - name: Set up conda env (sundials from conda-forge)
+        uses: mamba-org/setup-micromamba@v2
+        with:
+          environment-name: aragog-test
+          create-args: >-
+            python=${{ matrix.python-version }}
+            sundials
+          condarc: |
+            channels:
+              - conda-forge
+
+      - name: Install pip-only test deps (scikits-odes-sundials builds against conda SUNDIALS)
+        run: |
+          pip install pytest pytest-xdist pytest-dependency ruff
+          # Install scikits-odes-sundials directly (skip the scikits-odes
+          # metapackage). This avoids scikits-odes-daepack, which is a
+          # transitive Fortran dep with f2py incompatibility with numpy
+          # 2.3+ and needs gfortran on macOS. aragog imports CVODE
+          # directly from scikits_odes_sundials.cvode and has no use for
+          # the DAE solvers in the metapackage.
+          pip install scikits-odes-sundials jax equinox
+
+      - name: Install aragog
+        run: |
+          pip install -e .
+
+      - name: Fetch SPIDER EOS test data
+        run: |
+          mkdir -p /tmp/aragog-test-data
+          curl -sL -o /tmp/spider_eos.tar.gz \
+            https://github.com/FormingWorlds/aragog/releases/download/test-data-v1/spider_eos_test_data.tar.gz
+          tar xzf /tmp/spider_eos.tar.gz -C /tmp/aragog-test-data/
+          echo "ARAGOG_TEST_EOS_DIR=/tmp/aragog-test-data/spider_eos" >> $GITHUB_ENV
+
+      - name: Validate test markers
+        if: ${{ matrix.os == 'ubuntu-latest' && matrix.python-version == '3.12' }}
+        run: bash tools/validate_test_structure.sh
+
+      - name: Lint with Ruff
+        run: |
+          ruff check src/ tests/
+          ruff format --check src/ tests/
+
+      - name: Pre-flight fail_under ratchet check
+        if: ${{ matrix.os == 'ubuntu-latest' && matrix.python-version == '3.12' }}
+        env:
+          BASE_REF: ${{ github.event.pull_request.base.ref || 'main' }}
+        run: |
+          # Reject PRs that lower [tool.coverage.report] fail_under below
+          # the value on the base branch. The 95% ceiling is the maximum
+          # ratchet per ecosystem policy; this check enforces the
+          # one-way-only direction (down forbidden) so the gate cannot be
+          # weakened to land an under-tested change.
+          git fetch --no-tags --depth=1 origin "$BASE_REF" || true
+          python - <<'PY'
+          import os
+          import pathlib
+          import subprocess
+          import sys
+          import tomllib
+
+          base_ref = os.environ.get('BASE_REF', 'main')
+          out = subprocess.run(
+              ['git', 'show', f'origin/{base_ref}:pyproject.toml'],
+              capture_output=True,
+              text=True,
+              check=False,
+          )
+          if out.returncode != 0:
+              print(f'Could not read pyproject.toml from origin/{base_ref}; skipping check.')
+              sys.exit(0)
+
+          base = tomllib.loads(out.stdout)
+          head = tomllib.loads(pathlib.Path('pyproject.toml').read_text())
+
+          def fail_under(d):
+              try:
+                  return float(d['tool']['coverage']['report']['fail_under'])
+              except KeyError:
+                  return None
+
+          base_v = fail_under(base)
+          head_v = fail_under(head)
+          print(f'fail_under: base={base_v} head={head_v}')
+
+          # Base has no [tool.coverage.report].fail_under: nothing to
+          # ratchet against. The first PR that introduces the gate is
+          # allowed regardless of head_v.
+          if base_v is None:
+              print(
+                  f'origin/{base_ref} has no [tool.coverage.report].fail_under; '
+                  'no ratchet to enforce. PASS.'
+              )
+              sys.exit(0)
+
+          # Head removed the gate that base had: that's a decrease.
+          if head_v is None:
+              print(
+                  f'ERROR: head removed [tool.coverage.report].fail_under '
+                  f'(base had {base_v}). The gate is one-way only.'
+              )
+              sys.exit(1)
+
+          if head_v < base_v:
+              print(
+                  f'ERROR: fail_under decreased ({head_v} < {base_v}). '
+                  'The coverage gate is one-way only; ask a maintainer if a '
+                  'downward adjustment is required.'
+              )
+              sys.exit(1)
+          print('OK: fail_under is non-decreasing.')
+          PY
+
+      - name: Run unit tests
+        run: |
+          pytest -m "unit and not slow" -n auto -o "addopts=" \
+            -o "junit_family=legacy" --junitxml=junit.xml
+
+      - name: Upload test results to Codecov
+        if: ${{ !cancelled() && matrix.os == 'ubuntu-latest' && matrix.python-version == '3.12' }}
+        uses: codecov/test-results-action@v1
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          slug: FormingWorlds/Aragog
+          files: junit.xml

.github/workflows/docs.yaml

@@ -1,4 +1,4 @@
-name: Documentation
+name: Docs
 on:
   push:
     branches:

.github/workflows/nightly.yml

@@ -4,6 +4,10 @@ on:
   schedule:
     # 02:30 UTC daily (offset from Zalmoxis 02:00 UTC to spread runner load).
     - cron: "30 2 * * *"
+  # Refresh the canonical (full-suite) Codecov upload on every push to main
+  # so the badge tracks the merged state of the repo, not just the last cron.
+  push:
+    branches: [main]
   # Manual trigger for ad-hoc runs (e.g. before cutting a release, or to
   # verify the suite on a feature branch via
   # `gh workflow run "Aragog nightly" -r <branch>`).
@@ -15,36 +19,85 @@ on:
 permissions:
   contents: read
 
-# Nightly: full unit + slow suite with coverage instrumentation and the
-# 85% floor enforced via [tool.coverage.report].fail_under in pyproject.toml.
-# Excluded from push and PR CI because the slow tier alone takes ~110 s and
-# coverage instrumentation pushes the full run to ~10 min on a 2-vCPU
-# runner, over the 5 min budget for fast CI.
+# Nightly: full unit + smoke + slow suite with coverage instrumentation
+# and the 95% floor enforced via [tool.coverage.report].fail_under in
+# pyproject.toml. Excluded from push and PR CI because the slow tier
+# alone takes ~110 s and coverage instrumentation pushes the full run
+# to ~10 min on a 2-vCPU runner, over the 5 min budget for fast CI.
+#
+# `smoke` is included because the integration tests in
+# test_entropy_advanced.py, test_entropy_verification.py and
+# test_entropy_solver_integration.py exercise the production
+# EntropySolver.solve() path and the JAX RHS at full physical scale —
+# without them, entropy_solver.py / cvode_jax.py / jax/solver.py sit
+# at ~33 % / ~51 % / ~63 % coverage respectively.
+#
+# Uses micromamba + conda-forge for the env so SUNDIALS, gfortran, and the
+# scikits-odes binary stack come prebuilt.
 jobs:
   nightly:
     name: Aragog-nightly (ubuntu-latest)
     runs-on: ubuntu-latest
-    timeout-minutes: 30
+    # 90 min: full suite is ~12 min on a 12-core Mac Studio, ~55 min on
+    # the 2-vCPU runner with --quiet + xdist. Leaves headroom for JAX
+    # warmup, conda env cache misses, and the additional smoke tests
+    # added for the 95 % coverage gate.
+    timeout-minutes: 90
+    defaults:
+      run:
+        shell: bash -el {0}
     steps:
       - name: Checkout repository
         uses: actions/checkout@v5
         with:
           fetch-depth: 0
 
-      - name: Set up Python 3.12
-        uses: actions/setup-python@v5
+      - name: Set up conda env (sundials from conda-forge)
+        uses: mamba-org/setup-micromamba@v2
         with:
-          python-version: "3.12"
+          environment-name: aragog-nightly
+          create-args: >-
+            python=3.12
+            sundials
+          condarc: |
+            channels:
+              - conda-forge
 
-      - name: Install dependencies
+      - name: Install pip-only test deps (scikits-odes-sundials builds against conda SUNDIALS)
         run: |
-          python -m pip install --upgrade pip
-          pip install .
           pip install pytest pytest-cov pytest-xdist pytest-dependency
+          # Direct install of scikits-odes-sundials only; aragog imports
+          # CVODE directly from scikits_odes_sundials.cvode and skips the
+          # daepack-pulling metapackage. See ci_tests.yml for details.
+          # diffrax is the JAX-native ODE solver used by
+          # tests/test_jax_entropy.py::TestJAXSolverIntegration; without
+          # it those 8 smoke tests fail with ModuleNotFoundError.
+          pip install scikits-odes-sundials jax equinox diffrax
 
-      - name: Run unit + slow tests with coverage
+      - name: Install aragog
         run: |
-          pytest -m "unit or slow" -o "addopts=" \
+          pip install -e .
+
+      - name: Fetch SPIDER EOS test data
+        run: |
+          mkdir -p /tmp/aragog-test-data
+          curl -sL -o /tmp/spider_eos.tar.gz \
+            https://github.com/FormingWorlds/aragog/releases/download/test-data-v1/spider_eos_test_data.tar.gz
+          tar xzf /tmp/spider_eos.tar.gz -C /tmp/aragog-test-data/
+          echo "ARAGOG_TEST_EOS_DIR=/tmp/aragog-test-data/spider_eos" >> $GITHUB_ENV
+
+      - name: Run unit + smoke + slow tests with coverage
+        run: |
+          # -n auto runs xdist workers; coverage's parallel mode collects
+          # per-worker .coverage.* files which pytest-cov auto-combines
+          # before the report. Without xdist the serial run hit the
+          # 30-minute timeout at 23 % of the 271-test suite, dominated by
+          # JAX-traced functions where line tracing is the bottleneck.
+          # -ra prints a short summary of failed/error/skipped tests at
+          # the end of the run; without it --quiet suppresses the
+          # FAILED list entirely if there are no other final messages,
+          # making post-mortem on cancelled runs near-impossible.
+          pytest -m "unit or smoke or integration or slow" -n auto -o "addopts=" -ra \
             --cov=src/aragog --cov-report=term-missing --cov-report=xml
 
       - name: Upload coverage to Codecov