add numerical validation evidence

Author: CameronBrooks11

.github/workflows/numerical-validation-evidence.yml

@@ -0,0 +1,98 @@
+name: Numerical Validation Evidence
+
+on:
+  workflow_dispatch:
+    inputs:
+      duration_s:
+        description: Simulation duration in seconds
+        required: false
+        default: "10.0"
+        type: string
+      dt_values:
+        description: Comma-separated dt sweep
+        required: false
+        default: "0.4,0.2,0.1,0.05,0.025"
+        type: string
+      enforce_order:
+        description: Enforce convergence thresholds
+        required: false
+        default: true
+        type: boolean
+  schedule:
+    - cron: "30 7 * * 1"
+
+env:
+  VCPKG_COMMIT: "66c0373dc7fca549e5803087b9487edfe3aca0a1"
+
+jobs:
+  windows-validation:
+    name: Windows numerical validation evidence
+    runs-on: windows-2022
+    timeout-minutes: 45
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Setup vcpkg
+        uses: lukka/run-vcpkg@v11
+        with:
+          vcpkgGitCommitId: ${{ env.VCPKG_COMMIT }}
+          doNotCache: false
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Resolve validation options
+        id: opts
+        shell: pwsh
+        run: |
+          $duration = "10.0"
+          $dtValues = "0.4,0.2,0.1,0.05,0.025"
+          $enforce = "true"
+
+          if ("${{ github.event_name }}" -eq "workflow_dispatch") {
+            if ("${{ github.event.inputs.duration_s }}" -ne "") {
+              $duration = "${{ github.event.inputs.duration_s }}"
+            }
+            if ("${{ github.event.inputs.dt_values }}" -ne "") {
+              $dtValues = "${{ github.event.inputs.dt_values }}"
+            }
+            if ("${{ github.event.inputs.enforce_order }}" -eq "false") {
+              $enforce = "false"
+            }
+          }
+
+          $validationDir = "artifacts/validation/${{ github.run_id }}_${{ github.run_attempt }}_ci-windows-release"
+
+          "duration=$duration" >> $env:GITHUB_OUTPUT
+          "dt_values=$dtValues" >> $env:GITHUB_OUTPUT
+          "enforce=$enforce" >> $env:GITHUB_OUTPUT
+          "validation_dir=$validationDir" >> $env:GITHUB_OUTPUT
+
+      - name: Run numerical validation
+        shell: pwsh
+        run: |
+          $args = @(
+            "scripts/run_numerical_validation.py",
+            "--preset", "ci-windows-release",
+            "--config", "Release",
+            "--output-dir", "${{ steps.opts.outputs.validation_dir }}",
+            "--duration-s", "${{ steps.opts.outputs.duration }}",
+            "--dt-values", "${{ steps.opts.outputs.dt_values }}"
+          )
+
+          if ("${{ steps.opts.outputs.enforce }}" -eq "true") {
+            $args += "--enforce-order"
+          }
+
+          python @args
+
+      - name: Upload validation artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: numerical-validation-evidence-windows-${{ github.run_id }}-${{ github.run_attempt }}
+          path: ${{ steps.opts.outputs.validation_dir }}
+          if-no-files-found: error

docs/numerical-methods.md

@@ -5,6 +5,7 @@
 This document defines the numerical integration policy for FluxGraph models.
 It complements `docs/semantics_spec.md` and is authoritative for solver
 selection and stability interpretation.
+Validation protocol details are documented in `docs/validation-methodology.md`.
 
 ## Current Policy (Phase 3.1 baseline)
 
@@ -69,3 +70,7 @@ Windows example:
 ```powershell
 python .\scripts\run_numerical_validation.py --preset dev-windows-release --config Release --enforce-order
 ```
+
+CI evidence runs are produced by:
+
+1. `.github/workflows/numerical-validation-evidence.yml`

docs/validation-methodology.md

@@ -0,0 +1,94 @@
+# Numerical Validation Methodology
+
+## Purpose
+
+This document defines the validation protocol used to support Phase 3
+numerical claims for FluxGraph model integration behavior.
+
+## Systems Under Test
+
+Current validation scope targets `thermal_mass` with two canonical scenarios:
+
+1. `thermal.cooling.v1`
+2. `thermal.forced_response.v1`
+
+Each scenario is evaluated with:
+
+1. `forward_euler`
+2. `rk4`
+
+## Reference Solution
+
+Validation uses the analytical solution of the first-order linear thermal model:
+
+`dT/dt = (P - h*(T - T_amb)) / C`
+
+Closed-form references are used for both cooling and forced-response settings.
+
+## Error Metrics
+
+For each `(scenario, method, dt)` run:
+
+1. `L2` error over the sampled trajectory
+2. `Linf` error over the sampled trajectory
+3. Final-time absolute error
+
+## Convergence Estimation
+
+Observed order is estimated by linear regression on log-log scale:
+
+`log(error) = p * log(dt) + b`
+
+`p` is reported as:
+
+1. `observed_order_l2`
+2. `observed_order_linf`
+
+## Evidence Thresholds
+
+Current CI-enforced minima (`Linf`):
+
+1. `forward_euler >= 0.9`
+2. `rk4 >= 3.5`
+
+These thresholds are conservative guards. They are not intended as publication
+claims by themselves; publication claims should cite full artifact sets.
+
+## Reproducibility
+
+Local run:
+
+```bash
+python scripts/run_numerical_validation.py --preset dev-release --enforce-order
+```
+
+Windows run:
+
+```powershell
+python .\scripts\run_numerical_validation.py --preset dev-windows-release --config Release --enforce-order
+```
+
+CI run:
+
+1. GitHub Actions workflow: `.github/workflows/numerical-validation-evidence.yml`
+2. Output artifact directory: `artifacts/validation/<timestamp_or_runid>_*`
+
+Each evidence run includes:
+
+1. `validation_results.json`
+2. `validation_results.csv`
+3. `validation_evaluation.json`
+4. stdout/stderr and build/configure logs
+
+## Threats to Validity
+
+1. Current suite validates only one model family (`thermal_mass`).
+2. Convergence is measured on fixed `dt` sweeps and selected parameter regimes.
+3. Floating-point behavior may vary slightly across compilers/architectures.
+4. Threshold-based pass/fail does not replace full statistical or multi-platform analysis.
+
+## Next Extensions
+
+1. Add validation scenarios for additional models as they are introduced.
+2. Extend to transform stochastic validation (e.g., noise distribution checks).
+3. Add publication-grade plotting/notebook pipeline over stored CSV results.