Skip to content

Commit faa857e

Browse files
SebastianAmentSebastianAment
committed
explorer + ci: V2 strength GP web hookup + artifact / parity gates
Hooks the V2 strength GP up to the in-browser BOxCrete explorer and wires up the CI gates that enforce coherence between the deployed boxcrete model and the published model artifacts. JS-side V2 implementation (docs/explorer): * docs/gp_v2_fast.mjs — pure-JS implementation of the V2 gated-kernel posterior, optimised for the in-browser explorer. * docs/feature_registry.mjs — JS port of the F5_alllog feature builders to keep the JS path byte-identical to the Python path. * docs/gp.mjs / docs/ui.mjs / docs/units.mjs — explorer integration. * docs/generate_mix_analyses.py — produces per-mix analysis pages consumed by the explorer. * docs/model/README.md — documents the docs/model/ artifact layout. * docs/model/strength.json + compositions.json + test_vectors.json — refreshed model artifacts produced by the V2 fit. CI gates: * .github/workflows/strength-parity.yml — runs test_pretrained_loader_fidelity.py to guard that load_pretrained_strength_gp() faithfully reconstructs the deployed V2 strength GP from docs/model/strength_model.pt. * .github/workflows/model-artifacts-coherence.yml — guards that the published docs/model/ artifacts match the boxcrete fit output (catches stale artifacts after model code changes). Uses a cross-architecture-portable numerical drift check (experiments/check_artifacts_drift.py) instead of byte-level git diff. * .github/workflows/notebooks.yml + tests.yml — Python version bumps to match pyproject.toml's requires-python = ">=3.11" and BOXCRETE_SMOKE_TEST=1 env wiring for the notebook matrix. Regen pipeline (invoked by the artifacts-coherence gate): * experiments/regenerate_all_artifacts.sh — orchestrator. * experiments/regenerate_strength_json.py — Python regen of strength.json + test_vectors.json from a fresh V2 fit. * experiments/augment_test_vectors_with_gwp_cost.mjs — adds GWP / cost columns via the JS predictors. * experiments/regenerate_compositions_strength_predictions.mjs — regen of the static Pareto scatter dots. CI helpers: * experiments/check_artifacts_drift.py — numerical drift check. * experiments/run_notebook_with_progress.py — cell-by-cell timing helper used as a fast-fail step before nbconvert. * experiments/measure_fit_stability.py — empirical run-to-run fit determinism measurement (used during precision debugging). JS-side regression tests: * test/test_js_strength_v2.mjs — V2 posterior parity (Python vs JS). * test/test_js_physical_constraints.mjs — JS-side f(x, t=0) = 0 guard. * test/test_lengthscales_v2.mjs — JS-side lengthscale parity. * test/test_curve_monotonicity.mjs — strength-curve monotonicity. * test/test_data_freshness.mjs — guards that test_vectors.json is derived from the same data the deployed model was fit on. * test/test_js_ui_smoke.mjs — explorer UI smoke test.
1 parent 580d693 commit faa857e

36 files changed

Lines changed: 5055 additions & 581 deletions

‎.github/workflows/e2e.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ jobs:
4646
project: [desktop, mobile]
4747

4848
steps:
49-
- uses: actions/checkout@v4
49+
- uses: actions/checkout@v5
5050

5151
- uses: actions/setup-node@v4
5252
with:
@@ -58,7 +58,7 @@ jobs:
5858

5959
- name: Cache Playwright browsers
6060
id: pw-cache
61-
uses: actions/cache@v4
61+
uses: actions/cache@v5
6262
with:
6363
path: ~/.cache/ms-playwright
6464
key: playwright-${{ runner.os }}-${{ hashFiles('package-lock.json') }}

‎.github/workflows/js-sync.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ jobs:
3030
js-model-sync:
3131
runs-on: ubuntu-latest
3232
steps:
33-
- uses: actions/checkout@v4
33+
- uses: actions/checkout@v5
3434

3535
- uses: actions/setup-node@v4
3636
with:

‎.github/workflows/lighthouse.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ jobs:
3131
timeout-minutes: 15
3232

3333
steps:
34-
- uses: actions/checkout@v4
34+
- uses: actions/checkout@v5
3535

3636
- uses: actions/setup-node@v4
3737
with:

‎.github/workflows/model-artifacts-coherence.yml‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# Verifies that docs/model/*.json artifacts are coherent with the
2+
# Python V2 strength GP fit. Runs the regen pipeline and asserts no
3+
# diff — any uncommitted drift in strength.json / test_vectors.json /
4+
# compositions.json (e.g., from a manual edit, a partial regen, or
5+
# upstream model code that wasn't followed by a regen run) fails CI.
6+
#
7+
# See docs/model/README.md for the artifact schema + regen workflow.
8+
9+
name: Model Artifacts Coherence
10+
11+
permissions:
12+
contents: read
13+
14+
on:
15+
push:
16+
branches: [main, master]
17+
paths:
18+
- 'boxcrete/strength_model.py'
19+
- 'boxcrete/kernels.py'
20+
- 'boxcrete/likelihoods.py'
21+
- 'boxcrete/priors.py'
22+
- 'boxcrete/features.py'
23+
- 'boxcrete/utils.py'
24+
- 'boxcrete/__init__.py'
25+
- 'data/**'
26+
- 'experiments/regenerate_strength_json.py'
27+
- 'experiments/regenerate_compositions_strength_predictions.mjs'
28+
- 'experiments/augment_test_vectors_with_gwp_cost.mjs'
29+
- 'experiments/regenerate_all_artifacts.sh'
30+
- 'docs/model/**'
31+
- 'docs/feature_registry.mjs'
32+
- 'docs/gp.mjs'
33+
- 'docs/gp_v2_fast.mjs'
34+
- '.github/workflows/model-artifacts-coherence.yml'
35+
pull_request:
36+
branches: [main, master]
37+
paths:
38+
- 'boxcrete/strength_model.py'
39+
- 'boxcrete/kernels.py'
40+
- 'boxcrete/likelihoods.py'
41+
- 'boxcrete/priors.py'
42+
- 'boxcrete/features.py'
43+
- 'boxcrete/utils.py'
44+
- 'boxcrete/__init__.py'
45+
- 'data/**'
46+
- 'experiments/regenerate_strength_json.py'
47+
- 'experiments/regenerate_compositions_strength_predictions.mjs'
48+
- 'experiments/augment_test_vectors_with_gwp_cost.mjs'
49+
- 'experiments/regenerate_all_artifacts.sh'
50+
- 'docs/model/**'
51+
- 'docs/feature_registry.mjs'
52+
- 'docs/gp.mjs'
53+
- 'docs/gp_v2_fast.mjs'
54+
- '.github/workflows/model-artifacts-coherence.yml'
55+
workflow_dispatch: {}
56+
57+
jobs:
58+
regen-idempotency:
59+
runs-on: ubuntu-latest
60+
steps:
61+
- uses: actions/checkout@v5
62+
63+
- uses: actions/setup-python@v6
64+
with:
65+
python-version: '3.12'
66+
cache: 'pip'
67+
68+
- uses: actions/setup-node@v4
69+
with:
70+
node-version: '20'
71+
72+
- name: Install Python deps
73+
run: |
74+
python -m pip install --upgrade pip
75+
pip install -e .
76+
77+
- name: Save committed docs/model/ for later comparison
78+
# Snapshot the JSON artifacts BEFORE regen overwrites them, so
79+
# the post-regen comparison can diff against the committed copy.
80+
# We snapshot only the JSONs (not the .pt) because cross-arch
81+
# determinism on binary state_dicts requires bit-equality which
82+
# we can't expect from a multi-modal MLL fit; the JSON-level
83+
# checks cover the same coverage surface (lengthscales +
84+
# prediction surface) via experiments/check_artifacts_drift.py.
85+
run: |
86+
mkdir -p /tmp/committed_docs_model
87+
cp docs/model/strength.json /tmp/committed_docs_model/
88+
cp docs/model/test_vectors.json /tmp/committed_docs_model/
89+
cp docs/model/compositions.json /tmp/committed_docs_model/
90+
91+
- name: Run regen pipeline
92+
run: bash experiments/regenerate_all_artifacts.sh
93+
94+
- name: Assert artifacts agree with committed copy within tolerance
95+
# Replaces the legacy ``git diff --exit-code docs/model/`` check,
96+
# which was over-strict: it required bit-equality of JSON output
97+
# across architectures, but the V2 strength GP fit goes through
98+
# scipy's L-BFGS-B against a multi-modal MLL surface, and
99+
# different CPU architectures land in different local optima
100+
# (Apple Silicon via qemu-emulated amd64 vs GitHub-runner native
101+
# x86_64 produce ~2x different lengthscales while predicting
102+
# nearly the same surface). The new check tolerates this
103+
# cross-architecture basin divergence (10x ratio band on
104+
# internal hyperparameters) while still catching the original
105+
# failure mode (a stale export typically shifts predictions
106+
# by 100s of psi at OOT compositions). See the docstring of
107+
# ``experiments/check_artifacts_drift.py`` for the full rationale.
108+
run: |
109+
python experiments/check_artifacts_drift.py \
110+
--committed-dir /tmp/committed_docs_model \
111+
--fresh-dir docs/model

‎.github/workflows/notebooks.yml‎

Lines changed: 48 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -41,15 +41,15 @@ jobs:
4141

4242
steps:
4343
- name: Checkout repository
44-
uses: actions/checkout@v4
44+
uses: actions/checkout@v5
4545

4646
- name: Set up Python 3.10
47-
uses: actions/setup-python@v5
47+
uses: actions/setup-python@v6
4848
with:
49-
python-version: '3.10'
49+
python-version: '3.12'
5050

5151
- name: Cache pip dependencies
52-
uses: actions/cache@v4
52+
uses: actions/cache@v5
5353
with:
5454
path: ~/.cache/pip
5555
key: ${{ runner.os }}-pip-3.10-${{ hashFiles('pyproject.toml') }}
@@ -66,59 +66,48 @@ jobs:
6666
run: |
6767
python -m ipykernel install --user --name python3
6868
69-
- name: Execute mode-dependent notebooks (${{ matrix.optimization-mode }}, cost=${{ matrix.include-cost }})
69+
- name: Notebook fast-fail check (cell-by-cell timing)
70+
# Run every code cell of the heavy notebook sequentially as
71+
# plain Python (no Jupyter kernel) with per-cell timing
72+
# markers. This step is a debugging aid: if a cell hangs,
73+
# the action log will show the last `=== cell N: ... ===`
74+
# marker before whatever runner-level cancellation fires,
75+
# pinpointing the offending cell. If this step PASSES, the
76+
# subsequent nbconvert step is highly likely to pass too.
77+
# If it FAILS, it fails fast with the cell number, source,
78+
# and traceback already in the log — no more digging.
79+
# See experiments/run_notebook_with_progress.py docstring
80+
# for full background and local-debugging usage.
7081
env:
7182
BOXCRETE_OPTIMIZATION_MODE: ${{ matrix.optimization-mode }}
7283
BOXCRETE_INCLUDE_COST: ${{ matrix.include-cost }}
84+
BOXCRETE_SMOKE_TEST: "1"
7385
run: |
74-
python - << 'PYEOF'
75-
import subprocess, sys, os
76-
from pathlib import Path
77-
78-
# Only notebooks that use BOXCRETE_OPTIMIZATION_MODE
79-
MODE_DEPENDENT = [
80-
"notebooks/prediction_and_optimization_tutorial.ipynb",
81-
]
82-
83-
mode = os.environ.get("BOXCRETE_OPTIMIZATION_MODE", "concrete")
84-
failed = []
85-
86-
for nb in MODE_DEPENDENT:
87-
nb_path = Path(nb)
88-
if not nb_path.exists():
89-
print(f"⚠️ Skipping (not found): {nb}")
90-
continue
91-
92-
print(f"{'=' * 40}")
93-
print(f"Executing ({mode}): {nb}")
94-
print(f"{'=' * 40}")
95-
96-
result = subprocess.run(
97-
[
98-
sys.executable, "-m", "jupyter", "nbconvert",
99-
"--to", "notebook",
100-
"--execute",
101-
"--inplace",
102-
"--ExecutePreprocessor.timeout=600",
103-
"--ExecutePreprocessor.kernel_name=python3",
104-
str(nb),
105-
],
106-
capture_output=False,
107-
)
108-
109-
if result.returncode != 0:
110-
failed.append(str(nb))
111-
print(f"❌ Failed ({mode}): {nb}")
112-
else:
113-
print(f"✅ Successfully executed ({mode}): {nb}")
114-
print()
86+
python experiments/run_notebook_with_progress.py \
87+
notebooks/prediction_and_optimization_tutorial.ipynb \
88+
--per-cell-timeout 180
11589
116-
if failed:
117-
print(f"\n{len(failed)} notebook(s) failed in {mode} mode:")
118-
for f in failed:
119-
print(f" - {f}")
120-
sys.exit(1)
121-
PYEOF
90+
- name: Execute mode-dependent notebooks (${{ matrix.optimization-mode }}, cost=${{ matrix.include-cost }})
91+
# nbconvert produces the rendered .ipynb artifact (cells + plots
92+
# embedded as base64) for the upload-artifact step below. The
93+
# cell-by-cell fast-fail check above runs first and catches
94+
# hangs / cell errors with clear progress markers, so we don't
95+
# need the diagnostic wrapper that previously surrounded this
96+
# step (free -h / dmesg / signal-name parsing). If a cell ever
97+
# hangs again, the cell-by-cell step will surface it before we
98+
# reach this nbconvert call.
99+
env:
100+
BOXCRETE_OPTIMIZATION_MODE: ${{ matrix.optimization-mode }}
101+
BOXCRETE_INCLUDE_COST: ${{ matrix.include-cost }}
102+
BOXCRETE_SMOKE_TEST: "1"
103+
run: |
104+
python -m jupyter nbconvert \
105+
--to notebook \
106+
--execute \
107+
--inplace \
108+
--ExecutePreprocessor.timeout=600 \
109+
--ExecutePreprocessor.kernel_name=python3 \
110+
notebooks/prediction_and_optimization_tutorial.ipynb
122111
123112
- name: Upload executed notebooks as artifacts
124113
uses: actions/upload-artifact@v4
@@ -139,15 +128,15 @@ jobs:
139128

140129
steps:
141130
- name: Checkout repository
142-
uses: actions/checkout@v4
131+
uses: actions/checkout@v5
143132

144133
- name: Set up Python 3.10
145-
uses: actions/setup-python@v5
134+
uses: actions/setup-python@v6
146135
with:
147-
python-version: '3.10'
136+
python-version: '3.12'
148137

149138
- name: Cache pip dependencies
150-
uses: actions/cache@v4
139+
uses: actions/cache@v5
151140
with:
152141
path: ~/.cache/pip
153142
key: ${{ runner.os }}-pip-3.10-${{ hashFiles('pyproject.toml') }}
@@ -227,12 +216,12 @@ jobs:
227216
runs-on: ubuntu-latest
228217
steps:
229218
- name: Checkout repository
230-
uses: actions/checkout@v4
219+
uses: actions/checkout@v5
231220

232221
- name: Set up Python
233-
uses: actions/setup-python@v5
222+
uses: actions/setup-python@v6
234223
with:
235-
python-version: '3.10'
224+
python-version: '3.12'
236225

237226
- name: Install dependencies
238227
run: |

‎.github/workflows/strength-parity.yml‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Verifies that ``boxcrete.load_pretrained_strength_gp()`` faithfully
2+
# reconstructs the deployed V2 strength GP from
3+
# ``docs/model/strength_model.pt``.
4+
#
5+
# Runs:
6+
# - test/test_pretrained_loader_fidelity.py (atol=1e-5 / 1e-3, ~3s)
7+
#
8+
# See that test file for the rationale.
9+
10+
name: Strength GP Parity
11+
12+
permissions:
13+
contents: read
14+
15+
on:
16+
push:
17+
branches: [main, master]
18+
paths:
19+
- 'boxcrete/strength_model.py'
20+
- 'boxcrete/kernels.py'
21+
- 'boxcrete/likelihoods.py'
22+
- 'boxcrete/priors.py'
23+
- 'boxcrete/features.py'
24+
- 'boxcrete/__init__.py'
25+
- 'test/test_pretrained_loader_fidelity.py'
26+
- 'docs/model/strength.json'
27+
- 'docs/model/strength_model.pt'
28+
- 'docs/model/test_vectors.json'
29+
- '.github/workflows/strength-parity.yml'
30+
pull_request:
31+
branches: [main, master]
32+
paths:
33+
- 'boxcrete/strength_model.py'
34+
- 'boxcrete/kernels.py'
35+
- 'boxcrete/likelihoods.py'
36+
- 'boxcrete/priors.py'
37+
- 'boxcrete/features.py'
38+
- 'boxcrete/__init__.py'
39+
- 'test/test_pretrained_loader_fidelity.py'
40+
- 'docs/model/strength.json'
41+
- 'docs/model/strength_model.pt'
42+
- 'docs/model/test_vectors.json'
43+
- '.github/workflows/strength-parity.yml'
44+
workflow_dispatch: {}
45+
46+
jobs:
47+
parity:
48+
runs-on: ubuntu-latest
49+
steps:
50+
- uses: actions/checkout@v5
51+
52+
- uses: actions/setup-python@v6
53+
with:
54+
python-version: '3.12'
55+
cache: 'pip'
56+
57+
- name: Install Python deps
58+
run: |
59+
python -m pip install --upgrade pip
60+
pip install -e .
61+
pip install pytest
62+
63+
- name: Run pretrained-loader fidelity test
64+
run: python -m pytest test/test_pretrained_loader_fidelity.py -v --tb=short

0 commit comments

Comments
 (0)