Skip to content

Commit f87748b

Browse files
SebastianAmentSebastianAment
committed
explorer + ci: V2 strength GP web hookup + artifact / parity gates
Hooks the V2 strength GP up to the in-browser BOxCrete explorer and wires up the CI gates that enforce coherence between the deployed boxcrete model and the published model artifacts. JS-side V2 implementation (docs/explorer): * docs/gp_v2_fast.mjs — pure-JS implementation of the V2 gated-kernel posterior, optimised for the in-browser explorer. * docs/feature_registry.mjs — JS port of the F5_alllog feature builders to keep the JS path byte-identical to the Python path. * docs/gp.mjs / docs/ui.mjs / docs/units.mjs — explorer integration. * docs/generate_mix_analyses.py — produces per-mix analysis pages consumed by the explorer. * docs/model/README.md — documents the docs/model/ artifact layout. * docs/model/strength.json + compositions.json + test_vectors.json — refreshed model artifacts produced by the V2 fit. CI gates: * .github/workflows/strength-parity.yml — runs test_pretrained_loader_fidelity.py to guard that load_pretrained_strength_gp() faithfully reconstructs the deployed V2 strength GP from docs/model/strength_model.pt. * .github/workflows/model-artifacts-coherence.yml — guards that the published docs/model/ artifacts match the boxcrete fit output (catches stale artifacts after model code changes). Uses a cross-architecture-portable numerical drift check (experiments/check_artifacts_drift.py) instead of byte-level git diff. * .github/workflows/notebooks.yml + tests.yml — Python version bumps to match pyproject.toml's requires-python = ">=3.11" and BOXCRETE_SMOKE_TEST=1 env wiring for the notebook matrix. Regen pipeline (invoked by the artifacts-coherence gate): * experiments/regenerate_all_artifacts.sh — orchestrator. * experiments/regenerate_strength_json.py — Python regen of strength.json + test_vectors.json from a fresh V2 fit. * experiments/augment_test_vectors_with_gwp_cost.mjs — adds GWP / cost columns via the JS predictors. * experiments/regenerate_compositions_strength_predictions.mjs — regen of the static Pareto scatter dots. CI helpers: * experiments/check_artifacts_drift.py — numerical drift check. * experiments/run_notebook_with_progress.py — cell-by-cell timing helper used as a fast-fail step before nbconvert. * experiments/measure_fit_stability.py — empirical run-to-run fit determinism measurement (used during precision debugging). JS-side regression tests: * test/test_js_strength_v2.mjs — V2 posterior parity (Python vs JS). * test/test_js_physical_constraints.mjs — JS-side f(x, t=0) = 0 guard. * test/test_lengthscales_v2.mjs — JS-side lengthscale parity. * test/test_curve_monotonicity.mjs — strength-curve monotonicity. * test/test_data_freshness.mjs — guards that test_vectors.json is derived from the same data the deployed model was fit on. * test/test_js_ui_smoke.mjs — explorer UI smoke test.
1 parent 0319a0b commit f87748b

38 files changed

Lines changed: 5453 additions & 665 deletions

‎.github/workflows/e2e.yml‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,17 +4,23 @@
44

55
name: E2E (Playwright)
66

7+
# Cancel obsolete runs on rapid pushes (PR force-push storms otherwise
8+
# pile up redundant runs across all workflows).
9+
concurrency:
10+
group: ${{ github.workflow }}-${{ github.ref }}
11+
cancel-in-progress: true
12+
713
on:
814
push:
9-
branches: [main]
15+
branches: [main, master]
1016
paths:
1117
- 'docs/**'
1218
- 'test/e2e/**'
1319
- 'playwright.config.ts'
1420
- 'package.json'
1521
- '.github/workflows/e2e.yml'
1622
pull_request:
17-
branches: [main]
23+
branches: [main, master]
1824
paths:
1925
- 'docs/**'
2026
- 'test/e2e/**'
@@ -46,7 +52,7 @@ jobs:
4652
project: [desktop, mobile]
4753

4854
steps:
49-
- uses: actions/checkout@v4
55+
- uses: actions/checkout@v5
5056

5157
- uses: actions/setup-node@v4
5258
with:
@@ -58,7 +64,7 @@ jobs:
5864

5965
- name: Cache Playwright browsers
6066
id: pw-cache
61-
uses: actions/cache@v4
67+
uses: actions/cache@v5
6268
with:
6369
path: ~/.cache/ms-playwright
6470
key: playwright-${{ runner.os }}-${{ hashFiles('package-lock.json') }}

‎.github/workflows/js-sync.yml‎

Lines changed: 80 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,20 @@
1-
# Verifies that the JavaScript GP implementation matches Python reference predictions.
2-
# Catches drift if model parameters or JS code are changed without re-syncing.
1+
# Runs the full JS-side test suite that pairs with the Python boxcrete
2+
# package: Python ↔ JS predictor parity, V2 posterior parity, JS-side
3+
# physical-constraint guards, lengthscale parity, curve monotonicity,
4+
# data-freshness check, units, and explorer UI smoke.
5+
#
6+
# Catches drift between the Python fit (boxcrete) and the in-browser
7+
# JavaScript GP (docs/gp*.mjs + docs/model/*.json artifacts) — if the
8+
# model, JS code, or model artifacts change without re-syncing the JS
9+
# port and the test_vectors.json baseline, this workflow fails.
310

411
name: JS Model Sync
512

13+
# Cancel obsolete runs on rapid pushes.
14+
concurrency:
15+
group: ${{ github.workflow }}-${{ github.ref }}
16+
cancel-in-progress: true
17+
618
# Restrict GITHUB_TOKEN to the minimum needed: read-only access to repo
719
# contents (required by actions/checkout). The workflow never writes to the
820
# repo, comments on PRs, deploys, or interacts with any other GitHub APIs.
@@ -14,14 +26,48 @@ on:
1426
branches: [main, master]
1527
paths:
1628
- 'docs/gp.mjs'
29+
- 'docs/gp_v2_fast.mjs'
30+
- 'docs/feature_registry.mjs'
31+
- 'docs/units.mjs'
1732
- 'docs/model/**'
33+
- 'boxcrete/features.py'
34+
- 'package.json'
35+
- 'package-lock.json'
1836
- 'test/test_js_gp.mjs'
37+
- 'test/test_js_strength_v2.mjs'
38+
- 'test/test_js_predictor_parity.mjs'
39+
- 'test/test_js_physical_constraints.mjs'
40+
- 'test/test_js_ui_smoke.mjs'
41+
- 'test/test_js_units.mjs'
42+
- 'test/test_lengthscales_v2.mjs'
43+
- 'test/test_js_feature_parity.mjs'
44+
- 'test/fixtures/feature_parity_fixture.json'
45+
- 'test/test_curve_monotonicity.mjs'
46+
- 'test/test_data_freshness.mjs'
47+
- '.github/workflows/js-sync.yml'
1948
pull_request:
2049
branches: [main, master]
2150
paths:
2251
- 'docs/gp.mjs'
52+
- 'docs/gp_v2_fast.mjs'
53+
- 'docs/feature_registry.mjs'
54+
- 'docs/units.mjs'
2355
- 'docs/model/**'
56+
- 'boxcrete/features.py'
57+
- 'package.json'
58+
- 'package-lock.json'
2459
- 'test/test_js_gp.mjs'
60+
- 'test/test_js_strength_v2.mjs'
61+
- 'test/test_js_predictor_parity.mjs'
62+
- 'test/test_js_physical_constraints.mjs'
63+
- 'test/test_js_ui_smoke.mjs'
64+
- 'test/test_js_units.mjs'
65+
- 'test/test_lengthscales_v2.mjs'
66+
- 'test/test_js_feature_parity.mjs'
67+
- 'test/fixtures/feature_parity_fixture.json'
68+
- 'test/test_curve_monotonicity.mjs'
69+
- 'test/test_data_freshness.mjs'
70+
- '.github/workflows/js-sync.yml'
2571

2672
env:
2773
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
@@ -30,11 +76,41 @@ jobs:
3076
js-model-sync:
3177
runs-on: ubuntu-latest
3278
steps:
33-
- uses: actions/checkout@v4
79+
- uses: actions/checkout@v5
3480

3581
- uses: actions/setup-node@v4
3682
with:
3783
node-version: '20'
3884

39-
- name: Run JS GP sync test
85+
# Run each JS test individually rather than via a globbing wrapper
86+
# so a single test failure surfaces with its own step in the CI UI
87+
# (easy to skim "which test broke" without scrolling logs).
88+
- name: Python ↔ JS GP equivalence (test_vectors.json)
4089
run: node test/test_js_gp.mjs
90+
91+
- name: JS ↔ Python feature builder parity
92+
run: node test/test_js_feature_parity.mjs
93+
94+
- name: V2 posterior parity (Python vs JS)
95+
run: node test/test_js_strength_v2.mjs
96+
97+
- name: Single-vs-batch JS predictor contract
98+
run: node test/test_js_predictor_parity.mjs
99+
100+
- name: JS-side f(x, t=0) = 0 physical-constraint guard
101+
run: node test/test_js_physical_constraints.mjs
102+
103+
- name: Explorer UI smoke
104+
run: node test/test_js_ui_smoke.mjs
105+
106+
- name: JS unit conversions
107+
run: node test/test_js_units.mjs
108+
109+
- name: JS-side lengthscale parity / identifiability
110+
run: node test/test_lengthscales_v2.mjs
111+
112+
- name: JS-side strength-curve monotonicity
113+
run: node test/test_curve_monotonicity.mjs
114+
115+
- name: Data-freshness (compositions.json vs live JS GP)
116+
run: node test/test_data_freshness.mjs

‎.github/workflows/lighthouse.yml‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,20 @@
44

55
name: Lighthouse CI
66

7+
# Cancel obsolete runs on rapid pushes.
8+
concurrency:
9+
group: ${{ github.workflow }}-${{ github.ref }}
10+
cancel-in-progress: true
11+
712
on:
813
push:
9-
branches: [main]
14+
branches: [main, master]
1015
paths:
1116
- 'docs/**'
1217
- 'lighthouserc.json'
1318
- '.github/workflows/lighthouse.yml'
1419
pull_request:
15-
branches: [main]
20+
branches: [main, master]
1621
paths:
1722
- 'docs/**'
1823
- 'lighthouserc.json'
@@ -31,7 +36,7 @@ jobs:
3136
timeout-minutes: 15
3237

3338
steps:
34-
- uses: actions/checkout@v4
39+
- uses: actions/checkout@v5
3540

3641
- uses: actions/setup-node@v4
3742
with:

‎.github/workflows/model-artifacts-coherence.yml‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
# Verifies that docs/model/*.json artifacts are coherent with the
2+
# Python V2 strength GP fit. Runs the regen pipeline and asserts no
3+
# diff — any uncommitted drift in strength.json / test_vectors.json /
4+
# compositions.json (e.g., from a manual edit, a partial regen, or
5+
# upstream model code that wasn't followed by a regen run) fails CI.
6+
#
7+
# See docs/model/README.md for the artifact schema + regen workflow.
8+
9+
name: Model Artifacts Coherence
10+
11+
# Cancel obsolete runs on rapid pushes.
12+
concurrency:
13+
group: ${{ github.workflow }}-${{ github.ref }}
14+
cancel-in-progress: true
15+
16+
permissions:
17+
contents: read
18+
19+
on:
20+
push:
21+
branches: [main, master]
22+
paths:
23+
- 'boxcrete/strength_model.py'
24+
- 'boxcrete/kernels.py'
25+
- 'boxcrete/likelihoods.py'
26+
- 'boxcrete/priors.py'
27+
- 'boxcrete/features.py'
28+
- 'boxcrete/utils.py'
29+
- 'boxcrete/__init__.py'
30+
- 'data/**'
31+
- 'experiments/regenerate_strength_json.py'
32+
- 'experiments/regenerate_compositions_strength_predictions.mjs'
33+
- 'experiments/augment_test_vectors_with_gwp_cost.mjs'
34+
- 'experiments/regenerate_all_artifacts.sh'
35+
- 'docs/model/**'
36+
- 'docs/feature_registry.mjs'
37+
- 'docs/gp.mjs'
38+
- 'docs/gp_v2_fast.mjs'
39+
- '.github/workflows/model-artifacts-coherence.yml'
40+
pull_request:
41+
branches: [main, master]
42+
paths:
43+
- 'boxcrete/strength_model.py'
44+
- 'boxcrete/kernels.py'
45+
- 'boxcrete/likelihoods.py'
46+
- 'boxcrete/priors.py'
47+
- 'boxcrete/features.py'
48+
- 'boxcrete/utils.py'
49+
- 'boxcrete/__init__.py'
50+
- 'data/**'
51+
- 'experiments/regenerate_strength_json.py'
52+
- 'experiments/regenerate_compositions_strength_predictions.mjs'
53+
- 'experiments/augment_test_vectors_with_gwp_cost.mjs'
54+
- 'experiments/regenerate_all_artifacts.sh'
55+
- 'docs/model/**'
56+
- 'docs/feature_registry.mjs'
57+
- 'docs/gp.mjs'
58+
- 'docs/gp_v2_fast.mjs'
59+
- '.github/workflows/model-artifacts-coherence.yml'
60+
workflow_dispatch: {}
61+
62+
jobs:
63+
regen-idempotency:
64+
runs-on: ubuntu-latest
65+
steps:
66+
- uses: actions/checkout@v5
67+
68+
- uses: actions/setup-python@v6
69+
with:
70+
python-version: '3.12'
71+
cache: 'pip'
72+
73+
- uses: actions/setup-node@v4
74+
with:
75+
node-version: '20'
76+
77+
- name: Install Python deps
78+
run: |
79+
python -m pip install --upgrade pip
80+
pip install -e .
81+
82+
- name: Save committed docs/model/ for later comparison
83+
# Snapshot the JSON artifacts BEFORE regen overwrites them, so
84+
# the post-regen comparison can diff against the committed copy.
85+
# We snapshot only the JSONs (not the .pt) because cross-arch
86+
# determinism on binary state_dicts requires bit-equality which
87+
# we can't expect from a multi-modal MLL fit; the JSON-level
88+
# checks cover the same coverage surface (lengthscales +
89+
# prediction surface) via experiments/check_artifacts_drift.py.
90+
run: |
91+
mkdir -p /tmp/committed_docs_model
92+
cp docs/model/strength.json /tmp/committed_docs_model/
93+
cp docs/model/test_vectors.json /tmp/committed_docs_model/
94+
cp docs/model/compositions.json /tmp/committed_docs_model/
95+
96+
- name: Run regen pipeline
97+
run: bash experiments/regenerate_all_artifacts.sh
98+
99+
- name: Assert artifacts agree with committed copy within tolerance
100+
# Replaces the legacy ``git diff --exit-code docs/model/`` check,
101+
# which was over-strict: it required bit-equality of JSON output
102+
# across architectures, but the V2 strength GP fit goes through
103+
# scipy's L-BFGS-B against a multi-modal MLL surface, and
104+
# different CPU architectures land in different local optima
105+
# (Apple Silicon via qemu-emulated amd64 vs GitHub-runner native
106+
# x86_64 produce ~2x different lengthscales while predicting
107+
# nearly the same surface). The new check tolerates this
108+
# cross-architecture basin divergence (10x ratio band on
109+
# internal hyperparameters) while still catching the original
110+
# failure mode (a stale export typically shifts predictions
111+
# by 100s of psi at OOT compositions). See the docstring of
112+
# ``experiments/check_artifacts_drift.py`` for the full rationale.
113+
run: |
114+
python experiments/check_artifacts_drift.py \
115+
--committed-dir /tmp/committed_docs_model \
116+
--fresh-dir docs/model

0 commit comments

Comments
 (0)