Skip to content

Commit ee6b671

Browse files
explorer + ci: V2 strength GP web hookup + artifact / parity gates
Hooks the V2 strength GP up to the in-browser BOxCrete explorer and wires up the CI gates that enforce coherence between the deployed boxcrete model and the published model artifacts. JS-side V2 implementation (docs/explorer): * docs/gp_v2_fast.mjs — pure-JS implementation of the V2 gated-kernel posterior, optimised for the in-browser explorer. * docs/feature_registry.mjs — JS port of the F5_alllog feature builders to keep the JS path byte-identical to the Python path. * docs/gp.mjs / docs/ui.mjs / docs/units.mjs — explorer integration. * docs/generate_mix_analyses.py — produces per-mix analysis pages consumed by the explorer. * docs/model/README.md — documents the docs/model/ artifact layout. * docs/model/strength.json + compositions.json + test_vectors.json — refreshed model artifacts produced by the V2 fit. CI gates: * .github/workflows/strength-parity.yml — guards that the V2 fit factory produces byte-equivalent posteriors to the research-side catalog (test_strength_model_parity.py). * .github/workflows/model-artifacts-coherence.yml — guards that the published docs/model/ artifacts match the boxcrete fit output (catches stale artifacts after model code changes). JS-side regression tests: * test/test_js_strength_v2.mjs — V2 posterior parity (Python vs JS). * test/test_js_physical_constraints.mjs — JS-side f(x, t=0) = 0 guard. * test/test_lengthscales_v2.mjs — JS-side lengthscale parity. * test/test_curve_monotonicity.mjs — strength-curve monotonicity. * test/test_data_freshness.mjs — guards that test_vectors.json is derived from the same data the deployed model was fit on. * test/test_js_ui_smoke.mjs — explorer UI smoke test. * experiments/regenerate_compositions_strength_predictions.mjs + augment_test_vectors_with_gwp_cost.mjs — JS-side artifact regeneration scripts.
1 parent f9c1d55 commit ee6b671

26 files changed

Lines changed: 3665 additions & 509 deletions
Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
# Verifies that docs/model/*.json artifacts are coherent with the
2+
# Python V2 strength GP fit. Runs the regen pipeline and asserts no
3+
# diff — any uncommitted drift in strength.json / test_vectors.json /
4+
# compositions.json (e.g., from a manual edit, a partial regen, or
5+
# upstream model code that wasn't followed by a regen run) fails CI.
6+
#
7+
# See docs/model/README.md for the artifact schema + regen workflow.
8+
9+
name: Model Artifacts Coherence
10+
11+
permissions:
12+
contents: read
13+
14+
on:
15+
push:
16+
branches: [main, master]
17+
paths:
18+
- 'boxcrete/strength_model.py'
19+
- 'boxcrete/kernels.py'
20+
- 'boxcrete/likelihoods.py'
21+
- 'boxcrete/priors.py'
22+
- 'boxcrete/features.py'
23+
- 'boxcrete/utils.py'
24+
- 'boxcrete/__init__.py'
25+
- 'data/**'
26+
- 'experiments/regenerate_strength_json.py'
27+
- 'experiments/regenerate_compositions_strength_predictions.mjs'
28+
- 'experiments/augment_test_vectors_with_gwp_cost.mjs'
29+
- 'experiments/regenerate_all_artifacts.sh'
30+
- 'docs/model/**'
31+
- 'docs/feature_registry.mjs'
32+
- 'docs/gp.mjs'
33+
- 'docs/gp_v2_fast.mjs'
34+
- '.github/workflows/model-artifacts-coherence.yml'
35+
pull_request:
36+
branches: [main, master]
37+
paths:
38+
- 'boxcrete/strength_model.py'
39+
- 'boxcrete/kernels.py'
40+
- 'boxcrete/likelihoods.py'
41+
- 'boxcrete/priors.py'
42+
- 'boxcrete/features.py'
43+
- 'boxcrete/utils.py'
44+
- 'boxcrete/__init__.py'
45+
- 'data/**'
46+
- 'experiments/regenerate_strength_json.py'
47+
- 'experiments/regenerate_compositions_strength_predictions.mjs'
48+
- 'experiments/augment_test_vectors_with_gwp_cost.mjs'
49+
- 'experiments/regenerate_all_artifacts.sh'
50+
- 'docs/model/**'
51+
- 'docs/feature_registry.mjs'
52+
- 'docs/gp.mjs'
53+
- 'docs/gp_v2_fast.mjs'
54+
- '.github/workflows/model-artifacts-coherence.yml'
55+
workflow_dispatch: {}
56+
57+
jobs:
58+
regen-idempotency:
59+
runs-on: ubuntu-latest
60+
steps:
61+
- uses: actions/checkout@v4
62+
63+
- uses: actions/setup-python@v5
64+
with:
65+
python-version: '3.12'
66+
cache: 'pip'
67+
68+
- uses: actions/setup-node@v4
69+
with:
70+
node-version: '20'
71+
72+
- name: Install Python deps
73+
run: |
74+
python -m pip install --upgrade pip
75+
pip install -e .
76+
77+
- name: Run regen pipeline
78+
run: bash experiments/regenerate_all_artifacts.sh
79+
80+
- name: Assert artifacts are byte-identical to committed copy
81+
run: |
82+
if ! git diff --exit-code docs/model/; then
83+
echo ""
84+
echo "::error::docs/model/*.json drifted after running"
85+
echo "::error::experiments/regenerate_all_artifacts.sh."
86+
echo "::error::This usually means one of:"
87+
echo "::error:: 1. docs/model/*.json was manually edited (don't do that)."
88+
echo "::error:: 2. boxcrete production code changed but the artifacts"
89+
echo "::error:: weren't regenerated. Run:"
90+
echo "::error:: bash experiments/regenerate_all_artifacts.sh"
91+
echo "::error:: and commit the resulting docs/model/ changes."
92+
echo "::error:: 3. Non-determinism in the V2 strength GP fit (should not"
93+
echo "::error:: happen with seed=0; if it does, that's a real bug)."
94+
echo "::error::See docs/model/README.md for the full regen workflow."
95+
exit 1
96+
fi

.github/workflows/notebooks.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ jobs:
4646
- name: Set up Python 3.10
4747
uses: actions/setup-python@v5
4848
with:
49-
python-version: '3.10'
49+
python-version: '3.12'
5050

5151
- name: Cache pip dependencies
5252
uses: actions/cache@v4
@@ -144,7 +144,7 @@ jobs:
144144
- name: Set up Python 3.10
145145
uses: actions/setup-python@v5
146146
with:
147-
python-version: '3.10'
147+
python-version: '3.12'
148148

149149
- name: Cache pip dependencies
150150
uses: actions/cache@v4
@@ -232,7 +232,7 @@ jobs:
232232
- name: Set up Python
233233
uses: actions/setup-python@v5
234234
with:
235-
python-version: '3.10'
235+
python-version: '3.12'
236236

237237
- name: Install dependencies
238238
run: |
Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
# Verifies that boxcrete.fit_strength_gp (production) and the
2+
# experiments.model_variant_study variant catalog produce byte-equivalent
3+
# posteriors for the V2 strength GP configuration, AND that
4+
# boxcrete.load_pretrained_strength_gp() faithfully reconstructs the
5+
# deployed V2 strength GP from docs/model/strength_model.pt.
6+
#
7+
# Runs:
8+
# - test/test_strength_model_parity.py (atol=1e-8 / rtol=1e-6, ~3s)
9+
# - test/test_pretrained_loader_fidelity.py (atol=1e-5 / 1e-3, ~3s)
10+
#
11+
# See those test files for the rationale.
12+
13+
name: Strength GP Parity
14+
15+
permissions:
16+
contents: read
17+
18+
on:
19+
push:
20+
branches: [main, master]
21+
paths:
22+
- 'boxcrete/strength_model.py'
23+
- 'boxcrete/kernels.py'
24+
- 'boxcrete/likelihoods.py'
25+
- 'boxcrete/priors.py'
26+
- 'boxcrete/features.py'
27+
- 'boxcrete/__init__.py'
28+
- 'experiments/model_variant_study.py'
29+
- 'experiments/_research_features.py'
30+
- 'test/test_strength_model_parity.py'
31+
- 'test/test_pretrained_loader_fidelity.py'
32+
- 'docs/model/strength.json'
33+
- 'docs/model/strength_model.pt'
34+
- 'docs/model/test_vectors.json'
35+
- '.github/workflows/strength-parity.yml'
36+
pull_request:
37+
branches: [main, master]
38+
paths:
39+
- 'boxcrete/strength_model.py'
40+
- 'boxcrete/kernels.py'
41+
- 'boxcrete/likelihoods.py'
42+
- 'boxcrete/priors.py'
43+
- 'boxcrete/features.py'
44+
- 'boxcrete/__init__.py'
45+
- 'experiments/model_variant_study.py'
46+
- 'experiments/_research_features.py'
47+
- 'test/test_strength_model_parity.py'
48+
- 'test/test_pretrained_loader_fidelity.py'
49+
- 'docs/model/strength.json'
50+
- 'docs/model/strength_model.pt'
51+
- 'docs/model/test_vectors.json'
52+
- '.github/workflows/strength-parity.yml'
53+
workflow_dispatch: {}
54+
55+
jobs:
56+
parity:
57+
runs-on: ubuntu-latest
58+
steps:
59+
- uses: actions/checkout@v4
60+
61+
- uses: actions/setup-python@v5
62+
with:
63+
python-version: '3.12'
64+
cache: 'pip'
65+
66+
- name: Install Python deps
67+
run: |
68+
python -m pip install --upgrade pip
69+
pip install -e .
70+
pip install pytest
71+
72+
- name: Run parity test
73+
run: python -m pytest test/test_strength_model_parity.py -v --tb=short
74+
75+
- name: Run pretrained-loader fidelity test
76+
run: python -m pytest test/test_pretrained_loader_fidelity.py -v --tb=short

.github/workflows/tests.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ jobs:
2525
strategy:
2626
fail-fast: false
2727
matrix:
28-
python-version: ['3.10', '3.11', '3.12']
28+
python-version: ['3.11', '3.12']
2929

3030
steps:
3131
- name: Checkout repository

docs/feature_registry.mjs

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
/**
2+
* Engineered-feature registry for the strength GP.
3+
*
4+
* Each feature is computed from the RAW 10-dim composition vector
5+
* [Cement, Fly Ash, Slag, Water, HRWR, Fine, Coarse, Source, Temp, Time].
6+
* The same offsets (+1.0, +1e-3, +1e-4) used by the Python builders
7+
* apply here to match the GP's training-time numerical conditioning.
8+
*
9+
* Two callers consume this:
10+
* - gp.mjs's `transformInput` (general path, uses `appendFeatures`)
11+
* - gp_v2_fast.mjs's batched curve predictor (which inlines for speed,
12+
* so it imports `FEATURE_FNS` and walks the names array directly).
13+
*
14+
* Adding a new feature: add it to `FEATURE_FNS`. The strength.json's
15+
* `engineered_feature_names` field is the source of truth for which
16+
* features are active in the deployed model.
17+
*/
18+
19+
// Helper accessors for the raw composition vector.
20+
const I_CEMENT = 0, I_FLYASH = 1, I_SLAG = 2, I_WATER = 3;
21+
const I_HRWR = 4, I_FINE = 5, I_COARSE = 6;
22+
// I_SOURCE = 7 (not a feature)
23+
const I_TEMP = 8, I_TIME = 9;
24+
25+
/**
26+
* Map: feature_name → (xRaw: number[10]) → number.
27+
*
28+
* Source-of-truth: the union of the V2 deployed builders in
29+
* `boxcrete/features.py::FEATURE_BUILDERS` (the 7 names in
30+
* `engineered_feature_names`) and a subset of the research-only catalog
31+
* `experiments/_research_features.py::RESEARCH_FEATURE_BUILDERS` (kept
32+
* here so a research strength.json shipped with a non-default
33+
* `engineered_feature_names` list still resolves on the JS side without
34+
* the explorer needing a redeploy). The deployed model only consumes
35+
* the names it advertises in `engineered_feature_names`; the rest are
36+
* dead at runtime. Where Python uses
37+
* `x[..., _IDX["water"]:_IDX["water"]+1]`, we read `xRaw[I_WATER]`.
38+
*/
39+
export const FEATURE_FNS = {
40+
wb_ratio: (x) => x[I_WATER] / (x[I_CEMENT] + x[I_FLYASH] + x[I_SLAG] + 1.0),
41+
scm_frac: (x) => (x[I_FLYASH] + x[I_SLAG]) / (x[I_CEMENT] + x[I_FLYASH] + x[I_SLAG] + 1.0),
42+
hrwr_binder: (x) => x[I_HRWR] / (x[I_CEMENT] + x[I_FLYASH] + x[I_SLAG] + 1.0),
43+
log_hrwr_binder: (x) => Math.log(x[I_HRWR] / (x[I_CEMENT] + x[I_FLYASH] + x[I_SLAG] + 1.0) + 1e-4),
44+
wc_ratio: (x) => x[I_WATER] / (x[I_CEMENT] + 1.0),
45+
log_wc_ratio: (x) => Math.log(x[I_WATER] / (x[I_CEMENT] + 1.0) + 1e-3),
46+
coarse_fine: (x) => x[I_COARSE] / (x[I_FINE] + 1.0),
47+
log_coarse_fine: (x) => Math.log(x[I_COARSE] / (x[I_FINE] + 1.0) + 1e-3),
48+
agg_paste: (x) => (x[I_FINE] + x[I_COARSE]) /
49+
(x[I_CEMENT] + x[I_FLYASH] + x[I_SLAG] + x[I_WATER] + 1.0),
50+
log_agg_paste: (x) => Math.log(
51+
(x[I_FINE] + x[I_COARSE]) /
52+
(x[I_CEMENT] + x[I_FLYASH] + x[I_SLAG] + x[I_WATER] + 1.0) + 1e-3,
53+
),
54+
maturity_robust: (x) => Math.max(0, x[I_TEMP] + 10.0) * x[I_TIME],
55+
log_maturity_robust: (x) => Math.log(Math.max(0, x[I_TEMP] + 10.0) * x[I_TIME] + 1.0),
56+
};
57+
58+
/**
59+
* Append features to a raw input. Returns a NEW array of length
60+
* `xRaw.length + names.length`.
61+
*/
62+
export function appendFeatures(xRaw, names) {
63+
const out = new Array(xRaw.length + names.length);
64+
for (let i = 0; i < xRaw.length; i++) out[i] = xRaw[i];
65+
for (let k = 0; k < names.length; k++) {
66+
const fn = FEATURE_FNS[names[k]];
67+
if (!fn) {
68+
throw new Error(`Unknown engineered feature: '${names[k]}'. Add it to docs/feature_registry.mjs.`);
69+
}
70+
out[xRaw.length + k] = fn(xRaw);
71+
}
72+
return out;
73+
}

0 commit comments

Comments
 (0)