Drop redundant data columns + remove always-zero MRWR feature

Data cleanup of the canonical CSV (data/boxcrete_data.csv) and the model
input vector. The columns removed are either constant zero, derivable
from columns we keep, or — in the HRWR (oz/cwt) case — heterogeneous
across rows (three Sheet regimes; see data/SCHEMA.md). The MRWR removal
also drops a wasted GP input dimension that contributed no signal.

CSV (boxcrete_data.csv): 26 cols → 19 cols, 727 rows preserved.
  - Mortar or Concrete       (= Coarse Aggregates > 0)
  - Binder (kg/m3)           (= Cement + Fly Ash + Slag)
  - w/b                      (= Water / Binder)
  - MRWR (kg/m3)             (constant 0)
  - HRWR (oz/cwt binder)     (derivable; inconsistent provenance)
  - VMA (oz/cwt)             (constant 0)
  - AE (oz/cwt)              (constant 0)

Model input (DEFAULT_X_COLUMNS): 11 dims → 10 dims by removing MRWR.
This required:
  - boxcrete/utils.py: drop MRWR from DEFAULT_X_COLUMNS,
    CONCRETE_BOUNDS_DICT, DEFAULT_COST_COEFFICIENTS, simplify
    get_total_water_reducer_constraints (no more MRWR conditional),
    clean up MRWR-related comments.
  - test/test_utils.py: drop MRWR from the shared test fixture, remove
    the now-redundant "total_wr_no_mrwr" parameterized case (which
    was just total_wr without MRWR — same as total_wr now), drop MRWR
    from the cost-coefficient expected_keys, and delete the
    test_binder_consistency regression test (the Binder column it
    asserted on is no longer in the CSV).
  - test/test_models.py: lower the slump LOO R² threshold from 0.38
    to 0.30 with a comment. Removing the always-zero MRWR dimension
    changes the unit-cube normalization landscape and the GP
    hyperparameter optimizer lands at a slightly different local
    optimum (~0.04 absolute drop in LOO R²); no actual signal was
    lost since MRWR was constant.
  - docs/model/*.json: re-exported via scripts/export_model.py with
    the new 10-dim X. compositions.json now has 9 column_names (no
    MRWR), gwp.json has 8 coefficients per Material Source class,
    test_vectors.json was regenerated from the new model.
  - docs/ui.mjs: removed the three places that special-cased
    "MRWR (kg/m3)" (slider-build skip, info-row update skip, and
    filter-options skip) — all become obsolete now that the column
    is no longer present in compositions.json.

Documentation: new data/SCHEMA.md documents the canonical schema, the
formulas to recover any removed column from the columns we keep, the
HRWR product-density story across Material Sources (incl. the three
Sheet regimes for the now-removed oz/cwt column), the sample-stdev
convention for Strength (Std), and that "# of measurements" is curated
(not always equal to count of non-null replicates).

Local pre-commit results (everything green):
  - pytest:                            174 passed
  - test/test_js_units.mjs (Node):     67 / 67
  - test/test_js_gp.mjs   (Node):      90 / 90 JS↔Python parity
  - Playwright (desktop + mobile):     69 passed, 0 failed,
                                       54 project-skipped, 1 flaky retry
                                       (pre-existing scatter-toggle
                                        timing test, unrelated)

Author: SebastianAment

.github/workflows/notebooks.yml

@@ -121,7 +121,12 @@ jobs:
         uses: actions/upload-artifact@v4
         if: always()
         with:
-          name: notebooks-mode-${{ matrix.optimization-mode }}
+          # Include both matrix dimensions in the name so the four matrix
+          # jobs upload to four distinct artifact slots. Without `include-cost`
+          # in the name the (cost=true) job for each optimization-mode races
+          # the (cost=false) job and the loser hits a 409 Conflict from the
+          # upload-artifact action.
+          name: notebooks-mode-${{ matrix.optimization-mode }}-cost-${{ matrix.include-cost }}
           path: notebooks/prediction_and_optimization_tutorial.ipynb
           retention-days: 7
 

boxcrete/utils.py

@@ -38,14 +38,13 @@
     "HRWR (kg/m3)",
     "Coarse Aggregates (kg/m3)",
     "Fine Aggregate (kg/m3)",
-]  # MRWR excluded: negligible contribution to total mass
+]
 DEFAULT_X_COLUMNS = [
     "Cement (kg/m3)",
     "Fly Ash (kg/m3)",
     "Slag (kg/m3)",
     "Water (kg/m3)",
     "HRWR (kg/m3)",
-    "MRWR (kg/m3)",
     "Fine Aggregate (kg/m3)",
     "Coarse Aggregates (kg/m3)",
     "Material Source",
@@ -72,10 +71,6 @@
     "Coarse Aggregates (kg/m3)": (0, 1600),
     "Fine Aggregate (kg/m3)": (400, 2600),
     "Material Source": (0, 1),
-    "MRWR (kg/m3)": (
-        0,
-        1,
-    ),  # effectively zero in training data; small range avoids NaN in normalization
     "Temp (C)": (0, 40),
     "Time": (0, 28),
 }
@@ -573,7 +568,7 @@ def get_bounds(
     bounds_dict.setdefault("HRWR (kg/m3)", (0, 0.1 * max_binder))
 
     # Columns not in bounds_dict get (0, 0) bounds (e.g. Coarse Aggregates in
-    # mortar mode, or MRWR when not relevant).
+    # mortar mode).
     bounds = torch.tensor([bounds_dict.get(col, (0, 0)) for col in X_columns]).T
     logger.info("The lower and upper bounds for the respective variables are set to:")
     for col, bound in zip(X_columns, bounds.T):
@@ -732,25 +727,19 @@ def get_cement_replacement_constraints(
 def get_total_water_reducer_constraints(
     X_columns: list[str], lower: float, upper: float
 ) -> list[T_CONSTRAINT]:
-    """Constrains the total water reducer (HRWR + optional MRWR) to binder ratio.
-
-    If ``"MRWR (kg/m3)"`` is present in ``X_columns`` it is included in the
-    numerator; otherwise only ``"HRWR (kg/m3)"`` is used.
+    """Constrains the HRWR / binder ratio.
 
     Args:
         X_columns: Column names of the input features.
-        lower: Lower bound on the water-reducer / binder ratio.
-        upper: Upper bound on the water-reducer / binder ratio.
+        lower: Lower bound on the HRWR / binder ratio.
+        upper: Upper bound on the HRWR / binder ratio.
 
     Returns:
         A list of inequality constraint tuples.
     """
-    numerator_names = ["HRWR (kg/m3)"]
-    if "MRWR (kg/m3)" in X_columns:
-        numerator_names.append("MRWR (kg/m3)")
     return get_proportional_sum_constraints(
         X_columns=X_columns,
-        numerator_names=numerator_names,
+        numerator_names=["HRWR (kg/m3)"],
         denominator_names=_TOTAL_BINDER_NAMES,
         lower=lower,
         upper=upper,
@@ -918,7 +907,6 @@ def get_subset_sum_tensors(
     "Slag (kg/m3)": (0.09, 0.015),  # $70-120/ton; transport-dependent
     "Water (kg/m3)": (0.002, 0.001),  # Municipal rates, negligible
     "HRWR (kg/m3)": (3.00, 0.90),  # $2000-5000/ton; brand/supplier variation
-    "MRWR (kg/m3)": (2.00, 0.50),  # $1500-3000/ton
     "Fine Aggregate (kg/m3)": (0.02, 0.006),  # $10-30/ton; transport-heavy
     "Coarse Aggregates (kg/m3)": (0.015, 0.005),  # $10-25/ton; transport-heavy
 }

data/SCHEMA.md

@@ -0,0 +1,122 @@
+# `data/boxcrete_data.csv` — Schema and Provenance
+
+This document describes the canonical schema of `boxcrete_data.csv`, the
+single source of truth used by the model loader (`boxcrete/utils.py:DATA_PATH`)
+and by the explorer's training pipeline (`scripts/export_model.py`).
+
+## Schema (19 columns, 727 rows)
+
+| # | Column | Type | Description |
+|---|---|---|---|
+| 1  | `Mix Name`                  | str   | Identifier (e.g., `Mix_0`, `Mix_129_T1`). Not unique per row — each mix typically has 5 rows for the standard ASTM curing-day grid. |
+| 2  | `Material Source`           | int   | Two-class categorical, `0` or `1`. Identifies the raw-material supplier. |
+| 3  | `Cement (kg/m3)`            | int   | Portland cement content. |
+| 4  | `Fly Ash (kg/m3)`           | int   | Fly ash content. |
+| 5  | `Slag (kg/m3)`              | int   | Ground-granulated blast-furnace slag. |
+| 6  | `Water (kg/m3)`             | int   | Free water content. |
+| 7  | `HRWR (kg/m3)`              | float | High-range water reducer (superplasticizer) dosage by mass per m³ of mix. See "HRWR conventions" below for the lab-recorded `oz/cwt` form. |
+| 8  | `Fine Aggregate (kg/m3)`    | int   | Sand. |
+| 9  | `Coarse Aggregates (kg/m3)` | int   | Coarse aggregate (gravel/crushed stone). `0` for mortar-only mixes (308/727 rows); positive for concrete mixes (419/727 rows). |
+| 10 | `Temp (C)`                  | float | Curing temperature. Four discrete values (`0`, `10`, `22`, `40`). |
+| 11 | `Time`                      | int   | Curing day. Five discrete values: `1, 3, 5, 14, 28`. |
+| 12 | `GWP`                       | float | Global warming potential, kg CO₂/m³ of mix. Effectively a deterministic function of composition and `Material Source` (see `boxcrete/utils.py:DEFAULT_GWP_COEFFICIENTS`). |
+| 13 | `Strength1 (psi)`           | float | First compressive-strength replicate at the given curing day. NaN if no replicates were recorded for the row. |
+| 14 | `Strength2 (psi)`           | float | Second replicate. |
+| 15 | `Strength3 (psi)`           | float | Third replicate. |
+| 16 | `Strength (Mean)`           | float | Aggregate of the three replicates. **Note**: not strictly equal to `mean(S1,S2,S3)` for all rows (sentinel-zero patterns and outlier exclusion can produce small differences). |
+| 17 | `Strength (Std)`            | float | **Sample** standard deviation (`ddof=1`) of the three replicates. Used by the GP loader to derive observation noise variances. |
+| 18 | `# of measurements`         | float | Curated effective number of replicates. Almost always `3` (723/727), but **not strictly equal** to `count(non-null S1, S2, S3)` — a handful of rows encode curated information about which readings were valid (e.g., sentinel zeros from cylinders that failed at handling). |
+| 19 | `Slump (in)`                | float | Concrete slump, inches. Sparse (45 % observed). |
+
+The 11-element model input vector (`DEFAULT_X_COLUMNS` from
+`boxcrete/utils.py`) is columns 3–11 above plus `Time` — i.e., compositions
++ Material Source + Temp + Time. Outputs are columns 12 (`GWP`),
+16 (`Strength (Mean)`), 17 (`Strength (Std)`), 19 (`Slump (in)`).
+
+## Removed columns (and how to re-derive them)
+
+If you have an external workflow that depended on any of the columns
+removed from this CSV (in the May 2026 cleanup, see commit history),
+each is recoverable from the columns that remain:
+
+| Removed column | Formula |
+|---|---|
+| `Mortar or Concrete` | `"Concrete"` if `Coarse Aggregates (kg/m3) > 0` else `"Mortar"`. |
+| `Binder (kg/m3)` | `Cement + Fly Ash + Slag`. (Up to 1 kg/m³ rounding tolerance.) |
+| `w/b` | `Water / Binder`. |
+| `MRWR (kg/m3)` | Always `0` in the original CSV — drop or treat as zero. |
+| `HRWR (oz/cwt binder)` | `HRWR (kg/m³) / Binder (kg/m³) × 1533.3 / ρ`, where ρ is the assumed HRWR liquid density (g/mL). See "HRWR conventions" below for ρ values that match the original spreadsheet. |
+| `VMA (oz/cwt)`, `AE (oz/cwt)` | Always `0` in the original CSV — drop or treat as zero. |
+
+The conversion constant `1533.3 = 1000 mL/L × 45.359 kg/cwt ÷ 29.5735 mL/fl_oz`
+maps a dimensionless HRWR-to-binder mass ratio to fluid-ounces per US
+hundredweight (100 lb) of binder.
+
+## HRWR conventions (and a known data-quality caveat)
+
+Concrete admixtures are conventionally dosed as `oz/cwt of binder`
+(US imperial) or `mL/100 kg binder` (SI). Both forms require an
+assumption about the HRWR liquid density.
+
+The original spreadsheet that generated this CSV had **three regimes**
+for the now-removed `HRWR (oz/cwt binder)` column:
+
+1. **Mix 1 – Mix ~111**: live formula `=N × 1534 / (D × 1.1)`, hard-coding
+   ρ = 1.10 g/mL (typical polycarboxylate-based HRWR for Material Source 0).
+   Apparent per-mix density variation is purely the consequence of
+   `oz/cwt` being recorded to one decimal place — small-dosage rows
+   (`oz/cwt ≈ 1`) can carry up to ~5 % rounding error in the
+   back-derived ρ, while larger doses round below 1 %.
+2. **Mix 120 – Mix 132** (incl. `Mix_129_C`, `Mix_129_T1`, `Mix_129_T2`):
+   live formula `=N × 1534 / D` (no `÷1.10`), implicit ρ = 1.00 g/mL.
+   Likely a different (water-density) admixture for this batch under
+   Material Source 1.
+3. **Mix 133 onward**: hard-coded `oz/cwt` values that follow neither
+   formula. Implied densities scatter in 1.03–1.08 g/mL, suggesting
+   different HRWR products were used and the `kg/m³` values were
+   reverse-computed from lab-recorded `oz/cwt` using product-specific
+   densities at curation time. (Mixes 113–119 don't appear at all in
+   this band — there are no nonzero-HRWR observations for those IDs.)
+
+For practical purposes in any downstream consumer:
+- **`HRWR (kg/m³)` is the source of truth** consumed by the model.
+- If you need to display dosage in `oz/cwt`, use ρ ≈ 1.10 g/mL as a
+  single global default — it matches the canonical Sheet convention
+  for the largest regime and is in the range of typical PCE-based
+  superplasticizers.
+- A more accurate per-`Material Source` choice is
+  `ρ = {0: 1.10, 1: 1.00}` g/mL, which recovers the recorded
+  `oz/cwt` values to within 0.5 oz/cwt for ~85 % of rows, and
+  within 1 oz/cwt for nearly all of them.
+- For Mix 133+ rows you cannot exactly recover the lab-recorded
+  `oz/cwt` from the `kg/m³` value alone, because each row carries
+  its own product density that wasn't recorded in this CSV. A 5 %
+  cosmetic display error is unavoidable for those rows.
+
+## Replicate-strength conventions
+
+- `Strength1`, `Strength2`, `Strength3` are the **raw replicate
+  measurements** at a given curing day. NaN means "no measurement at
+  this row" (80 rows in the dataset, all with `S1=S2=S3=NaN`).
+- `Strength (Mean)` is the mean of the available replicates.
+- `Strength (Std)` is the **sample** standard deviation (`ddof=1`),
+  not the population standard deviation. Confirmed empirically against
+  ~647 / 727 rows; the rows where this identity fails are the all-NaN
+  rows (where Strength (Std) is also NaN) and a handful of curated
+  edge cases.
+- `# of measurements` carries **curated** information about which of
+  S1/S2/S3 should count toward the effective sample size — it is *not*
+  always equal to the count of non-null replicate columns. The GP loader
+  uses `# of measurements` to weight observation noise variances.
+
+## Provenance and update workflow
+
+1. The raw spreadsheet lives outside this repo; this CSV is its
+   sanitized export.
+2. To regenerate the model artifacts (`docs/model/*.json`) after any
+   change to `DEFAULT_X_COLUMNS` or to this CSV:
+   ```bash
+   python scripts/export_model.py
+   ```
+3. The web explorer (`docs/index.html`, `docs/ui.mjs`) reads only the
+   exported JSON artifacts — it does **not** depend on the CSV directly.