feat(scoring): replace weighted Urban95 scores with categorical status averages

- Remove weights and numeric total scores from Urban95 calculations
- Classify indicators as Disappointing, Functioning, or Thriving
- Infer continuous building and neighborhood status surfaces
- Redesign legends, explainers, and comparisons around stoplight statuses

Author: Tuvudel

README.md

@@ -9,8 +9,8 @@ This project maps how easy it is to reach everyday services and child-relevant f
 ## What you get
 
 - **Interactive map** (`docs/`) — MapLibre GL, with optional deck.gl clusters for dense points and Chart.js for dashboards.
-- **Two scoring models** — **Urban95** uses the weighted Urban95 methodology (0-100 score); **Amenities Focus** (internally `expanded`) uses a broader amenity-count classification.
-- **Three scales** — Single building (with walking-time area), neighborhood comparison, and citywide summary with rankings.
+- **Two scoring models** — **Urban95** uses equal-mean traffic-light statuses; **Amenities Focus** (internally `expanded`) uses a broader amenity-count classification.
+- **Three scales** — Single building (with walking-time area), neighborhood comparison, and citywide summary.
 
 ---
 
@@ -36,53 +36,33 @@ Processed layers for the website live under **`docs/data/`** (GeoJSON plus `neig
 
 Typical files include building footprints with accessibility fields, amenity layers (`amenities_new.geojson`, `amenities_all.geojson`), trees, parks, street lights, precomputed walking isochrones, and neighborhood boundaries. Exact filenames must match `docs/app.js`.
 
-### Building score fields used by the app
+### Building fields used by the app
 
-- **Urban95 top-level category scores**: `score_weighted_environmental_quality_*`, `score_weighted_nature_*`, `score_weighted_play_*`, `score_weighted_safety_mobility_*`, `score_weighted_family_services_*`.
-- **Urban95 subcategory scores**: `score_weighted_sub_<category>_<subcategory>_*` (generated by `python -m pipeline run score` via `stages/urban95_scoring.py` from `lib/urban95_weights.py`).
+- **Urban95 overview status**: `u95_status_*`.
+- **Urban95 category status**: `u95_status_<category>_*`.
+- **Urban95 direct indicator status**: `u95_status_sub_<category>_<indicator>_*`.
+- **Urban95 diagnostic status**: `u95_status_detail_<category>_<parent>_<child>_*`. School and Kindergarten are Education diagnostics; Clinic and Tipat Halav are Health diagnostics. They are visible in drill-downs but excluded from means.
 - **Amenities Focus score**: `score_expanded_*`.
 
-(`*` is one of `_5min`, `_10min`, `_15min`.)
+`*` is one of `_5min`, `_10min`, `_15min`; the frontend currently consumes `_10min`. Urban95 publishes status strings, raw explanation measurements such as `summer_si`, and no numeric overview total.
 
 ---
 
 ## Scoring methodology
 
-### Urban95 (displayed as direct 0-100 score)
-
-Urban95 total score is a weighted sum of five category scores. It is a methodology-driven score: category and subcategory definitions are fixed, and score logic is not directly tied to amenity filter categories in the UI.
-
-- Environmental Quality: **20%**
-- Nature: **15%**
-- Play: **15%**
-- Safety & Mobility: **25%**
-- Family Services: **25%**
-
-Each category is itself built from subcategory weights in `lib/urban95_weights.py`:
-
-- **Environmental Quality**
-  - Shade: 40% — Beer Sheva BDAR **Spatial Shade Index (`summer_SI`)** from the [Derech Tzel shading metrics guide](https://tzel.org.il/wp-content/uploads/2025/08/Shade-Indicators_eng-2.6.pdf). Values are used as-is (**SI, not SAI**; not recalculated). Each building gets a **300 m area-weighted mean `summer_SI` around the building footprint (near-edge buffer)**, then stored/displayed `summer_si` is **rounded to 1 decimal place with standard half-up ties before output and scoring** (`0.15 → 0.2`, `0.35 → 0.4`). The map shows continuous 0–1 SI using the official interpretation buckets (`<0.10 severe lack`, `0.10–<0.20 significant lack`, `0.20–<0.40 needs improvement`, `0.40–<0.60 good shade`, `≥0.60 excellent shade`). The Urban95 shade sub-score keeps a project-specific ternary mapping on that **rounded** building SI: `<0.20 → 0`, `0.20–<0.40 → 50`, `≥0.40 → 100`.
-  - Trees: 20% — count of trees within **20 m of the building footprint edge** (0 / 1–2 / ≥3 → 0 / 50 / 100)
-  - Distance from fast roads: 40% — nearest fast road distance measured to the **building footprint edge**
-- **Nature**
-  - Parks: 50%
-  - Urban nature areas: 50% (any site polygon within 300 m)
-- **Play**
-  - Playgrounds: 100%
-- **Safety & Mobility**
-  - Street lights: 15%
-  - Bicycle access: 15%
-  - Bus stops: 30%
-  - Shelters: 40%
-- **Family Services**
-  - Education: 30%
-  - Community centers: 20%
-  - Business centers: 20%
-  - Health: 30%
-
-`Explain score` in the app uses these saved top-level and subcategory score columns so each component shows both its weight and its score.
-
-Urban95 columns are stored per `_5min/_10min/_15min` for consistency with the rest of the data model, but Urban95 logic itself is mostly based on fixed-distance **near-edge** rules (buffers/distances from the building footprint) inside category functions in `lib/urban95_weights.py` rather than UI amenity filters. Amenities Focus walking isochrones still originate from building centroids.
+### Urban95 (equal-mean traffic-light statuses)
+
+Urban95 preserves the existing spatial indicator rules but presents every public Urban95 result as **Disappointing**, **Functioning**, **Thriving**, or **Unknown**. It does not publish, display, rank, or reconstruct a numeric Urban95 overview total.
+
+The hierarchy is Environmental Quality (Shade, Trees, Distance from fast roads), Nature (Parks, Urban nature areas), Play (Playgrounds), Safety & Mobility (Street lights, Bicycle access, Bus stops, Shelters), and Family Services (Education, Community centers, Business centers, Health). Direct indicators retain their project rules and internal 0/0.5/1 attainment levels. Sibling direct indicators contribute equally to their category; the five categories contribute equally to the overview. Thus categories with more indicators do not have extra influence.
+
+Category and overview means use the following status cutoffs: attainment below `0.25` is Disappointing, `0.25` through below `0.75` is Functioning, and `0.75` through `1.00` is Thriving. Exact midpoint ties classify upward. If any required direct child is Unknown, its category is Unknown; if any category is Unknown, the overview is Unknown. Unknown is never replaced with zero or omitted from a denominator.
+
+Missing, unreadable, schema-invalid, or failed source evidence produces Unknown for dependent results. A valid empty source or valid empty subtype remains available and follows the normal indicator rule. Per-building calculation failures produce Unknown only for that building. Area summaries count all four statuses and use the uniquely predominant building status only; ties and unsupported areas publish Unknown with a summary reason rather than a hidden average.
+
+**Environmental Quality / Shade** — Beer Sheva BDAR **Spatial Shade Index (`summer_SI`)** comes from the [Derech Tzel shading metrics guide](https://tzel.org.il/wp-content/uploads/2025/08/Shade-Indicators_eng-2.6.pdf). Values are used as-is (**SI, not SAI**; not recalculated). Each building gets a **300 m area-weighted mean `summer_SI` around the building footprint (near-edge buffer)**, then stored/displayed `summer_si` is **rounded to 1 decimal place with standard half-up ties before output and status calculation** (`0.15 → 0.2`, `0.35 → 0.4`). The separate official SI interpretation remains `<0.10 severe lack`, `0.10–<0.20 significant lack`, `0.20–<0.40 needs improvement`, `0.40–<0.60 good shade`, and `≥0.60 excellent shade`. Urban95's status rule is project-defined: rounded SI `<0.20`, `0.20–<0.40`, and `≥0.40` map to low, middle, and high attainment respectively.
+
+Urban95 status columns retain `_5min/_10min/_15min` for compatibility even though its direct rules are generally fixed-distance **near-edge** rules from building footprints. Amenities Focus walking isochrones still originate from building centroids.
 
 ### Amenities Focus (internally `expanded`)
 
@@ -153,7 +133,7 @@ The commands and browser-facing output contracts above are unchanged. In `run al
 
 For acceptance-only warm-run checks, set `PIPELINE_FORBID_MAPBOX=1`. A guarded run validates the complete `(building_id, minutes)` warm aggregate and aborts before token/session/network work when it is incomplete; ordinary unguarded runs retain their existing Mapbox/cache behavior.
 
-To recompute Urban95 weighted columns (including shade SI) on existing buildings without Mapbox or isochrones:
+To recompute Urban95 status fields (including shade SI) on existing buildings without Mapbox or isochrones:
 
 ```powershell
 $env:PYTHONPATH="src"
@@ -168,7 +148,7 @@ You can refresh **one concern** without Mapbox or a full rebuild — but only if
 
 | You changed… | Run (no Mapbox unless noted) | Must refresh for the live app |
 |--------------|------------------------------|-------------------------------|
-| Urban95 weights / shade SI mapping | `run shade` (if SI inputs changed) → `run rescore` → `run neighborhoods` | `buildings_lookup` (+ gz), buildings `.gz`, neighborhoods / charts / citywide |
+| Urban95 status rules / shade SI mapping | `run shade` (if SI inputs changed) → `run rescore` → `run neighborhoods` | `buildings_lookup` (+ gz), buildings `.gz`, neighborhoods / charts / citywide |
 | Clean amenities / trees / lights inventory (raw) | `run amenity_metrics` → `run score` → `run export_web` → `run neighborhoods` | Same publish set + `amenities_new` / trees / lights sync; **isochrones reused from cache** |
 | Isochrone geometry / walk sheds | `run isochrones` (**Mapbox** if cache cold) → `run amenity_metrics` → `run score` → `run export_web` → `run neighborhoods` | Isochrones + lookup + aggregates |
 | Neighborhood chart logic only | `run neighborhoods` | neighborhoods / surface / charts / citywide (buildings unchanged) |

docs/app.js

@@ -45,19 +45,17 @@ const {
   Urban95PointDataSources,
   urban95RuntimeLoaders,
   Urban95ScoreModel,
+  Urban95StatusScale,
   AMENITY_TYPE_CONFIG,
   DEFAULT_CONFIG,
   WEIGHTED_CATEGORY_LABEL_BY_STEM,
   getAmenityConfig,
   amenityTypeToBuildingStatKey,
   percentileBreakpoints,
-  buildHistogramDistributionFromScores,
   getColorForValue,
   bulkPercentileRanks,
   formatMetricNumber,
   formatScoreInteger,
-  weightedCategoryHighlightsFromSource,
-  weightedSubcategoryComparisonRows,
   Urban95WeightedMetricShowRegistry,
   Urban95ScoreContext,
   Urban95ScoreExplain,
@@ -70,6 +68,7 @@ const {
   Urban95NeighborhoodScores,
   Urban95RenderState,
   resolveBuildingContracts,
+  createBuildingFillColorExpression,
   createPmtilesProtocol,
   createBuildingsSource,
   createBuildingsFillLayer,
@@ -134,8 +133,10 @@ const BUILDINGS_FILL_LAYER_ID = BUILDING_LAYER_CONTRACTS.fillLayerId;
 const BUILDINGS_SELECTED_LAYER_ID = BUILDING_LAYER_CONTRACTS.selectedLayerId;
 const BUILDINGS_VECTOR_LAYER_ID = BUILDING_LAYER_CONTRACTS.vectorLayerId;
 const BUILDINGS_SYM_PCT_STATE_KEY = BUILDING_LAYER_CONTRACTS.symPctStateKey;
+const BUILDINGS_STATUS_STATE_KEY = BUILDING_LAYER_CONTRACTS.statusStateKey;
 const BUILDINGS_SELECTED_STATE_KEY = BUILDING_LAYER_CONTRACTS.selectedStateKey;
-const BUILDINGS_CHOROPLETH_FILL_COLOR_EXPR = BUILDING_LAYER_CONTRACTS.fillColorExpression;
+const DEFAULT_URBAN95_METRIC = Urban95ScoreModel.getWeightedMetric("u95.overall");
+const BUILDINGS_CHOROPLETH_FILL_COLOR_EXPR = createBuildingFillColorExpression(DEFAULT_URBAN95_METRIC);
 const _urban95BuildingsSource = createBuildingsSource({
   artifacts: GENERATED_ARTIFACTS,
   buildingsPmtilesPath: BUILDINGS_PMTILES_URL,
@@ -237,8 +238,6 @@ const getWeightedNeighborhoodMetricValue =
   Urban95RenderState.getWeightedNeighborhoodMetricValue;
 const hasWeightedNeighborhoodMetricData =
   Urban95RenderState.hasWeightedNeighborhoodMetricData;
-const getWeightedHistogramDistribution =
-  Urban95RenderState.getWeightedHistogramDistribution;
 const scoreContext = Urban95ScoreContext.create({
   scoreModel: Urban95ScoreModel,
   state: {
@@ -495,11 +494,15 @@ Urban95MapRenderers.configure({
   collectBuildingScores: collectBuildingScores,
   bulkPercentileRanks: bulkPercentileRanks,
   symPctKey: SYM_PCT_KEY,
+  normalizeStatus: Urban95StatusScale.normalize,
+  statusMatchExpression: Urban95StatusScale.matchExpression,
   buildingsMapSourceId: BUILDINGS_MAP_SOURCE_ID,
   buildingsVectorLayerId: BUILDINGS_VECTOR_LAYER_ID,
   buildingsSymPctStateKey: BUILDINGS_SYM_PCT_STATE_KEY,
+  buildingsStatusStateKey: BUILDINGS_STATUS_STATE_KEY,
   buildingsFillLayerId: BUILDINGS_FILL_LAYER_ID,
   buildingsChoroplethFillColorExpr: BUILDINGS_CHOROPLETH_FILL_COLOR_EXPR,
+  createBuildingFillColorExpression: createBuildingFillColorExpression,
   getNeighborhoodSurfaceData: function () {
     return neighborhoodSurfaceData;
   },
@@ -783,6 +786,7 @@ const neighborhoodSelection = Urban95NeighborhoodSelection.create({
     return Urban95ScoreModel.neighborhoodIsComparable(feature && feature.properties, {
       scoreMode: getScoreModeState(),
       minutes: getScoreMinutes(),
+      activeMetric: getActiveMetricState(),
     });
   },
 });
@@ -830,12 +834,6 @@ Urban95NeighborhoodSidebar.configure({
   renderDeps: {
     pieSlicesFromInventoryCounts: Urban95Dashboards.pieSlicesFromInventoryCounts,
     getActiveMetric: getActiveMetricState,
-    getWeightedHistogramDistribution: getWeightedHistogramDistribution,
-    weightedCategoryHighlightsFromSource: weightedCategoryHighlightsFromSource,
-    weightedSubcategoryComparisonRows: weightedSubcategoryComparisonRows,
-    renderWeightedSubcategoryComparisonList: scoreExplain.renderWeightedSubcategoryComparisonList,
-    buildHistogramDistributionFromScores: buildHistogramDistributionFromScores,
-    collectBuildingScores: collectBuildingScores,
     getColorForValue: getColorForValue,
     getNeighborhoodPercentileKey: getNeighborhoodPercentileKey,
     getOrdinalSuffix: scoreExplain.getOrdinalSuffix,
@@ -890,9 +888,7 @@ Urban95CitySidebar.configure({
     getScoreModeLabel: getScoreModeLabel,
     getOrdinalSuffix: scoreExplain.getOrdinalSuffix,
     heroPercentileMeterFillStyle: scoreExplain.heroPercentileMeterFillStyle,
-    getWeightedHistogramDistribution: getWeightedHistogramDistribution,
     getNeighborhoodPercentileKey: getNeighborhoodPercentileKey,
-    buildHistogramDistributionFromScores: buildHistogramDistributionFromScores,
     collectBuildingScores: collectBuildingScores,
     percentileBreakpoints: percentileBreakpoints,
     getColorForValue: getColorForValue,