docs(session): end Session 5 — medallion green end-to-end in Fabric

- CHANGELOG: new sub-section for the 2026-05-29 first-green cloud run +
  operational fixes (Spark conf for identity, lakehouse GUIDs in OneLake
  paths, 01 validation cell robust to large bundles, public_libraries.yml
  with pinned boto3/botocore, fabric-deploy.yml disabled, FabricPlatform
  files_path helper).
- ARCHITECTURE: lead paragraph rewritten for ADR-022 two-tier model
  (LocalLite + Fabric as independent end-to-end impls); Fabric execution
  status row flipped from '🔜 Session 5' to '✅ green on F4'; execution-
  surfaces paragraph updated to reflect Fabric notebooks as their own
  surface (not 'upcoming').
- HANDOFF: date bumped to 2026-05-31; content unchanged (state matches
  the pause point — capacity paused, all storage persists, next task is
  screenshot confirmation → Data Pipeline build → Power BI → PR).
- Generators (gen_data_dictionary, gen_corpus_schema) re-run, no diff.
- pytest: 128 passed + 1 skipped (workspace-only).

Author: sandeep-jay

CHANGELOG.md

@@ -8,6 +8,67 @@ Format: [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 ### Session 5 (in progress) — Fabric Spark-native rewrite (ADR-022) + dedup fix + Power BI
 Plan: [docs/roadmap/fabric-execution-plan.md](docs/roadmap/fabric-execution-plan.md).
 
+#### Milestone (2026-05-29 — first green cloud run)
+- **Notebooks 00–10 ran successfully end-to-end on Fabric F4 capacity**
+  against `SAMPLE_SIZE=100` Coherent bundles. All 10 Silver tables +
+  `gold.encounter_summary` + Bronze/Gold manifests materialized in the
+  `scribe_iq_synthea_coherent` lakehouse.
+- Branch `feat/fabric-spark-native` pushed to **both** GitHub (canonical
+  mirror) and Azure DevOps (Fabric Git Integration source) via
+  multi-push origin. Single `git push` fans out to both.
+
+#### Added (2026-05-29)
+- `fabric/notebooks/01_bronze_ingest.Notebook/` — self-contained Bronze
+  ingest. Pulls Synthea Coherent from `s3://synthea-open-data/coherent/`
+  via anonymous boto3, round-robin partitions into `cohort=A,B,C` under
+  `Files/bronze/fhir/`, writes an `IngestManifest`-shaped JSON under
+  `Files/bronze/_metadata/`. `SAMPLE_SIZE` knob for fast demo (`100`) vs
+  full corpus (`None`).
+- `fabric/environments/public_libraries.yml` — pip-block file Fabric's
+  Environment "Import .yml" UI accepts; pins `boto3==1.35.36` +
+  `botocore==1.35.36` for reproducibility.
+- `FabricPlatform.files_path(subpath)` — Files/-rooted URI helper for
+  non-table artifacts (Bronze JSON, Gold manifest). One place owns the
+  GUID-vs-name path detail.
+
+#### Changed (2026-05-29 — operational fixes from cloud run)
+- `FabricPlatform.ensure_env` now reads from Spark conf
+  (`trident.workspace.id`, `trident.lakehouse.id`) instead of
+  `mssparkutils.env.getWorkspaceId()` — the latter is a Synapse API
+  not present on Fabric. Returns workspace + lakehouse GUIDs (not name);
+  display name is best-effort, informational only.
+- OneLake paths now use lakehouse GUID throughout (drop `.Lakehouse`
+  suffix). Required for tenants with `FriendlyNameSupportDisabled`
+  (the trial tenant has this) — `<name>.Lakehouse` paths get HTTP 400.
+  Notebooks 00, 01, 10 updated to use `platform.files_path()` instead
+  of inline path construction.
+- `00_setup` Gate 1 reads `spark.conf.get("trident.workspace.id")`
+  (drops the broken `mssparkutils.env.getWorkspaceId` call).
+- `01_bronze_ingest` validation cell uses `spark.read.text(wholetext=True)`
+  to read the sample bundle — `mssparkutils.fs.head` silently truncates
+  at ~100 KB even when a larger maxBytes is passed, breaking
+  `json.loads`. Sample-histogram wrapped in try/except so a parse
+  failure prints a one-liner instead of halting the cell (manifest
+  write below it now always runs).
+- `fabric/environments/lakehouse_env.yml` — documentation-style spec
+  updated to match ADR-022; drops `pyarrow`/`pydicom`/`python-dateutil`
+  (not used by the pure-Spark Fabric tier — Fabric runtime supplies
+  pyarrow; pydicom is local-only; date parsing is Spark-native).
+- `.github/workflows/fabric-deploy.yml` renamed
+  `fabric-deploy.yml.disabled`. User removed the `fabric-prod` GitHub
+  Environment; the workflow's `environment: fabric-prod` would fail on
+  trigger. Matches the existing `aws-deploy.yml.disabled` /
+  `databricks-deploy.yml.disabled` convention. Active deploy path is
+  Azure DevOps Git Integration + manual UI wheel upload.
+
+#### Tests (2026-05-29)
+- `test_fabric_platform.py` updated for GUID-based API:
+  `test_storage_path_builds_onelake_uri` rewritten for the GUID shape
+  (no `.Lakehouse` suffix). New `test_files_path_builds_onelake_uri`
+  covers the helper. `FabricPlatform(lakehouse_id=...)` constructor
+  arg replaces `lakehouse_name=...` for path-shape tests.
+- Full suite: 128 passed + 1 skipped (workspace-only).
+
 #### Added (2026-05-29 — ADR-022 architecture pivot)
 - **ADR-022** (Independent per-platform implementations) — supersedes ADR-002
   (LakehousePlatform ABC as universal contract), ADR-004 (pa.Table as

HANDOFF.md

@@ -1,5 +1,5 @@
 # HANDOFF — Session 5 · Medallion green end-to-end in Fabric
-**Date:** 2026-05-29 · **Branch:** `feat/fabric-spark-native` · **Plan:** [docs/roadmap/fabric-execution-plan.md](docs/roadmap/fabric-execution-plan.md)
+**Date:** 2026-05-31 · **Branch:** `feat/fabric-spark-native` · **Plan:** [docs/roadmap/fabric-execution-plan.md](docs/roadmap/fabric-execution-plan.md)
 
 > State only. For what happened in this (or any prior) session see [CHANGELOG.md](CHANGELOG.md).
 > Narrative belongs there, not here.

docs/ARCHITECTURE.md

@@ -7,13 +7,15 @@ structure changes — it tracks reality, not the plan.
 
 ## Overview
 
-A medallion healthcare lakehouse on Synthea Coherent (synthetic FHIR R4). Engine-agnostic
-**pure transforms** return Apache Arrow tables; a **platform abstraction** handles all I/O
-so the same code runs locally (Polars + delta-rs) or on Microsoft Fabric. Today the
-**Bronze → Silver → Gold** path is fully built and runs end-to-end on the full
-1,278-patient dataset locally (143,946 encounter summaries), under either the
-`core.surfaces.cli.pipeline` CLI or a **Dagster** software-defined asset graph (ADR-015/016);
-**Fabric execution** is next.
+A medallion healthcare lakehouse on Synthea Coherent (synthetic FHIR R4). Per
+**ADR-022** the lakehouse runs as **two independent end-to-end implementations** that
+emit the same Gold corpus contract: a LocalLite tier (`core/` — Polars + delta-rs)
+and a Fabric tier (`fabric/` — Spark + OneLake Delta), each with its own Silver,
+Gold, and validation stack written engine-native. Today the **Bronze → Silver → Gold**
+path is fully built on both tiers — locally it runs end-to-end on the full
+1,278-patient dataset (143,946 encounter summaries) via the `core.surfaces.cli.pipeline`
+CLI or a **Dagster** software-defined asset graph (ADR-015/016); on Fabric it ran
+green end-to-end on F4 capacity against a 100-patient demo sample via notebooks 00–10.
 
 ```mermaid
 flowchart TD
@@ -41,11 +43,13 @@ flowchart TD
     class B,ST,G done;
 ```
 
-The same transforms run under **three execution surfaces** (ADR-015/016): the
-dependency-light `core.surfaces.cli.pipeline` CLI, a **Dagster** asset graph (cohorts =
-partitions, `validate_table` = asset checks, sensor watches `data/bronze/fhir/`),
-and the upcoming Fabric notebooks. The orchestration tier imports the pure
-transforms and platform — the lakehouse never imports orchestration.
+The LocalLite tier runs under **two local execution surfaces** (ADR-015/016): the
+dependency-light `core.surfaces.cli.pipeline` CLI and a **Dagster** asset graph
+(cohorts = partitions, `validate_table` = asset checks, sensor watches
+`data/bronze/fhir/`). The Fabric tier runs under its own independent execution
+surface — `fabric/notebooks/00–10` (`.Notebook/notebook-content.py` source-of-truth,
+ADR-021), each instantiating `FabricPlatform()` directly. The orchestration / notebook
+tiers import the per-tier transforms and platform — neither tier imports the other.
 
 For **read-only exploration** the same Delta tables are queryable from a
 **DuckDB UI notebook** ([`docs/demo/notebooks/demo_notebook.sql`](demo/notebooks/demo_notebook.sql))
@@ -63,7 +67,7 @@ renderers, three audiences. Recording guide: [`docs/demo/PLAYBOOK.md`](demo/PLAY
 | Silver | ✅ built (local) | 10 Delta tables + `ingest_log` | CDC enabled; validated; MERGE-upsert per cohort |
 | Gold | ✅ built (local) | `encounter_summary` Delta + manifest | 1 row/encounter; CDC; as-of-date problem list; corpus contract v1.1.0 (ADR-012/014) |
 | Dagster orchestration | ✅ built (local) | `core/orchestration/dagster/` package | medallion as asset graph; cohort partitions; `validate_table` as asset checks (ADR-015/016) |
-| Fabric execution | 🔜 Session 5 | OneLake | notebooks 00–10; S3 shortcut; same transforms |
+| Fabric execution | ✅ green end-to-end (F4, SAMPLE_SIZE=100) | OneLake | independent Spark-native impl (ADR-022); notebooks 00–10; anonymous S3 ingest in 01; full 1,278-bundle re-run pending |
 
 ## Module map