feat(model-apps): verify what persona security roles grant, + public-repo doc hygiene (#425)

* feat(model-apps): verify what persona security roles actually grant

Two metadata-only verification gaps, both found while auditing what `verify`
can and cannot prove. Neither needs a live browser or new infrastructure.

1. verify now proves what a persona role GRANTS, not just that it exists.

   The `role` check asserted only that a role ROW exists carrying the SDK
   ownership marker. It never looked at privileges - so a role created with the
   wrong access, or one whose privilege write failed after the row landed,
   verified clean.

   The new `role-privileges` check resolves each declared (entity, access) to
   its Dataverse PrivilegeId from the SAME metadata source the SDK writes
   against - EntityDefinitions(...)?$select=Privileges - and asserts the role
   holds it at AT LEAST the declared depth.

   SUBSET, not equality, and lib/role-privileges.js records why: appAccess
   injects appmodule read, unioned jobs escalate a shared entity+access to the
   max declared scope, and distinct entities can share ONE Dataverse privilege
   (a role holds one depth per privilege). Equality would fail on all three
   while telling us nothing true. Fails CLOSED on an unreadable role or table.

   The read deliberately does NOT go through sdk.fetchEntityMetadata: that
   returns a projected shape which drops Privileges entirely, so routing through
   it would have silently reported every privilege as unreadable.

2. personas[].jobs[].surfaces[] is checked instead of documentary.

   app-spec.js validated each entry as a non-empty string and stopped;
   spec-lint warned only when the array was EMPTY. So a job could name "My Open
   Work Orders" when no such view existed anywhere in the spec and every gate
   passed.

   lib/surface-resolver.js resolves each entry against the spec's own views,
   forms, pages (key OR name), dashboards, tables and sitemap titles. spec-lint
   WARNS on no match - a warning, never an error, because app-spec.js is loose
   on purpose: a surface may legitimately name an out-of-the-box artifact this
   spec does not author.

   verify adds a `job-surface` rollup - a PURE rollup over checks already
   computed, so it costs no extra reads - reporting a deployed failure as the
   job it broke ("persona P can no longer do job J") rather than only "view X
   is missing".

Both wire into verifySpec's existing READER-GATED seam (the pattern
entityRelationships / commandBar already use), so an existence-only reader
behaves exactly as it did before.

Author: akshaymaloo

.github/workflows/validate-repository-metadata.yml

@@ -10,6 +10,7 @@ jobs:
         runs-on: ubuntu-latest
         env:
             POWER_PLATFORM_SKILLS_TELEMETRY_POWER_PAGES_OPTOUT: "1"
+            POWER_PLATFORM_SKILLS_TELEMETRY_MODEL_APPS_OPTOUT: "1"
         steps:
             - name: checkout
               uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
@@ -36,3 +37,13 @@ jobs:
 
             - name: validate-secure-process-execution
               run: node scripts/validate-secure-process-execution.js
+
+            # This repository is public, and the genpage eval fixtures are captured
+            # agent transcripts that will faithfully record whatever live environment
+            # an eval was run against. Guarding this in CI is the only reliable
+            # control: a pasted transcript looks correct to a reviewer.
+            - name: test-no-real-environments-validator
+              run: node --test scripts/tests/validate-no-real-environments.test.js
+
+            - name: validate-no-real-environments
+              run: node scripts/validate-no-real-environments.js

AGENTS.md

@@ -6,6 +6,52 @@ This file provides guidance to AI Agents when working with code in this reposito
 
 A **plugin marketplace** for Power Platform development by Microsoft. The Open Plugins marketplace manifest (`marketplace.json`) references individual plugins in `plugins/`. Each plugin has its own `AGENTS.md` with plugin-specific guidance.
 
+## This Repo Is PUBLIC — keep internal detail out of it
+
+Everything here — code, docs, commit messages, PR descriptions, and branch names — is world-readable.
+Write for a reader outside Microsoft who cannot see any internal system.
+
+**Do not commit:**
+
+- **Internal hosts, repos or paths** — `*.ghe.com` links, internal repo names, or a path into an
+  internal repo's doc tree. Naming one discloses its existence and its structure.
+- **References to internal documents**, including indexes into them — "spec rank 14",
+  "Group N P1", "(resolves C2, I1)", "see the R1 review". A reader who cannot open the document
+  gets nothing from the pointer, and it advertises the document.
+- **Real environment, tenant or org identifiers** — Dataverse environment names/URLs, tenant GUIDs,
+  subscription ids. These name real infrastructure. In examples use an obvious placeholder
+  (`https://contoso.crm.dynamics.com`, `<envUrl>`); to record that something was live-verified, keep
+  the **claim** and drop the environment ("live-verified", not "live-verified on <envname>").
+- **Internal review process** — who or which model reviewed something, how many rounds, internal
+  finding ids. "Adversarially reviewed" is a useful signal; the rest is internal.
+- **Exploratory design docs for UNBUILT work** — roadmaps, prioritisation, and candid notes on
+  limitations. A design doc for something that **shipped** is fine and often valuable (see
+  `plugins/model-apps/docs/`); a proposal for something that has not is internal.
+- **Pointers to files that are not in the repo** — a citation nobody can resolve is noise at best.
+
+**Fine to commit:** ADO / `AB#` work-item ids (opaque, and `AB#` is the standard Azure Boards ↔
+GitHub link syntax), and the location of a first-party source repo a maintainer needs in order to
+rebuild a vendored bundle. Both carry real "why" context and disclose no content.
+
+**When you must record internal context**, put it in the PR conversation or an internal doc — not in
+a committed file. When editing an existing doc, keep this rule in mind for the lines around your
+change, not just the ones you add.
+
+**CI enforcement (partial).** `node scripts/validate-no-real-environments.js` (wired into the
+`validate-repository-metadata` workflow) fails the build when a real Dataverse host, tenant, or
+previously-removed identifier appears under `plugins/model-apps/**` or `evals/model-apps/**`. It
+matches on *shape* — `org<8 hex>` is what Dataverse auto-generates, so it is rejected even though it
+starts with the otherwise-allowed word `org` — rather than only re-catching known strings. Run it
+locally after touching eval fixtures or any file that quotes an environment URL.
+
+The scan is **scoped to model-apps only**, and this is a real gap rather than an oversight: other
+plugins still carry pre-existing references of this class (for example real `org<8 hex>` orgs cited
+in power-pages provenance comments), so widening the scan today would fail unrelated PRs. Scrub a
+plugin first, then add it to `SCAN_PATHS`. The guard also cannot see the *local part* of a UPN, so
+`firstname.lastname@contoso.onmicrosoft.com` passes — use a role word (`maker@`, `tester@`).
+Captured `pac auth list` transcripts are the most common source of all three; when scrubbing one,
+prefer an **equal-length** placeholder so the fixed-width table stays aligned.
+
 ## Repository Structure
 
 ```

evals/model-apps/app-builder/EVAL_GUIDE.md

@@ -1,7 +1,7 @@
 # App-Builder Offline Structural Eval Harness — Guide
 
 > Companion to `evals/model-apps/genpage/EVAL_GUIDE.md`.  
-> Source of truth: `plugins/model-apps/docs/app-builder-staged-flow-design.md` §13.
+> Source of truth: `plugins/model-apps/docs/app-builder-design.md` §13.
 
 ## What we evaluate
 
@@ -117,5 +117,5 @@ The live tier (`plugins/model-apps/scripts/smoke-eval.js`) is the thin live smok
 ## Cross-links
 
 - Plugin `AGENTS.md` → *Eval Suite*
-- `plugins/model-apps/docs/app-builder-staged-flow-design.md` §13 — structural eval oracles
+- `plugins/model-apps/docs/app-builder-design.md` §13 — structural eval oracles
 - `evals/model-apps/genpage/EVAL_GUIDE.md` — the parallel eval suite for `/genpage`

evals/model-apps/genpage/fixtures/1-account-card-gallery/genpage-plan.md

@@ -16,8 +16,8 @@ D:\Projects\power-platform-skills\plugins\model-apps
 
 ## Environment
 
-- Active Profile: aurora365-user1@auroratstgeo.onmicrosoft.com
-- URL: https://aurorabapenv4ab3f.crm10.dynamics.com/
+- Active Profile: contoso-user001@contosotest1.onmicrosoft.com
+- URL: https://contosobapenv0002.crm10.dynamics.com/
 - App: Sales Hub (12345678-1234-1234-1234-123456789abc)
 - Languages: English (1033) only
 - Solution: Default

evals/model-apps/genpage/fixtures/1-account-card-gallery/workflow-log.md

@@ -12,8 +12,8 @@
 - (Run separately, not chained with &&)
 
 ### Auth check
-- `pac auth list` → active profile aurora365-user1@auroratstgeo.onmicrosoft.com
-- Active environment: https://aurorabapenv4ab3f.crm10.dynamics.com/ (reported to user)
+- `pac auth list` → active profile contoso-user001@contosotest1.onmicrosoft.com
+- Active environment: https://contosobapenv0002.crm10.dynamics.com/ (reported to user)
 
 ### Entity discovery
 - `pac model list-tables --search 'account'` — Account entity detected as existing (exact logical-name match: `account`)

evals/model-apps/genpage/fixtures/11-recruitment-multi-page/genpage-plan.md

@@ -16,8 +16,8 @@ D:\Projects\power-platform-skills\plugins\model-apps
 
 ## Environment
 
-- Active Profile: aurora365-user1@auroratstgeo.onmicrosoft.com
-- URL: https://aurorabapenv4ab3f.crm10.dynamics.com/
+- Active Profile: contoso-user001@contosotest1.onmicrosoft.com
+- URL: https://contosobapenv0002.crm10.dynamics.com/
 - App: Recruitment Hub (44444444-3333-4444-5555-666666666666)
 - Languages: English (1033) only
 - Solution: Default

evals/model-apps/genpage/fixtures/11-recruitment-multi-page/workflow-log.md

@@ -11,8 +11,8 @@
 - `pac help` → PAC CLI Version 2.11.0 (> 2.10.0 verified)
 
 ### Auth check
-- `pac auth list` → active profile aurora365-user1@auroratstgeo.onmicrosoft.com
-- Active environment: https://aurorabapenv4ab3f.crm10.dynamics.com/ (reported to user)
+- `pac auth list` → active profile contoso-user001@contosotest1.onmicrosoft.com
+- Active environment: https://contosobapenv0002.crm10.dynamics.com/ (reported to user)
 
 ### Entity discovery
 - `pac model list-tables --search 'contact,appointment'` — both entities detected as existing

evals/model-apps/genpage/fixtures/11-recruitment-pages-real/README.md

@@ -6,7 +6,7 @@ First real capture taken after the v2.2 planner-spec tightening landed.
 ## Source
 
 Captured from a real `/genpage` session against
-`https://aurorabapenv610b3.crmtest.dynamics.com`. Driven by
+`https://contosobapenv0001.crmtest.dynamics.com`. Driven by
 `claude-sonnet-4-6` under the v2.1 plugin + v2.2 Phase 0.5 manifest generator
 + v2.2 spec-tightening. Working dir was `D:/temp/recruitment-app/`.
 

evals/model-apps/genpage/fixtures/11-recruitment-pages-real/genpage-plan.md

@@ -10,7 +10,7 @@ D:/temp/recruitment-app
 D:/Projects/power-platform-skills/plugins/model-apps
 
 ## Environment
-- URL: https://aurorabapenv610b3.crmtest.dynamics.com
+- URL: https://contosobapenv0001.crmtest.dynamics.com
 - App: create new: Recruitment App
 - Languages: English (1033) only
 - Solution: Crdec34

evals/model-apps/genpage/fixtures/11-recruitment-pages-real/workflow-log.md

@@ -12,14 +12,14 @@ PAC CLI Version 2.11.0+g06bb2eb (.NET 10.0.8)
 
 `pac auth list`
 Index Active Kind      Name User                                          Cloud  Type            Environment       Environment Url
-[1]          UNIVERSAL      akmaloo@microsoft.com                         Public OperatingSystem
-[2]          UNIVERSAL      aurorauser14@capintegration01.onmicrosoft.com Test   User            TMSBAPEnv5ee52    https://tmsbapenv5ee52.crmtest.dynamics.com/
-[3]   *      UNIVERSAL      Aurora365-User1@auroratstgeo.onmicrosoft.com  Test   User            AuroraBAPEnv610b3 https://aurorabapenv610b3.crmtest.dynamics.com/
+[1]          UNIVERSAL      developer@contoso.com                         Public OperatingSystem
+[2]          UNIVERSAL      fabrikamusr1@fabrikamtenant01.onmicrosoft.com Test   User            FabrikamEnv001    https://fabrikamenv001.crmtest.dynamics.com/
+[3]   *      UNIVERSAL      Contoso-User001@contosotest1.onmicrosoft.com  Test   User            ContosoBAPEnv0001 https://contosobapenv0001.crmtest.dynamics.com/
 
-AskUserQuestion: Which environment should be used for the recruitment app? → [3] Aurora365-User1@auroratstgeo.onmicrosoft.com — AuroraBAPEnv610b3 (currently active)
+AskUserQuestion: Which environment should be used for the recruitment app? → [3] Contoso-User001@contosotest1.onmicrosoft.com — ContosoBAPEnv0001 (currently active)
 
 `pac auth select --index 3`
-New default profile: * UNIVERSAL : Aurora365-User1@auroratstgeo.onmicrosoft.com Test https://aurorabapenv610b3.crmtest.dynamics.com/
+New default profile: * UNIVERSAL : Contoso-User001@contosotest1.onmicrosoft.com Test https://contosobapenv0001.crmtest.dynamics.com/
 
 `pac model list-languages`
 Found 1 enabled language(s):
@@ -75,7 +75,7 @@ User response: approved
 - Files written: package.json, genpage.d.ts (features: charts)
 
 ### Decisions
-- Environment: AuroraBAPEnv610b3 (https://aurorabapenv610b3.crmtest.dynamics.com/)
+- Environment: ContosoBAPEnv0001 (https://contosobapenv0001.crmtest.dynamics.com/)
 - PAC CLI: 2.11.0 (> 2.10.0 requirement met)
 - Node: v20.18.2
 - Languages: English only — no localization code needed