From 124c6bda90b850b50a5b8a887772ad9db9da932f Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 12:26:51 +0530 Subject: [PATCH 01/38] plan-alm: convert to strict plan-only planner (execution skills self-maintain the plan) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit plan-alm previously executed the whole deployment from a single "approve and execute" answer, which under autopilot ran an entire unattended deployment with a silently git-captured approver. Since a skill cannot reliably detect it is running unattended, the only robust fix is structural: make NOT executing the default. - plan-alm is now a 4-phase planner (Detect → Gather → Generate → Approve & save). Phases 5–8 (the execute orchestration) are removed; Phase 4 offers Save-approved / Save-draft / Change — there is no "execute" path. Adds PLAN_MODE, PLAN_QUALITY (degraded on auth/discovery failure), decisionsLog, an always-interactive approver capture, and a completeness gate. - Execution skills are self-sufficient: each ALM skill's Phase 0 recommends creating the plan via plan-alm if missing, and its final phase refreshes the plan + prints the next recommended step (refresh-alm-plan-data.js now emits `nextStep`). No auto-chaining — sequencing is user-driven. - Reverts an autopilot-defaults policy + an uncatalogued q2-manual-confirm gate that a prior session added (a skill can't detect autopilot; it also broke the gate lint); keeps the "(Recommended)" Q2 labels + inline recommendation. - Gate catalog, render comment, README, and AGENTS.md updated to match. PR 1 of 4 (stacked): plan-only → EDM → table-discovery → refresh-enforcement. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../power-pages/.claude-plugin/plugin.json | 2 +- plugins/power-pages/AGENTS.md | 8 +- plugins/power-pages/README.md | 16 +- .../power-pages/references/approval-gates.md | 14 +- .../scripts/lib/refresh-alm-plan-data.js | 199 ++++++- .../tests/plan-alm-q2-guardrails.test.js | 74 +++ .../tests/refresh-alm-plan-data.test.js | 193 +++++++ .../power-pages/skills/activate-site/SKILL.md | 14 +- .../skills/configure-env-variables/SKILL.md | 15 +- .../skills/deploy-pipeline/SKILL.md | 8 +- .../skills/export-solution/SKILL.md | 6 +- .../skills/import-solution/SKILL.md | 6 +- plugins/power-pages/skills/plan-alm/SKILL.md | 546 ++++-------------- .../plan-alm/scripts/render-alm-plan.js | 2 +- .../skills/setup-pipeline/SKILL.md | 6 +- .../skills/setup-solution/SKILL.md | 10 +- plugins/power-pages/skills/test-site/SKILL.md | 4 +- 17 files changed, 654 insertions(+), 469 deletions(-) create mode 100644 plugins/power-pages/scripts/tests/plan-alm-q2-guardrails.test.js diff --git a/plugins/power-pages/.claude-plugin/plugin.json b/plugins/power-pages/.claude-plugin/plugin.json index e531ef646..3021445a0 100644 --- a/plugins/power-pages/.claude-plugin/plugin.json +++ b/plugins/power-pages/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "power-pages", - "version": "2.1.0", + "version": "2.2.0", "description": "Create and deploy Power Pages sites using modern development approaches. Supports code sites (SPAs) with React, Angular, Vue, or Astro. Includes ALM orchestration (plan-alm) with a solution-splitting decision tree, per-solution pipelines, Azure Blob asset advisory, manifest schema v2 for multi-solution deployments, and force-link remediation for cross-host pipeline migrations.", "author": { "name": "Microsoft", diff --git a/plugins/power-pages/AGENTS.md b/plugins/power-pages/AGENTS.md index 7ce380da5..9d9ecc10d 100644 --- a/plugins/power-pages/AGENTS.md +++ b/plugins/power-pages/AGENTS.md @@ -118,13 +118,13 @@ skills/ ## ALM intent routing — `plan-alm` is the front door -When the user expresses an **ALM intent** in natural language — *promote this site to {env}, ship to staging, deploy to production, set up CI/CD, move to next environment, push out a release, run the pipeline, export and import to staging* — invoke **`/power-pages:plan-alm` first**, before any individual ALM skill. The orchestrator detects the project state, runs the pre-plan completeness check, asks about promotion strategy, and dispatches to the right skills (`setup-solution`, `setup-pipeline`, `deploy-pipeline`, `activate-site`, `test-site`) in the right order. +When the user expresses an **ALM intent** in natural language — *promote this site to {env}, ship to staging, deploy to production, set up CI/CD, move to next environment, push out a release, run the pipeline, export and import to staging* — invoke **`/power-pages:plan-alm` first**, before any individual ALM skill. `plan-alm` is a **planner**: it detects the project state, runs the pre-plan completeness check, asks about promotion strategy, and writes a rendered HTML plan (whose `steps[]` array is the recommended execution sequence). **It does not deploy anything.** After the user approves the plan, *the user runs* the individual skills (`setup-solution`, `setup-pipeline`, `deploy-pipeline`, or `export-solution`/`import-solution`, plus `activate-site`/`test-site`) in the plan's order. Each detects the approved plan via its Phase 0 gate, proceeds without re-nagging, refreshes the plan on completion, and points the user at the next step — but never auto-chains. This separation keeps `plan-alm` safe to run unattended (no single answer can trigger an irreversible deployment). -**Do not** jump straight to `/power-pages:setup-pipeline`, `/power-pages:deploy-pipeline`, `/power-pages:export-solution`, or `/power-pages:import-solution` in response to an ALM intent. Those are individual building blocks; running them out of order misses the orchestrator's gates (completeness check, host resolution, deployment-strategy selection, post-deploy validation, rendered HTML plan). +**Do not** jump straight to `/power-pages:setup-pipeline`, `/power-pages:deploy-pipeline`, `/power-pages:export-solution`, or `/power-pages:import-solution` in response to an ALM intent. Those are individual building blocks; running them without a plan first misses the planner's analysis (completeness check, host resolution, deployment-strategy selection, size/split decisions, rendered HTML plan). **Skip `plan-alm` only when the user is explicit about the individual skill.** Phrases like *"just run setup-pipeline"*, *"skip planning, just deploy"*, *"I only need to export the solution zip"* are direct invocations — honor them. Anything ambiguous about deployment intent → `plan-alm` first. -`setup-pipeline` and `deploy-pipeline` enforce this with a Phase 0 ALM-plan gate. If a user invokes them directly without a plan, those skills surface the recommendation to run `plan-alm` first (with an "I know what I'm doing" escape hatch). The Phase 0 gate is meant to fail closed — don't bypass it on the user's behalf. +Every ALM execution skill enforces this with a Phase 0 ALM-plan gate. If a user invokes one directly without a plan, the skill recommends running `plan-alm` first (option 1, recommended) with a *"continue without a plan"* escape hatch; choosing to plan runs `plan-alm` (which only plans) and then the skill proceeds. The Phase 0 gate is meant to fail closed — don't bypass it on the user's behalf. ## Plugin Components @@ -160,7 +160,7 @@ User-invocable via `/power-pages:`: - `setup-pipeline`: 7-phase workflow — detect project context (`powerpages.config.json`, `.solution-manifest.json`, `pac env who`, `pac env list`, `RetrieveSetting('DefaultCustomPipelinesHostEnvForTenant')` on dev env to auto-discover host environment), select platform (Power Platform Pipelines = full; GitHub/ADO = coming soon), confirm pipeline configuration with auto-filled values (pipeline name, host env URL, target environments), run preflight checks (Pipelines installed, solution exists, no name conflict), create `deploymentenvironments` records for source + each target (poll `validationstatus` until Succeeded), create `deploymentpipelines` record + `$ref` associate source env (relative path + `@odata.context`) + create `deploymentstages` per target, verify and write `docs/alm/last-pipeline.json` + `docs/pipeline-setup.md` + commit. Uses `references/cicd-pipeline-patterns.md` for all HAR-confirmed API patterns. - `deploy-pipeline`: 8-phase workflow — verify prerequisites (`docs/alm/last-pipeline.json`, az login, host env token), select target stage (from stages in `docs/alm/last-pipeline.json`; warn if last deploy failed), **pre-flight check on the target env's `blockedattachments` setting** via `fix-blocked-attachments.js --dry-run` (Phase 2.5, Power Pages projects only — prompts the user to unblock `.js`/`.css` proactively when they're on the env's blocklist, saving the ~50-75 min wasted import for sites with thousands of bundle chunks; complementary to the reactive Phase 7.6 handler), resolve pipeline info via `RetrieveDeploymentPipelineInfo` (v9.1) to get `SourceDeploymentEnvironmentId` and available artifacts, create `deploymentstageruns` record + call `ValidatePackageAsync` (204) + poll `operation` field until not `200000201` (surface `validationresults` issues), optionally PATCH `deploymentsettingsjson` for env var / connection reference overrides, **final deploy consent gate at Phase 6.0** (explicit `Deploy now / Cancel` `AskUserQuestion` before either `DeployPackageAsync` or the `pac pipeline deploy` fallback — closes a gap where Phase 5 → Phase 6.1 could fire without a final confirmation when validation passes cleanly), call `DeployPackageAsync` + poll `stagerunstatus` until terminal (handle approval gates with user pause), write `docs/alm/last-deploy.json` + present deployment summary. - `force-link-environment`: 6-phase workflow — verify prerequisites (Azure CLI token for the target host, PAC CLI auth) and ground in Microsoft Learn (`custom-host-pipelines#using-force-link…`), identify host env URL (from `docs/alm/last-host-check.json`, `docs/alm/last-pipeline.json`, or user input) and source dev env's BAP env GUID, resolve or create the `deploymentenvironments` record on the new host (re-querying by `environmentid` to recover the record ID when `create-deployment-environment.js` throws on the "already associated" validation failure), require explicit `AskUserQuestion` consent for the destructive cross-host stamp move (makers in the previous host lose pipeline access for this env; previous host's record is left with stale `validationstatus`; reversible by re-running from the previous host), call `scripts/lib/force-link-environment.js` to POST `ManageEnvironmentStamp` + re-poll `validationstatus` until Succeeded, write `docs/alm/last-force-link.json` marker. Auto-fix entry point for Pattern 15 in `references/deployment-error-catalog.md`. -- `plan-alm`: 8-phase orchestrator workflow — detect project state (powerpages.config.json, existing manifests, pac env who), gather ALM strategy via branched question flow (PP Pipelines or Manual export/import path), generate HTML ALM plan (docs/alm-plan.html with pipeline diagram and execution checklist), get user approval, then execute: setup-solution (conditional), setup-pipeline or export-solution (path-dependent), deploy-pipeline or import-solution per stage, finalize with HTML status update and git commit. +- `plan-alm`: 4-phase **planner** workflow — detect project state (powerpages.config.json, existing manifests, pac env who), gather ALM strategy via branched question flow (PP Pipelines or Manual export/import path), generate HTML ALM plan (docs/alm-plan.html with pipeline diagram and a recommended-execution checklist), then save it (Approved or Draft) and commit. **It does not execute any deployment.** The user runs the individual ALM skills afterward — `setup-solution`, `setup-pipeline`/`export-solution`, `deploy-pipeline`/`import-solution`, `activate-site`, `test-site` — each of which detects the plan (Phase 0 gate), proceeds, and refreshes the plan on completion (via `refresh-alm-plan-data.js`, which also reports the next recommended step). This keeps `plan-alm` safe under autopilot: it never triggers an irreversible action. For small mid-cycle changes (one file, one snippet, one site setting) that previously used a separate hotfix solution: instead, run `setup-solution` in sync mode to adopt the modified components into the existing base solution, bump the solution version, and use `deploy-pipeline` to ship. This keeps a single solution lineage (cleaner audit trail, simpler dependency management) and avoids solution sprawl. Power Platform Pipelines computes incremental imports internally, so re-deploying the base after a small fix is fast. diff --git a/plugins/power-pages/README.md b/plugins/power-pages/README.md index 8260eb8e9..b079b863e 100644 --- a/plugins/power-pages/README.md +++ b/plugins/power-pages/README.md @@ -257,15 +257,15 @@ Runs a guided, end-to-end security review of a Power Pages site and consolidates > "Plan how to promote this site to staging and production" -Orchestrator skill that creates an ALM (Application Lifecycle Management) plan for deploying a Power Pages site across environments. Gathers your promotion strategy, target environments, and approval requirements, generates a visual HTML plan, and after your approval executes the plan by calling the right ALM skills in sequence. +Planner skill that creates an ALM (Application Lifecycle Management) plan for deploying a Power Pages site across environments. Gathers your promotion strategy, target environments, and approval requirements, then generates a visual HTML plan for your review and approval. **It does not deploy anything itself** — after you approve the plan, you run the individual ALM skills, which detect the plan and execute the right step in order. - Detects project state (config, manifests, current environment) - Branched flow for Power Platform Pipelines or manual export/import -- Generates `docs/alm-plan.html` for review and approval -- Dispatches to `setup-solution`, `setup-pipeline`, `export-solution`, `deploy-pipeline`, or `import-solution` +- Generates `docs/alm-plan.html` for review and approval (the recommended execution sequence is the plan of record) +- Recommends the skill sequence to run next — `setup-solution`, `setup-pipeline`/`export-solution`, `deploy-pipeline`/`import-solution` — each of which detects this plan, proceeds, and keeps it updated as it runs > [!TIP] -> `/plan-alm` is the front door for any ALM intent. Use it instead of jumping straight to individual ALM skills when you want to deploy to staging, ship to production, or set up CI/CD. +> `/plan-alm` is the front door for any ALM intent — run it first to produce the plan. It plans only; you then run the execution skills it recommends. Use it instead of jumping straight to individual ALM skills when you want to deploy to staging, ship to production, or set up CI/CD. #### `/setup-solution` @@ -443,10 +443,14 @@ A common end-to-end workflow looks like this: 12. /deploy-site → Push final changes live 13. /test-site → Runtime smoke test on the live URL 14. /security-review → Full security review (headers, firewall, scan, permissions) -15. /plan-alm → Plan multi-environment promotion -16. /deploy-pipeline → Promote through staging → production +15. /plan-alm → Plan multi-environment promotion (planning only — produces the plan) +16. /setup-solution → Package the site into a Dataverse solution +17. /setup-pipeline → Set up the Power Platform pipeline +18. /deploy-pipeline → Promote through staging → production (run per stage) ``` +> Steps 16–18 are the execution sequence `/plan-alm` recommends — you run them yourself; each detects the approved plan and keeps it updated. `/plan-alm` never runs them for you. + Steps can be run independently — you don't need to follow this exact order. Each skill checks its own prerequisites and will tell you if something is missing. If something goes wrong, `/diagnose-deployment` pattern-matches deployment errors and `/report-issue` opens a pre-filled GitHub issue. ## Running Without Interruption diff --git a/plugins/power-pages/references/approval-gates.md b/plugins/power-pages/references/approval-gates.md index c4dc67c24..5ada7883c 100644 --- a/plugins/power-pages/references/approval-gates.md +++ b/plugins/power-pages/references/approval-gates.md @@ -256,7 +256,9 @@ Each section lists every `AskUserQuestion` in that skill. Catalog rows are marke --- -### 6.1 `plan-alm` (19 calls; orchestrator) +### 6.1 `plan-alm` (15 calls; planner) + +> `plan-alm` is a **planner** — it produces an approved/draft HTML plan and never executes. The execution gates that used to live in Phases 5–8 (deploy-failure, post-deploy activation, manual export/import checkpoint) now belong to the individual ALM skills the user runs afterward; they are catalogued under those skills' sections, not here. | ID | Kind | Category | Phase | Trigger / question | Cancel leaves | |---|---|---|---|---|---| @@ -273,12 +275,8 @@ Each section lists every `AskUserQuestion` in that skill. Catalog rows are marke | `plan-alm:2.q3-manual` | gate | plan | 2 (Q3 Manual) | *"How many target envs?"* | nothing | | `plan-alm:2.q4-manual-target` | gate | plan | 2 (Q4 Manual per stage) | *"URL for target env {N}?"* | nothing | | `plan-alm:2.q5-manual-type` | gate | plan | 2 (Q5 Manual) | *"Export managed or unmanaged?"* | nothing | -| `plan-alm:2.q6-manual-checkpoint` | gate | plan | 2 (Q6 Manual) | *"Pause between export and import?"* | nothing | -| `plan-alm:4.approve` | gate | plan | 4 | *"Approve and execute / save for later / change something"* | nothing | -| `plan-alm:4.approver-fallback` | not-a-gate | — | 4 | Free-text "approver name" — pure data-gathering | — | -| `plan-alm:7.manual-checkpoint` | gate | progress | 7 (Manual path) | `MANUAL_CHECKPOINT=true` — *"Export done; proceed to import?"* | partial-manifest | -| `plan-alm:7.deploy-failure` | gate | plan | 7 (Step A.1) | deploy-pipeline halted before completing — *"Retry / Skip stage / Exit"*. Fires per failed stage. | nothing | -| `plan-alm:7.activate-step-b` | gate | plan | 7 (Step B) | Post-deploy activation prompt per stage — *"Activate now / skip"* | nothing | +| `plan-alm:4.approve` | gate | plan | 4 | *"Save approved / Save draft / Change something"* — saves the plan; never executes | nothing | +| `plan-alm:4.approver` | not-a-gate | — | 4 | Approver-name capture (option 1 only) — always-on interactive prompt with git/OS-name prefill; data-gathering for the audit trail | — | --- @@ -724,7 +722,7 @@ These need explicit confirmation from the reviewer before SKILL.md edits land. R These are honest unresolved questions — not necessary to answer before v2 lands, but flagged for future tightening: -- **Does `intent` need a sub-category for plan-alm itself?** plan-alm is the orchestrator; it doesn't have a Phase 0 ALM-plan gate (because it *is* the plan). The closest analogue is `plan-alm:1.deferral` (handle `.alm-deferred` marker) and `plan-alm:1.completeness` (completeness check). Both are tagged `progress` in §6.1 — defensible but worth a second look. +- **Does `intent` need a sub-category for plan-alm itself?** plan-alm is the front-door planner; it doesn't have a Phase 0 ALM-plan gate (because it *is* the plan). The closest analogue is `plan-alm:1.deferral` (handle `.alm-deferred` marker) and `plan-alm:1.completeness` (completeness check). Both are tagged `progress` in §6.1 — defensible but worth a second look. - **Should `pause` gates be allowed to auto-resume?** Currently the lint rule would flag any tooling that auto-responds. But if PP Pipelines exposes a polling endpoint that detects approval state, a deterministic auto-resume becomes possible. Worth a future rule extension. - **Telemetry on gate cancellation.** A gate that's cancelled 80% of the time is asking the wrong question. Out of scope for v2; worth instrumenting once §5 lint lands. - **Multi-prompt gates.** Some entries in §6 cover multiple `AskUserQuestion` calls under one marker (e.g., `setup-solution:5.5*` is one logical gate but renders three multiSelect prompts). The lint rule says one marker can cover multiple calls if the catalog row documents it. Worth a more precise rule once we see drift. diff --git a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js index d61719fc8..f58fc44c1 100644 --- a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js +++ b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js @@ -36,6 +36,11 @@ // finalize: // - PLAN_STATUS = "Completed" // +// stdout JSON includes `nextStep: { name, skill } | null` — the first +// still-pending checklist step and the slash command that runs it. Execution +// skills echo this so the user knows the next step to invoke (user-driven +// sequencing — never auto-fired). null when every step is complete. +// // Exit 0 on success (including no-op when planData missing — caller decides). // Exit 1 on argparse / fatal error. @@ -70,6 +75,12 @@ const PHASES = new Set([ 'import-solution', 'activate-site', 'test-site', + // ensure-pipelines-host: host-only update from docs/alm/last-host-check.json. + // Distinct from 'setup-pipeline' (which also ingests last-pipeline.json + flips + // the "Setup pipeline" step) — this runs when the host was resolved/provisioned + // but the pipeline does not exist yet (e.g. the host install crossed a session + // boundary before setup-pipeline ran). + 'ensure-pipelines-host', 'finalize', ]); @@ -81,6 +92,7 @@ function parseArgs(argv) { render: false, rendererPath: null, stageName: null, + reconcile: false, }; for (let i = 0; i < args.length; i++) { if (args[i] === '--projectRoot' && args[i + 1]) out.projectRoot = args[++i]; @@ -88,6 +100,7 @@ function parseArgs(argv) { else if (args[i] === '--render') out.render = true; else if (args[i] === '--rendererPath' && args[i + 1]) out.rendererPath = args[++i]; else if (args[i] === '--stageName' && args[i + 1]) out.stageName = args[++i]; + else if (args[i] === '--reconcile') out.reconcile = true; } return out; } @@ -319,6 +332,25 @@ function refreshSetupPipeline(planData, projectRoot) { return planData; } +// Host-only refresh from docs/alm/last-host-check.json. Used when the Pipelines +// host was resolved/provisioned (ensure-pipelines-host) but the pipeline doesn't +// exist yet — so we update `hostResolution` + drop the NoHost risks WITHOUT +// flipping the "Setup pipeline" step or touching `pipelineMeta` (those belong to +// the later setup-pipeline phase). This is what lets a host install that crossed +// a session boundary still surface in the plan. +function refreshEnsurePipelinesHost(planData, projectRoot) { + const hostCheck = readJson(almPath(projectRoot, 'lastHostCheck')); + if (hostCheck) { + const next = buildHostResolutionFromCheck(hostCheck); + if (next) planData.hostResolution = next; + mirrorHostResolutionSnapshot(planData, projectRoot); + // Drop the pre-run NoHost / *Unbound* / Platform-Host warnings now that the + // host is resolved (same set setup-pipeline clears). + planData.risks = dropResolvedRisks(planData.risks, 'setup-pipeline'); + } + return planData; +} + function refreshDeployPipeline(planData, projectRoot) { // Refresh hostResolution from the most recent host-check (and mirror into // rawDiscovery.hostResolution if the plan was generated with that envelope). @@ -419,8 +451,8 @@ function refreshDeployPipeline(planData, projectRoot) { function refreshTestSite(planData, projectRoot, stageName) { // Stage resolution: explicit --stageName arg wins; falls back to the - // marker's stageName field (test-site writes it when known, e.g. when - // plan-alm orchestrates the call); finally falls back to the FIRST target + // marker's stageName field (test-site writes it when known, e.g. from the + // upstream deploy context); finally falls back to the FIRST target // stage in planData.stages when planData has only one target. Standalone // single-stage test-site invocations work without --stageName via the // last fallback; multi-stage standalone runs need the explicit arg. @@ -690,7 +722,7 @@ function refreshImportSolution(planData, projectRoot, stageName) { // targetEnvironment, importedAt, status, componentResults }. For Manual // path with multiple targets, the file reflects the MOST RECENT import — // not a per-stage history. We resolve the stage label from --stageName - // (passed by plan-alm Phase 7's per-target loop) or by matching + // (passed by import-solution's final-phase refresh) or by matching // docs/alm/last-import.json's targetEnvironment URL against planData.stages[].envUrl. // The result writes into planData.manualImports[stageName] (parallel to // validationRuns[stageName]) so reviewers see per-target outcome on the @@ -854,6 +886,50 @@ function refreshConfigureEnvVariables(planData, projectRoot) { return planData; } +// Map a checklist step name (planData.steps[].name) to the slash command that +// runs it. Single source of truth for the "what runs next" lookup so the 8 ALM +// execution skills don't each re-derive it in SKILL.md prose. Matched by keyword +// because step names carry stage suffixes ("Deploy via pipeline to Staging"). +// `finalize` and any unmatched name return null — there is no user-invocable +// skill for them. +const STEP_TO_SKILL = [ + { test: /\bsetup\s+solution\b/i, skill: '/power-pages:setup-solution' }, + { test: /\bsetup\s+pipeline\b/i, skill: '/power-pages:setup-pipeline' }, + { test: /\bconfigure\s+env(?:ironment)?\s+var/i, skill: '/power-pages:configure-env-variables' }, + { test: /\bexport\b/i, skill: '/power-pages:export-solution' }, + { test: /\bimport\b/i, skill: '/power-pages:import-solution' }, + { test: /\bdeploy\b/i, skill: '/power-pages:deploy-pipeline' }, + { test: /\bactivate\b/i, skill: '/power-pages:activate-site' }, + { test: /\btest\s+site\b/i, skill: '/power-pages:test-site' }, +]; + +function mapStepToSkill(stepName) { + if (typeof stepName !== 'string') return null; + for (const entry of STEP_TO_SKILL) { + if (entry.test.test(stepName)) return entry.skill; + } + return null; +} + +// Compute the next recommended step a user should run: the first steps[] entry +// that is still pending (not completed/failed/in_progress) and not opted-out +// (skip !== true). Returns { name, skill } or null when nothing remains. The +// execution skills echo this after their final-phase refresh so the user is +// pointed at the next skill — without ever auto-firing it (user-driven +// sequencing). `skill` is null when the pending step has no user-invocable +// command (e.g. an internal "Finalize" step), so callers still get the name. +function computeNextStep(planData) { + if (!planData || !Array.isArray(planData.steps)) return null; + for (const step of planData.steps) { + if (!step || typeof step.name !== 'string') continue; + if (step.skip === true) continue; + const status = step.status || 'pending'; + if (status !== 'pending') continue; + return { name: step.name, skill: mapStepToSkill(step.name) }; + } + return null; +} + function applyRefresh(planData, phase, projectRoot, stageName) { switch (phase) { case 'setup-solution': return refreshSetupSolution(planData, projectRoot); @@ -864,11 +940,112 @@ function applyRefresh(planData, phase, projectRoot, stageName) { case 'import-solution': return refreshImportSolution(planData, projectRoot, stageName); case 'activate-site': return refreshActivateSite(planData, projectRoot, stageName); case 'test-site': return refreshTestSite(planData, projectRoot, stageName); + case 'ensure-pipelines-host': return refreshEnsurePipelinesHost(planData, projectRoot); case 'finalize': return refreshFinalize(planData); default: throw new Error('Unknown phase: ' + phase); } } +// Marker key (alm-paths FILE_NAMES) -> the refresh phase that ingests it. Used by +// the reconcile pass. `lastForceLink` is intentionally absent — there is no +// force-link refresh phase. `lastPipeline` maps to setup-pipeline, which also +// ingests lastHostCheck, so a present lastPipeline supersedes the host-only phase. +const MARKER_TO_PHASE = Object.freeze({ + lastPipeline: 'setup-pipeline', + lastHostCheck: 'ensure-pipelines-host', + lastDeploy: 'deploy-pipeline', + lastExport: 'export-solution', + lastImport: 'import-solution', + lastActivate: 'activate-site', + lastTestSite: 'test-site', + // lastEnvVars resolved dynamically (configure-env-variables vs setup-solution). +}); + +// Returns the mtime (ms) of a file, or 0 if it doesn't exist / can't be stat'd. +function mtimeMs(filePath) { + try { + return fs.statSync(filePath).mtimeMs; + } catch { + return 0; + } +} + +// Reconcile: the enforcement backstop. Scans the ALM marker files and, for each +// one that is NEWER than docs/.alm-plan-data.json (i.e. written by a skill whose +// refresh step was skipped), applies the corresponding phase refresh against a +// single loaded planData, writes once, and renders once. Idempotent — a marker +// the plan already reflects (plan newer than marker) is skipped, so the steady +// state is a cheap no-op. Honors the .alm-deferred opt-out and soft-no-ops when +// there is no plan. +function reconcile({ projectRoot, render, rendererPath }) { + if (!projectRoot) throw new Error('--projectRoot is required'); + const dataPath = path.join(projectRoot, 'docs', '.alm-plan-data.json'); + const htmlPath = path.join(projectRoot, 'docs', 'alm-plan.html'); + + // Respect the project-level ALM opt-out. + if (fs.existsSync(path.join(projectRoot, '.alm-deferred'))) { + return { ok: true, reconciled: [], rendered: false, reason: 'deferred' }; + } + if (!fs.existsSync(dataPath)) { + return { ok: false, reconciled: [], rendered: false, reason: 'no-plan' }; + } + + const planMtime = mtimeMs(dataPath); + + // Decide the env-var marker's phase: configure-env-variables when a + // deployment-settings.json is present (its per-stage values), else setup-solution. + const envVarPhase = fs.existsSync(path.join(projectRoot, 'deployment-settings.json')) + ? 'configure-env-variables' + : 'setup-solution'; + + // Collect pending phases (marker newer than the plan), deduped, in a stable order. + const pending = new Set(); + for (const [markerKey, phase] of Object.entries(MARKER_TO_PHASE)) { + if (mtimeMs(almPath(projectRoot, markerKey)) > planMtime) pending.add(phase); + } + // lastHostCheck is covered by setup-pipeline when a pipeline marker is also + // pending — drop the host-only phase to avoid a redundant ingest. + if (pending.has('setup-pipeline')) pending.delete('ensure-pipelines-host'); + if (mtimeMs(almPath(projectRoot, 'lastEnvVars')) > planMtime) pending.add(envVarPhase); + + if (pending.size === 0) { + return { ok: true, reconciled: [], rendered: false }; + } + + // Deterministic application order (source schema first, host/pipeline, then + // per-stage outcomes). Only the phases actually pending are applied. + const ORDER = [ + 'setup-solution', 'configure-env-variables', 'ensure-pipelines-host', + 'setup-pipeline', 'export-solution', 'import-solution', + 'deploy-pipeline', 'activate-site', 'test-site', + ]; + const phases = ORDER.filter((p) => pending.has(p)); + + let planData; + try { + planData = JSON.parse(fs.readFileSync(dataPath, 'utf8')); + } catch (e) { + throw new Error('Could not parse docs/.alm-plan-data.json: ' + e.message); + } + + for (const phase of phases) { + try { + applyRefresh(planData, phase, projectRoot, null); + } catch { + // A single phase failing must not abort the reconcile — keep healing the rest. + } + } + fs.writeFileSync(dataPath, JSON.stringify(planData, null, 2), 'utf8'); + + let rendered = false; + if (render) { + invokeRenderer(findRendererPath(rendererPath), dataPath, htmlPath); + rendered = true; + } + + return { ok: true, reconciled: phases, rendered, nextStep: computeNextStep(planData) }; +} + function findRendererPath(rendererPath) { if (rendererPath) return rendererPath; // The helper lives at scripts/lib/; the renderer at skills/plan-alm/scripts/. @@ -916,15 +1093,20 @@ function refresh({ projectRoot, phase, render, rendererPath, stageName }) { rendered = true; } - return { ok: true, phase, dataPath, htmlPath, rendered }; + // nextStep: the first still-pending checklist step + its skill command, so + // the calling execution skill can tell the user what to run next (user-driven + // sequencing — never auto-fired). null when the plan is fully executed. + const nextStep = computeNextStep(planData); + + return { ok: true, phase, dataPath, htmlPath, rendered, nextStep }; } if (require.main === module) { const args = parseArgs(process.argv); try { - const result = refresh(args); + const result = args.reconcile ? reconcile(args) : refresh(args); process.stdout.write(JSON.stringify(result) + '\n'); - process.exit(result.ok ? 0 : 0); // ok:false is a soft no-op (missing planData) + process.exit(0); // ok:false (missing planData / deferred) is a soft no-op } catch (err) { process.stderr.write('refresh-alm-plan-data: ' + err.message + '\n'); process.exit(1); @@ -933,10 +1115,15 @@ if (require.main === module) { module.exports = { refresh, + reconcile, buildHostResolutionFromCheck, dropResolvedRisks, setStepStatus, backfillEnvVarValuesFromSettings, extractPerStageValues, + computeNextStep, + mapStepToSkill, + STEP_TO_SKILL, + MARKER_TO_PHASE, PHASES, }; diff --git a/plugins/power-pages/scripts/tests/plan-alm-q2-guardrails.test.js b/plugins/power-pages/scripts/tests/plan-alm-q2-guardrails.test.js new file mode 100644 index 000000000..e316d5a1c --- /dev/null +++ b/plugins/power-pages/scripts/tests/plan-alm-q2-guardrails.test.js @@ -0,0 +1,74 @@ +'use strict'; + +/** + * Regression tests guarding the plan-alm Q2 "Strategy Selection" UX. + * + * Context: a prior run picked Manual export/import because + * (a) no option in Q2 was labeled "(Recommended)", and + * (b) the comparison/recommendation text was hidden behind option 4 ("Help me decide"). + * The reader had nothing to anchor on, so it guessed wrong. + * + * Fix: option 1 carries an explicit "(Recommended ...)" label, option 2 is qualified + * as one-off, and the recommendation is surfaced inline in the prompt body. These tests + * fail loudly if any of those guardrails regress. + * + * (Note: an earlier draft also added an "autopilot defaults" policy + a manual-confirm + * gate. Those were reverted — a skill can't reliably detect it is running unattended, + * and the "(Recommended)" label already anchors the choice in attended and unattended + * runs alike. plan-alm is also plan-only, so a wrong Q2 pick produces a reviewable plan, + * not an action.) + */ + +const test = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('fs'); +const path = require('path'); + +const SKILL_MD = path.resolve( + __dirname, + '..', + '..', + 'skills', + 'plan-alm', + 'SKILL.md' +); + +function readSkill() { + return fs.readFileSync(SKILL_MD, 'utf8'); +} + +function extractQ2Section(skill) { + const q2Heading = skill.indexOf('### Q2 — Strategy Selection'); + assert.notEqual(q2Heading, -1, 'Q2 section heading must exist'); + // Q2 section ends at the next "### " heading (e.g. the PP Pipelines Path section) + const nextHeading = skill.indexOf('\n### ', q2Heading + 5); + assert.notEqual(nextHeading, -1, 'A section after Q2 must exist'); + return skill.slice(q2Heading, nextHeading); +} + +test('plan-alm Q2: PP Pipelines option is labeled "(Recommended ...)"', () => { + const q2 = extractQ2Section(readSkill()); + assert.match( + q2, + /Power Platform Pipelines\s*\(Recommended[^)]*\)/i, + 'Q2 option 1 must carry an explicit "(Recommended ...)" marker so the recommendation is in the option label itself, not hidden behind another option.' + ); +}); + +test('plan-alm Q2: Manual option is qualified as "one-off"', () => { + const q2 = extractQ2Section(readSkill()); + assert.match( + q2, + /Manual export\/import\s*\(one-off[^)]*\)/i, + 'Q2 option 2 must include a "(one-off ...)" qualifier so Manual is not treated as equivalent to PP Pipelines for ongoing CI/CD.' + ); +}); + +test('plan-alm Q2: recommendation comparison is shown inline (not only behind option 4)', () => { + const q2 = extractQ2Section(readSkill()); + assert.match( + q2, + /Recommendation:\s*Power Platform Pipelines/i, + 'Q2 must surface the recommendation paragraph inline in the prompt body. Hiding it behind option 4 ("Help me decide") leaves the reader guessing.' + ); +}); diff --git a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js index 265da8564..65428d570 100644 --- a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js +++ b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js @@ -8,8 +8,11 @@ const os = require('os'); const { refresh, + reconcile, buildHostResolutionFromCheck, dropResolvedRisks, + computeNextStep, + mapStepToSkill, } = require('../lib/refresh-alm-plan-data'); function makeProject(t) { @@ -1593,3 +1596,193 @@ test('extractPerStageValues: defensive against null / wrong-type inputs', () => }); assert.deepEqual(result, { y: { Staging: 'v' } }); }); + +// --- nextStep (DRY next-step lookup for user-driven sequencing) --------------- + +test('mapStepToSkill: maps each step-name family to its slash command', () => { + assert.equal(mapStepToSkill('Setup solution'), '/power-pages:setup-solution'); + assert.equal(mapStepToSkill('Setup pipeline'), '/power-pages:setup-pipeline'); + assert.equal(mapStepToSkill('Export solution'), '/power-pages:export-solution'); + assert.equal(mapStepToSkill('Import to Production'), '/power-pages:import-solution'); + assert.equal(mapStepToSkill('Deploy via pipeline to Staging'), '/power-pages:deploy-pipeline'); + assert.equal(mapStepToSkill('Activate site in Staging'), '/power-pages:activate-site'); + assert.equal(mapStepToSkill('Test site in Production'), '/power-pages:test-site'); + assert.equal(mapStepToSkill('Configure environment variables'), '/power-pages:configure-env-variables'); + // No user-invocable skill → null (e.g. an internal finalize step or junk). + assert.equal(mapStepToSkill('Finalize'), null); + assert.equal(mapStepToSkill(''), null); + assert.equal(mapStepToSkill(undefined), null); +}); + +test('computeNextStep: returns the first pending step + its skill', () => { + const next = computeNextStep({ + steps: [ + { name: 'Setup solution', status: 'completed' }, + { name: 'Setup pipeline', status: 'pending' }, + { name: 'Deploy via pipeline to Staging', status: 'pending' }, + ], + }); + assert.deepEqual(next, { name: 'Setup pipeline', skill: '/power-pages:setup-pipeline' }); +}); + +test('computeNextStep: skips skip:true steps', () => { + const next = computeNextStep({ + steps: [ + { name: 'Setup solution', status: 'completed' }, + { name: 'Setup pipeline', status: 'pending', skip: true }, + { name: 'Deploy via pipeline to Staging', status: 'pending' }, + ], + }); + assert.deepEqual(next, { name: 'Deploy via pipeline to Staging', skill: '/power-pages:deploy-pipeline' }); +}); + +test('computeNextStep: skips non-pending statuses (completed/failed/in_progress)', () => { + const next = computeNextStep({ + steps: [ + { name: 'Setup solution', status: 'completed' }, + { name: 'Setup pipeline', status: 'failed' }, + { name: 'Deploy via pipeline to Staging', status: 'in_progress' }, + { name: 'Activate site in Staging', status: 'pending' }, + ], + }); + assert.deepEqual(next, { name: 'Activate site in Staging', skill: '/power-pages:activate-site' }); +}); + +test('computeNextStep: treats a missing status as pending', () => { + const next = computeNextStep({ steps: [{ name: 'Setup solution' }] }); + assert.deepEqual(next, { name: 'Setup solution', skill: '/power-pages:setup-solution' }); +}); + +test('computeNextStep: returns null when every step is complete or there are no steps', () => { + assert.equal(computeNextStep({ steps: [{ name: 'Setup solution', status: 'completed' }] }), null); + assert.equal(computeNextStep({ steps: [] }), null); + assert.equal(computeNextStep({}), null); + assert.equal(computeNextStep(null), null); +}); + +test('refresh: surfaces nextStep in its return value after a phase refresh', (t) => { + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'TestSite', + steps: [ + { name: 'Setup solution', status: 'pending' }, + { name: 'Setup pipeline', status: 'pending' }, + { name: 'Deploy via pipeline to Staging', status: 'pending' }, + ], + }); + // setup-solution refresh flips "Setup solution" → completed, so nextStep is Setup pipeline. + const result = refresh({ projectRoot: root, phase: 'setup-solution', render: false }); + assert.equal(result.ok, true); + assert.deepEqual(result.nextStep, { name: 'Setup pipeline', skill: '/power-pages:setup-pipeline' }); +}); + +test('refresh: nextStep is null once the final pending step completes', (t) => { + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'TestSite', + steps: [ + { name: 'Setup solution', status: 'completed' }, + { name: 'Setup pipeline', status: 'pending' }, + ], + }); + const result = refresh({ projectRoot: root, phase: 'setup-pipeline', render: false }); + assert.equal(result.ok, true); + assert.equal(result.nextStep, null, 'all steps complete → nextStep null'); +}); + +// --- reconcile (the enforcement backstop / auto-heal) ------------------------ + +// Backdate the plan file so a just-written marker is unambiguously "newer". +function backdatePlan(root, secondsAgo = 60) { + const p = path.join(root, 'docs', '.alm-plan-data.json'); + const t = (Date.now() - secondsAgo * 1000) / 1000; + fs.utimesSync(p, t, t); +} + +test('reconcile: heals a skipped refresh — a newer last-deploy.json is ingested', (t) => { + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'T', pipelineMeta: { lastDeploy: null }, steps: [{ name: 'Deploy via pipeline to Staging', status: 'pending' }], + stages: [{ label: 'Staging', envUrl: 'https://stg.crm.dynamics.com/', type: 'target' }], + }); + // Marker written AFTER the plan (skill ran but its refresh step was skipped). + writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { + stageRunId: 'sr1', stageName: 'Staging', status: 'Succeeded', deployedAt: '2026-06-16T00:00:00.000Z', + artifactVersion: '1.0.0.2', componentCount: 118, + }); + backdatePlan(root); + + const result = reconcile({ projectRoot: root, render: false }); + assert.equal(result.ok, true); + assert.deepEqual(result.reconciled, ['deploy-pipeline']); + const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); + assert.equal(planData.pipelineMeta.lastDeploy.status, 'Succeeded'); + assert.equal(planData.pipelineMeta.lastDeploy.componentCount, 118); +}); + +test('reconcile: idempotent no-op when the plan already reflects the markers', (t) => { + const root = makeProject(t); + writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { stageName: 'Staging', status: 'Succeeded' }); + // Plan written AFTER the marker -> already reflected -> nothing pending. + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { SITE_NAME: 'T' }); + const future = (Date.now() + 60 * 1000) / 1000; + fs.utimesSync(path.join(root, 'docs', '.alm-plan-data.json'), future, future); + + const result = reconcile({ projectRoot: root, render: false }); + assert.deepEqual(result.reconciled, []); +}); + +test('reconcile: host-only phase when only last-host-check.json is pending (no pipeline yet)', (t) => { + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'T', + hostResolution: { status: 'NoHost', hostEnvUrl: null }, + pipelineMeta: null, + steps: [{ name: 'Setup pipeline', status: 'pending' }], + }); + writeJson(path.join(root, 'docs', 'alm', 'last-host-check.json'), { + resolutionStatus: 'AvailableUsingCustomHost', + finalHostEnvUrl: 'https://host.crm.dynamics.com/', + hostType: 'custom', + }); + backdatePlan(root); + + const result = reconcile({ projectRoot: root, render: false }); + assert.deepEqual(result.reconciled, ['ensure-pipelines-host']); + const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); + assert.equal(planData.hostResolution.status, 'AvailableUsingCustomHost'); + assert.equal(planData.hostResolution.hostEnvUrl, 'https://host.crm.dynamics.com/'); + assert.equal(planData.pipelineMeta, null, 'host-only phase must NOT fabricate pipelineMeta'); + assert.equal(planData.steps[0].status, 'pending', 'host-only phase must NOT flip the Setup pipeline step'); +}); + +test('reconcile: a pending last-pipeline.json supersedes the host-only phase', (t) => { + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { SITE_NAME: 'T', steps: [{ name: 'Setup pipeline', status: 'pending' }] }); + writeJson(path.join(root, 'docs', 'alm', 'last-host-check.json'), { resolutionStatus: 'AvailableUsingCustomHost', finalHostEnvUrl: 'https://h/' }); + writeJson(path.join(root, 'docs', 'alm', 'last-pipeline.json'), { pipelineId: 'p1', pipelineName: 'My Pipeline', stages: [] }); + backdatePlan(root); + + const result = reconcile({ projectRoot: root, render: false }); + assert.ok(result.reconciled.includes('setup-pipeline')); + assert.ok(!result.reconciled.includes('ensure-pipelines-host'), 'setup-pipeline covers the host — no redundant host-only phase'); +}); + +test('reconcile: honors the .alm-deferred opt-out (no-op)', (t) => { + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { SITE_NAME: 'T' }); + writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { stageName: 'Staging', status: 'Succeeded' }); + fs.writeFileSync(path.join(root, '.alm-deferred'), 'ni-dev — handled separately'); + backdatePlan(root); + + const result = reconcile({ projectRoot: root, render: false }); + assert.equal(result.reconciled.length, 0); + assert.equal(result.reason, 'deferred'); +}); + +test('reconcile: soft no-op when there is no plan', (t) => { + const root = makeProject(t); + const result = reconcile({ projectRoot: root, render: false }); + assert.equal(result.ok, false); + assert.equal(result.reason, 'no-plan'); +}); diff --git a/plugins/power-pages/skills/activate-site/SKILL.md b/plugins/power-pages/skills/activate-site/SKILL.md index 7b87201c8..7476cf187 100644 --- a/plugins/power-pages/skills/activate-site/SKILL.md +++ b/plugins/power-pages/skills/activate-site/SKILL.md @@ -269,10 +269,10 @@ For consumers like the rendered ALM plan (Manual path's per-target Activate step ```bash node -e "require('fs').mkdirSync('docs/alm',{recursive:true})" -# Determine the stage label this activation was for. plan-alm orchestration -# passes it via context (e.g. "Staging", "Production"); standalone invocations -# may leave it null — refreshActivateSite falls back to env-URL matching -# against planData.stages[].envUrl. +# Determine the stage label this activation was for. The upstream ALM context +# (e.g. the user running this after import-solution) supplies it as "Staging"/ +# "Production"; standalone invocations may leave it null — refreshActivateSite +# falls back to env-URL matching against planData.stages[].envUrl. node -e "require('fs').writeFileSync('docs/alm/last-activate.json', JSON.stringify({ stageName: , siteName: '', @@ -304,9 +304,11 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ --render ``` -`{stageNameOrEmpty}` is the Manual-path target stage label (e.g. `Staging`, `Production`) the agent was activating — usually carried in by plan-alm Phase 7 orchestration. Pass an empty string when unknown; `refreshActivateSite` falls back to URL-matching `docs/alm/last-activate.json`'s `environmentUrl` against `planData.stages[].envUrl`, so standalone invocations still get captured. +`{stageNameOrEmpty}` is the Manual-path target stage label (e.g. `Staging`, `Production`) the agent was activating — stated by the user, or inferred from the target env. Pass an empty string when unknown; `refreshActivateSite` falls back to URL-matching `docs/alm/last-activate.json`'s `environmentUrl` against `planData.stages[].envUrl`, so standalone invocations still get captured. -The helper reads `docs/alm/last-activate.json`, writes a per-target entry into `planData.activations[stageName]` (siteUrl, status, activatedAt), and re-renders `docs/alm-plan.html` so the matching `Activate site in {stageName}` checklist step shows an `ACTIVATED` badge with the live site URL inline. When `docs/.alm-plan-data.json` is absent (standalone, not via plan-alm), the helper returns `ok:false` as a soft no-op. +The helper reads `docs/alm/last-activate.json`, writes a per-target entry into `planData.activations[stageName]` (siteUrl, status, activatedAt), and re-renders `docs/alm-plan.html` so the matching `Activate site in {stageName}` checklist step shows an `ACTIVATED` badge with the live site URL inline. When `docs/.alm-plan-data.json` is absent (standalone, not part of an ALM plan), the helper returns `ok:false` as a soft no-op. + +**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill } | null`. When non-null, tell the user: *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."* When `null` or the helper returned `ok:false`, say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. #### 5.3 Suggest Next Steps diff --git a/plugins/power-pages/skills/configure-env-variables/SKILL.md b/plugins/power-pages/skills/configure-env-variables/SKILL.md index e355add2b..683a3d62d 100644 --- a/plugins/power-pages/skills/configure-env-variables/SKILL.md +++ b/plugins/power-pages/skills/configure-env-variables/SKILL.md @@ -80,7 +80,7 @@ The helper returns JSON with `{ exists, deferred, stale, staleness: { reason, de |---|---|---| | Run `/power-pages:plan-alm` first? | ALM plan gate | Yes — run /power-pages:plan-alm now (Recommended), Continue without a plan (advanced — I know what I'm doing), Cancel | -- **Yes (Recommended)** → invoke `/power-pages:plan-alm`. plan-alm's Phase 7 dispatches back into this skill at the appropriate stage with the pre-classified `siteSettings` already passed via `docs/alm/alm-plan-context.json`. +- **Yes (Recommended)** → invoke `/power-pages:plan-alm`. It builds the plan and returns — `plan-alm` is a planner and does not deploy. This skill then re-runs the Phase 0 check (now `exists:true`) and proceeds to Phase 1, picking up the pre-classified `siteSettings` from `docs/alm/alm-plan-context.json`. - **Continue without a plan** → set `BYPASSED_PLAN_GATE = true` and proceed to Phase 1. - **Cancel** → exit cleanly. @@ -482,6 +482,19 @@ Next steps: Follow the skill tracking instructions in the reference to record this skill's usage. Use `--skillName "ConfigureEnvVariables"`. +**7.5b Refresh the ALM plan (if one exists):** + +```bash +node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ + --projectRoot "." \ + --phase configure-env-variables \ + --render +``` + +The helper re-reads `docs/alm/last-env-vars.json` so newly-created definitions appear in `planData.envVars[]`, backfills per-stage values from `deployment-settings.json` into the "Values by Environment" matrix, zeroes `plannedEnvVarCount`, stamps `LAST_SYNC_AT`, and re-renders `docs/alm-plan.html`. When `docs/.alm-plan-data.json` is absent (standalone invocation, not part of an ALM plan), the helper returns `ok:false` as a soft no-op — safe to run unconditionally. + +**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill } | null`. When non-null, tell the user: *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."* When `null` or the helper returned `ok:false`, say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. + ## Key Decision Points (Wait for User) | Phase | Decision | Options | diff --git a/plugins/power-pages/skills/deploy-pipeline/SKILL.md b/plugins/power-pages/skills/deploy-pipeline/SKILL.md index 671deda4f..a686c1d1a 100644 --- a/plugins/power-pages/skills/deploy-pipeline/SKILL.md +++ b/plugins/power-pages/skills/deploy-pipeline/SKILL.md @@ -83,7 +83,7 @@ The helper returns JSON with `{ exists, stale, staleness: { reason, detail }, ge |---|---|---| | Run `/power-pages:plan-alm` first? | ALM plan gate | Yes — run /power-pages:plan-alm now (Recommended), Continue without a plan (advanced — I just want to deploy), Cancel | -- **Yes (Recommended)** → invoke `/power-pages:plan-alm`. plan-alm's Phase 7 dispatches back into this skill at the appropriate stage. +- **Yes (Recommended)** → invoke `/power-pages:plan-alm`. It builds the plan and returns — `plan-alm` is a planner and does not deploy. This skill then re-runs the Phase 0 check (now `exists:true`) and proceeds to Phase 1. - **Continue without a plan** → set `BYPASSED_PLAN_GATE = true` and proceed to Phase 1. The deploy will still work, but env-var per-stage values, activation, and post-deploy validation aren't orchestrated. - **Cancel** → exit cleanly. @@ -942,9 +942,11 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ --render ``` -The helper reads the `docs/alm/last-deploy.json` you just wrote, ingests it into `planData.pipelineMeta.lastDeploy`, drops any pre-deploy "host not yet provisioned" risks, and re-renders `docs/alm-plan.html` so the Pipelines tab shows the actual run state (status, version, component count, activation, site URL). When `docs/.alm-plan-data.json` is absent (the skill was invoked standalone, not via plan-alm), the helper returns `ok:false` as a soft no-op — safe to run unconditionally. +The helper reads the `docs/alm/last-deploy.json` you just wrote, ingests it into `planData.pipelineMeta.lastDeploy`, drops any pre-deploy "host not yet provisioned" risks, and re-renders `docs/alm-plan.html` so the Pipelines tab shows the actual run state (status, version, component count, activation, site URL). When `docs/.alm-plan-data.json` is absent (the skill was invoked standalone, not part of an ALM plan), the helper returns `ok:false` as a soft no-op — safe to run unconditionally. -This step is what makes the rendered plan stay current after a direct `/power-pages:deploy-pipeline` invocation. plan-alm Phase 7 also runs the same refresh as belt-and-suspenders; running it twice is idempotent (same input → same output). +This step is what keeps the rendered plan current — `plan-alm` is a planner and does not refresh the plan itself, so each execution skill owns its own post-run refresh. Running it more than once is idempotent (same input → same output). + +**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill } | null`. When non-null, tell the user: *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."* (For a multi-stage pipeline this is typically the next stage's deploy, or the next stage's activate/test if those are separate steps.) When `null` (all steps done) or the helper returned `ok:false` (no plan), say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. **7.6 Present summary:** diff --git a/plugins/power-pages/skills/export-solution/SKILL.md b/plugins/power-pages/skills/export-solution/SKILL.md index 1e53baea9..5b34bcfb9 100644 --- a/plugins/power-pages/skills/export-solution/SKILL.md +++ b/plugins/power-pages/skills/export-solution/SKILL.md @@ -73,7 +73,7 @@ The helper returns JSON with `{ exists, deferred, stale, staleness: { reason, de |---|---|---| | Run `/power-pages:plan-alm` first? | ALM plan gate | Yes — run /power-pages:plan-alm now (Recommended), Continue without a plan (advanced — I know what I'm doing), Cancel | -- **Yes (Recommended)** → invoke `/power-pages:plan-alm`. plan-alm's Phase 7 dispatches back into this skill at the appropriate stage. +- **Yes (Recommended)** → invoke `/power-pages:plan-alm`. It builds the plan and returns — `plan-alm` is a planner and does not deploy. This skill then re-runs the Phase 0 check (now `exists:true`) and proceeds to Phase 1. - **Continue without a plan** → set `BYPASSED_PLAN_GATE = true` and proceed to Phase 1. - **Cancel** → exit cleanly. @@ -377,7 +377,9 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ --render ``` -Re-renders `docs/alm-plan.html` so any step-status updates the agent made during this skill (`Export solution` → `status-completed`) flow through. When `docs/.alm-plan-data.json` is absent (standalone invocation, not via plan-alm), the helper returns `ok:false` as a soft no-op — safe to run unconditionally. +Re-renders `docs/alm-plan.html` so any step-status updates the agent made during this skill (`Export solution` → `status-completed`) flow through. When `docs/.alm-plan-data.json` is absent (standalone invocation, not part of an ALM plan), the helper returns `ok:false` as a soft no-op — safe to run unconditionally. + +**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill } | null`. When non-null, tell the user: *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."* (Typically: review the exported zip, then run `/power-pages:import-solution` for the first target.) When `null` or the helper returned `ok:false`, say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. ## Key Decision Points (Wait for User) diff --git a/plugins/power-pages/skills/import-solution/SKILL.md b/plugins/power-pages/skills/import-solution/SKILL.md index e4bd9eafd..f3d538b05 100644 --- a/plugins/power-pages/skills/import-solution/SKILL.md +++ b/plugins/power-pages/skills/import-solution/SKILL.md @@ -69,7 +69,7 @@ The helper returns JSON with `{ exists, deferred, stale, staleness: { reason, de |---|---|---| | Run `/power-pages:plan-alm` first? | ALM plan gate | Yes — run /power-pages:plan-alm now (Recommended), Continue without a plan (advanced — I know what I'm doing), Cancel | -- **Yes (Recommended)** → invoke `/power-pages:plan-alm`. plan-alm's Phase 7 dispatches back into this skill at the appropriate stage. +- **Yes (Recommended)** → invoke `/power-pages:plan-alm`. It builds the plan and returns — `plan-alm` is a planner and does not deploy. This skill then re-runs the Phase 0 check (now `exists:true`) and proceeds to Phase 1. - **Continue without a plan** → set `BYPASSED_PLAN_GATE = true` and proceed to Phase 1. - **Cancel** → exit cleanly. @@ -544,10 +544,12 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ --render ``` -`{targetLabel}` is the Manual-path target stage (e.g. `Staging`, `Production`) the just-completed import was for — usually carried in by the orchestrator (plan-alm Phase 7) or derivable from the target env URL. The helper reads `docs/alm/last-import.json`, captures the import outcome (status, version, component count, component failures) into `planData.manualImports[targetLabel]`, and re-renders `docs/alm-plan.html` so the matching `Import to {targetLabel}` checklist step shows an `IMPORTED` / `FAILED` badge with version + component count inline. Subsequent imports to OTHER targets each get their own entry — the renderer surfaces a per-target history rather than overwriting on each call. +`{targetLabel}` is the Manual-path target stage (e.g. `Staging`, `Production`) the just-completed import was for — the user states it, or it is derivable from the target env URL. The helper reads `docs/alm/last-import.json`, captures the import outcome (status, version, component count, component failures) into `planData.manualImports[targetLabel]`, and re-renders `docs/alm-plan.html` so the matching `Import to {targetLabel}` checklist step shows an `IMPORTED` / `FAILED` badge with version + component count inline. Subsequent imports to OTHER targets each get their own entry — the renderer surfaces a per-target history rather than overwriting on each call. If `--stageName` is omitted the helper falls back to matching `docs/alm/last-import.json`'s `targetEnvironment` URL against `planData.stages[].envUrl`. When the match fails (rare — usually a stage-label/env-URL mismatch in planData), the import is captured under a synthetic key so it isn't silently lost; pass `--stageName` explicitly to keep the rendered plan clean. When `docs/.alm-plan-data.json` is absent, the helper returns `ok:false` as a soft no-op. +**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill } | null`. When non-null, tell the user: *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."* (Typically: import to the next target, or activate the site in this target.) When `null` or the helper returned `ok:false`, say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. + ## Key Decision Points (Wait for User) 1. **Phase 1**: Confirm target environment — import is not easily undoable for managed solutions diff --git a/plugins/power-pages/skills/plan-alm/SKILL.md b/plugins/power-pages/skills/plan-alm/SKILL.md index 88e5eccc1..8f4edeb53 100644 --- a/plugins/power-pages/skills/plan-alm/SKILL.md +++ b/plugins/power-pages/skills/plan-alm/SKILL.md @@ -3,9 +3,11 @@ name: plan-alm description: >- Creates an ALM (Application Lifecycle Management) plan for deploying a Power Pages site across environments. Gathers your promotion strategy, target environments, and - approval requirements upfront, generates a visual HTML plan document for review, then - — after your approval — executes the plan by calling setup-solution, setup-pipeline, - export-solution, and deploy-pipeline (or import-solution) in sequence. + approval requirements upfront, then generates a visual HTML plan document for your + review and approval. **plan-alm does not deploy anything itself** — it is a planner. + After you approve the plan, run the individual ALM skills (setup-solution, + setup-pipeline, deploy-pipeline, or export-solution/import-solution); each detects the + approved plan and executes the right step in order, keeping the plan updated as it runs. Use when asked to: "plan my alm", "set up alm", "create deployment plan", "plan my deployments", "help me deploy to multiple environments", "set up promotion strategy", "create cicd plan", "plan site promotion", @@ -20,13 +22,15 @@ model: opus # plan-alm -An 8-phase orchestrator that gathers ALM strategy from the user, generates an HTML deployment plan, gets approval, then executes the plan by calling existing skills in sequence. +A 4-phase **planner** that gathers ALM strategy from the user, generates an HTML deployment plan, and gets approval. **It does not execute anything** — execution is delegated to the individual ALM skills, which the user runs afterward. ## Overview -This skill detects the current project state (existing solution, pipeline), asks targeted questions about the desired promotion strategy (Power Platform Pipelines or Manual export/import), generates a visual `docs/alm-plan.html`, gets user approval, and then invokes `setup-solution`, `setup-pipeline` (or `export-solution`), and `deploy-pipeline` (or `import-solution`) in the correct order. +This skill detects the current project state (existing solution, pipeline), asks targeted questions about the desired promotion strategy (Power Platform Pipelines or Manual export/import), generates a visual `docs/alm-plan.html`, and gets user approval. The four phases are: **Phase 1 — Detect**, **Phase 2 — Gather strategy**, **Phase 3 — Generate plan**, **Phase 4 — Approve & save**. -**Do NOT create tasks at the start** — strategy is unknown until Phase 2 completes. Create all tasks in Phase 3 once the strategy is determined. +**plan-alm never deploys.** The plan's `steps[]` array records the **recommended execution sequence**. After approval, the user invokes the individual skills — `setup-solution`, `setup-pipeline` (or `export-solution`), and `deploy-pipeline` (or `import-solution`) — in that order. Each of those skills detects the approved plan via its Phase 0 gate, proceeds without re-nagging, and refreshes the plan on completion. This separation is deliberate: it keeps `plan-alm` safe to run unattended (e.g. under autopilot) because no single answer can trigger an irreversible deployment. + +**Do NOT create tasks at the start** — strategy is unknown until Phase 2 completes. Create both tasks in Phase 3 once the strategy is determined. --- @@ -59,16 +63,22 @@ Steps: If `deferred === false`, skip this step silently and proceed to step 1. -1. **Resolve the site identity from the local project.** ALM skills are normally invoked from a site-root directory where `pac pages download-code-site` (or a create-site scaffold followed by a deploy) has written `.powerpages-site/website.yml`. That YAML file is the source of truth for `websiteRecordId` and `siteName`. +1. **Resolve the site identity from the local project.** `.powerpages-site/website.yml` is the source of truth for `websiteRecordId` and `siteName`, and it is present for **both** Power Pages site types: + - **Code / SPA sites** — scaffolded by `/power-pages:create-site` and downloaded with `pac pages download-code-site`. These also have a `powerpages.config.json` and SPA source (`src/`, build output in `dist/`/`build/`). + - **Data-model sites (standard and enhanced data model / "EDM")** — downloaded with `pac pages download --modelVersion 1|2`. These have **no** `powerpages.config.json`; instead `.powerpages-site/` holds the config tree (`web-pages/`, `web-templates/`, `content-snippets/`, …) plus a `.powerpages-site/.portalconfig/` manifest pair. There is no local build output. **Resolution order** (first match wins): - 1. **`.powerpages-site/website.yml`** (preferred, present for every deployed site) — read with the `Read` tool and extract: + 1. **`.powerpages-site/website.yml`** (preferred, present for every downloaded/deployed site) — read with the `Read` tool and extract: - `id` field → `websiteRecordId` - `name` field → `siteName` (the file uses short keys; it is `name:`, not `adx_name:`) - 2. **`powerpages.config.json`** (fallback, used during plugin development from this repo root or for sites scaffolded but not yet deployed) — read `siteName` and `websiteRecordId`. + 2. **`powerpages.config.json`** (fallback — code/SPA sites only; used during plugin development from this repo root or for sites scaffolded but not yet deployed) — read `siteName` and `websiteRecordId`. + + **Determine `SITE_TYPE`** (recorded in planData as `siteType`, surfaced in the plan, and used to skip SPA-only assumptions below): + - `data-model` when `.powerpages-site/.portalconfig/` exists, **or** `.powerpages-site/website.yml` resolved while no `powerpages.config.json` is present. + - `code` when `powerpages.config.json` is present. - If neither is found, stop with: - > "No Power Pages site found in the current directory. Run this skill from your site project root (where `.powerpages-site/` exists after `pac pages download-code-site`). If you haven't created the site yet, run `/power-pages:create-site` first." + If neither marker is found, stop with: + > "No Power Pages site found in the current directory. Run this skill from your site project root — that's where `.powerpages-site/` lives after `pac pages download-code-site` (code/SPA site) or `pac pages download --modelVersion 2` (enhanced data-model site). If you haven't created the site yet, run `/power-pages:create-site` first." `environmentUrl` is always re-confirmed from `pac env who` in step 4 — it does not need to come from either source. @@ -96,7 +106,9 @@ Steps: ```bash node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/verify-alm-prerequisites.js" --envUrl "{DEV_ENV_URL}" ``` - Store `.token` as `DEV_TOKEN` and `.userId` as `userId`. If this fails (auth error), set `DEV_TOKEN = null` and continue — contents discovery will be skipped gracefully. + Store `.token` as `DEV_TOKEN` and `.userId` as `userId`. + + **Track plan quality.** Initialize a `PLAN_QUALITY` accumulator to `"complete"` at the start of Phase 1. If this token acquisition fails (auth error), set `DEV_TOKEN = null`, set `PLAN_QUALITY = "degraded"`, and record the cause (e.g. *"dev-environment auth failed — contents/size/host discovery skipped"*) — then continue. Contents discovery is skipped gracefully, but the resulting plan is built on partial inputs; Phase 3 surfaces this as a prominent risk so the user reviews before executing. (There is no execute path to block here — `plan-alm` only plans — but a degraded plan must be visibly flagged.) 7. Discover and classify site settings (if `DEV_TOKEN` is available and `websiteRecordId` is known): @@ -170,6 +182,8 @@ Steps: && mv ./docs/alm/alm-size-estimate.json.tmp ./docs/alm/alm-size-estimate.json ``` When `SOLUTION_DONE = false`, omit `--solutionId`; the estimator's output will include `envVarCountScope: "publisher-prefix"` to signal the wider scope, and the renderer surfaces this caveat in the Env Variables tab so reviewers know the number reflects the tenant view, not a specific solution. `--projectRoot "."` enables the disk cross-check — the estimator walks the local build output (`dist/`, `public-output/`, `build/`, `.output/`) and surfaces `webFilesDiskMeasuredMB`. When that number is much larger than the Dataverse-measured `webFilesAggregateMB`, the estimator flips `truncationSuspected: true` with a warning — file-typed columns whose bytes aren't returned by `$select=content` are the usual cause and the plan should trust the disk number. + +> **`SITE_TYPE = "data-model"` (EDM/standard) sites have no build output**, so the disk cross-check finds no `dist/`/`build/` directory and `webFilesDiskMeasuredMB` stays `null` — this is expected, not a problem. Web files for data-model sites live as records under `.powerpages-site/web-files/` and are measured via the Dataverse query, so the size estimate is still valid; there's simply no SPA bundle on disk to cross-check against. Pass `--projectRoot "."` regardless — it's a harmless no-op for these sites. Then run the decision tree (same tmp-file pattern): ```bash node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/compute-split-plan.js" \ @@ -187,10 +201,13 @@ Steps: Report to the user: ``` Estimated size: {totalSizeMB} MB — components: {count} — tier: {overall tier}. + Tables: {tableCount} — scoped to the site's table permissions ({tableCountScope}). Decision tree result: {splitStrategy} → {N} solutions recommended. Asset advisory: {K} files flagged for Azure Blob externalization. ``` + > **Table count is site-referenced, not publisher-prefix.** The estimator scopes custom tables to the tables the site actually references (its table permissions + datamodel manifest), so a shared/default publisher (`new_`) no longer inflates the count. `tableCountScope` reports how it was scoped: `site-referenced` (table permissions), `manifest-only`, or `unavailable` (no local `.powerpages-site/` signal — table count is 0, never an env-wide dump). When `unavailable`, note that the table-based split signal was skipped. The estimate command already passes `--projectRoot "."`, which supplies the local table permissions. + 10b. **Enumerate environment variable definitions** (runs whenever `DEV_TOKEN` is available — the size estimator gives a count but not per-variable metadata). The renderer's Env Variables tab needs schema name, type, default value, and bound site setting per definition. Without this step, the tab can only show a count-summary note while the size signal and the warning quote a number — three views that don't fully agree. Running this query produces the row-level data so the table renders properly. @@ -313,6 +330,8 @@ Steps: Ask questions in sequence. **Solution is always Q1** — it is the prerequisite for all other steps. Branch after Q2 based on promotion strategy selection. +**Log every major decision.** As each decision is made below (Q1 solution, Q1b split/override, Q2 strategy, Q3 stages/targets, Q4 host, Q5 approval mode, Q5 manual export type), append to a `DECISIONS_LOG` array: `{ field, value, source }` where `source = "default"` when the recommended/auto value was accepted without an active change, or `"explicit"` when the user picked a non-default option. Phase 4 renders a **"Decisions defaulted (please review)"** section from this log so a reviewer can see at a glance which choices were defaults vs. deliberate picks (closes the *"I never agreed to managed export"* gap). This adds no new prompts — it only records what the existing prompts produced. + ### Q1 — Solution Setup (always asked first) **If `SOLUTION_DONE = true`** (manifest found in Phase 1): @@ -404,22 +423,28 @@ Options: ### Q2 — Strategy Selection (always asked) -> 🚦 **Gate (plan · plan-alm:2.q2-strategy):** Pick promotion strategy — PP Pipelines, manual export/import, existing pipeline, or help-me-decide. Branches the rest of the plan. +> 🚦 **Gate (plan · plan-alm:2.q2-strategy):** Pick promotion strategy — PP Pipelines (Recommended), manual export/import, existing pipeline, or help-me-decide. Branches the rest of the plan. + +**Recommendation prelude (shown inline in the prompt body).** Before listing the options, surface the recommendation and any state that affects it, so the user sees it up front instead of having to drill into option 4: + +> **Recommendation: Power Platform Pipelines for ongoing CI/CD** — automated promotion, approval gates, and deployment history in one place. Manual export/import is intended for **one-off migrations** only; repeating it for every change loses the audit trail and silently allows version-skew bugs. +> +> *(Conditional, append when `HOST_RESOLUTION.resolutionStatus` ∈ {`AvailableUsingCustomHost`, `AvailableUsingCustomHostByAdminDefault`, `AvailableUsingPlatformHost`, `AvailableUnboundCustomHost`}):* +> A pipelines host already exists in your tenant at `{HOST_RESOLUTION.finalHostEnvUrl}` (Pipelines v`{HOST_RESOLUTION.pipelinesSolutionVersion}`). PP Pipelines requires **no new infrastructure** for this project. Ask via `AskUserQuestion`: > "How do you want to promote your solution between environments?" Options: -1. **Power Platform Pipelines** — Microsoft's native CI/CD, managed deployments, approval gates -2. **Manual export/import** — export a zip from dev and import directly to each target environment +1. **Power Platform Pipelines (Recommended for ongoing CI/CD)** — Microsoft's native CI/CD, managed deployments, approval gates +2. **Manual export/import (one-off migrations only)** — export a zip from dev and import directly to each target environment 3. **I already have a pipeline set up** — run a deployment now -4. **Help me decide** — show a quick comparison +4. **Help me decide** — show the full comparison again -**If option 4 selected:** Explain: -> "Power Platform Pipelines is recommended for teams and multiple environments — it provides automated promotion, approval gates, and deployment history in one place. Manual export/import is simpler for one-off migrations or when you only need to deploy once. For ongoing CI/CD, choose Power Platform Pipelines." +Record the pick in `DECISIONS_LOG` (`{ field: "strategy", value, source }`). -Then re-ask Q2 with only options 1–3. +**If option 4 selected:** Re-print the recommendation prelude above, then re-ask Q2 with only options 1–3. **If option 3 selected:** Read `docs/alm/last-pipeline.json`, confirm pipeline name and stages, then skip to Phase 3 (generate plan) with `strategy = pp-pipelines`, `PIPELINE_DONE = true`. @@ -545,19 +570,9 @@ Options: 1. Managed — for staging/production (cannot edit in target) 2. Unmanaged — for dev-to-dev (editable in target) -Store as `EXPORT_TYPE`. - - -> 🚦 **Gate (plan · plan-alm:2.q6-manual-checkpoint):** Pause between export and import for review, or proceed automatically. - -**Q6:** Ask via `AskUserQuestion`: -> "Do you want a checkpoint pause between export and import for review?" +Store as `EXPORT_TYPE`, and log the decision (`{ field: "exportType", value: EXPORT_TYPE, source: "default"|"explicit" }`). -Options: -1. Yes — pause after export so I can review the zip before importing -2. No — proceed automatically - -Store as `MANUAL_CHECKPOINT` (`true` or `false`). +> **No checkpoint preference is collected.** `plan-alm` no longer runs the export/import itself, so the old *"pause between export and import?"* question (which only gated execution) is gone. The plan records the recommended sequence — `export-solution` then `import-solution` per target — and the user reviews the zip between those steps when they run them. A standing note to that effect is added to the plan's risks/steps in Phase 3. **Q6 (auto-detect, no question):** Same as PP Pipelines Q6 — check for env var definitions. @@ -565,45 +580,14 @@ Store as `MANUAL_CHECKPOINT` (`true` or `false`). ## Phase 3 — Generate HTML Plan -**Now create all tasks** — strategy is known. - -### Task creation - -**For PP Pipelines path**, create these tasks (in order): +**Now create the two planner tasks** — strategy is known. `plan-alm` is a planner, so it has exactly **two** tasks regardless of path: | # | Subject | activeForm | Description | |---|---------|-----------|-------------| | 1 | Generate ALM plan | Generating ALM plan | Build planData, render docs/alm-plan.html | -| 2 | Approve ALM plan | Awaiting plan approval | Present inline summary, get user confirmation | -| 3 | Setup solution | Setting up solution | Invoke setup-solution skill (conditional) | -| 4 | Setup pipeline | Setting up pipeline | Invoke setup-pipeline skill (conditional) | -| 5..N | Deploy to {stageName} | Deploying to {stageName} | Invoke deploy-pipeline skill for this stage — one task per target stage | -| 5..N+1 | Activate site in {stageName} | Activating site in {stageName} | Check activation status; if Pending/null: `pac env select --environment "{stage.targetEnvironmentUrl}"` then invoke activate-site — one task per target stage | -| 5..N+2 | Test site in {stageName} | Testing site in {stageName} | Invoke /power-pages:test-site against the activated URL; capture pass/fail counts; non-blocking — one task per target stage | -| N+3 | Finalize | Finalizing | Update HTML status, commit, run skill tracking | - -Create one **Deploy to {stageName}** + **Activate site in {stageName}** + **Test site in {stageName}** task triplet for each target stage in `PP_STAGES` (e.g. Staging, Production). - -**For Manual path**, create: - -| # | Subject | activeForm | Description | -|---|---------|-----------|-------------| -| 1 | Generate ALM plan | Generating ALM plan | Build planData, render docs/alm-plan.html | -| 2 | Approve ALM plan | Awaiting plan approval | Present inline summary, get user confirmation | -| 3 | Setup solution | Setting up solution | Invoke setup-solution skill (conditional) | -| 4 | Export solution | Exporting solution | Invoke export-solution skill | -| 5..N | Import to {targetLabel} | Importing solution | Switch PAC CLI context, invoke import-solution (one task per target) | -| N+1 | Activate site in {targetLabel} | Activating site | Check activation status, invoke activate-site if not yet provisioned (one task per target, optional) | -| N+2 | Finalize | Finalizing | Update HTML status, commit, run skill tracking | - -If `SOLUTION_DONE = true`, add `(will skip — already set up)` to the setup-solution task description. -If `PIPELINE_DONE = true` (PP path), add `(will skip — already set up)` to the setup-pipeline task description. +| 2 | Approve & save ALM plan | Awaiting plan approval | Present inline summary, capture approver, save (approved or draft) | -**Activation steps (PP path):** Create a separate **"Activate site in {stageName}"** task for every target stage. After each `deploy-pipeline` invocation succeeds, the activation task for that stage runs immediately — do not wait until all stages are deployed. The planData `steps` array must include one `"Deploy to {stageName}"` + one `"Activate site in {stageName}"` + one `"Test site in {stageName}"` triplet per target stage. Activation and testing happen after every stage deployment — not just Production. - -**Test steps (PP path):** Create a separate **"Test site in {stageName}"** task for every target stage. After each activation completes (or is skipped), the test task runs immediately and is **non-blocking** — `test-site` writes `docs/alm/last-test-site.json`, plan-alm ingests it into `validationRuns[stageName]`, and the rendered HTML's **Validation** tab gets a per-stage sub-tab with categorized findings. Failures do not abort the plan. - -**Activation steps (Manual path):** For the Manual path, create one "Activate site in {targetLabel}" task per target environment. These run after the corresponding import completes. The Manual path does not include automatic test-site invocations — site testing is left to the user after manual deployment. +> **Do not create Setup/Deploy/Activate/Test/Import/Finalize tasks.** Those are *execution* steps performed by the individual ALM skills the user runs after approval — they are not `plan-alm` tasks. The recommended sequence still appears in the plan: the planData `steps[]` array below lists it (all `status: "pending"`), and the rendered HTML's Execution Checklist shows it as the plan of record. Each downstream skill marks its own step complete (via `refresh-alm-plan-data.js`) when the user runs it. Mark task 1 ("Generate ALM plan") as `in_progress`. @@ -614,6 +598,7 @@ Build a `planData` object with all gathered strategy inputs: ```json { "SITE_NAME": "{siteName}", + "siteType": "code | data-model", // from Phase 1 Step 1 — "data-model" for enhanced/standard data-model (EDM) sites (no SPA build output), "code" for SPA sites "GENERATED_AT": "{ISO timestamp}", "STRATEGY": "pp-pipelines | manual", "EXPORT_TYPE": "managed | unmanaged", // PP Pipelines path: always "managed" @@ -621,7 +606,12 @@ Build a `planData` object with all gathered strategy inputs: "HAS_ENV_VARS": true | false, "SOLUTION_DONE": true | false, "PIPELINE_DONE": true | false, - "PLAN_STATUS": "Draft", + "PLAN_STATUS": "Draft", // set to "Approved" or "Draft" in Phase 4; plan-alm never sets "In Execution"/"Completed" + "PLAN_MODE": "draft", // "approved" | "draft" — set in Phase 4. Tooling/downstream skills distinguish an approved plan from a draft. Never "executed" (plan-alm doesn't execute). + "PLAN_QUALITY": "complete", // "complete" | "degraded" — degraded when discovery was incomplete (Phase 1 auth failure or the Phase 3 completeness check failed). Surfaced as a prominent risk. + "decisionsLog": [ // from DECISIONS_LOG (Phase 2) — drives the Phase 4 "Decisions defaulted" section + { "field": "strategy", "value": "pp-pipelines", "source": "explicit" } + ], "LAST_INVOCATION_AT": null, "APPROVED_BY": "", "APPROVAL_DATE": "", @@ -711,7 +701,7 @@ Build a `planData` object with all gathered strategy inputs: `plannedEnvVarCount` is computed from `SITE_SETTINGS_DATA` (Phase 1 Step 7): `(SITE_SETTINGS_DATA.promoteToEnvVar?.length || 0) + (SITE_SETTINGS_DATA.credentialNeedsDecision?.length || 0)`. When `SITE_SETTINGS_DATA` is null (Step 7 query failed), set `plannedEnvVarCount = 0`. The renderer reads this alongside `envVars.length` (existing) and `sizeAnalysis.envVarCount.value` (size-estimator's count) to produce a "N today / +M planned" display in the Overview stat card and Size Analysis signal. -**`validationRuns` block** (PP Pipelines path only — initialize one entry per target stage with value `null`; populated during Phase 7 Step C by ingesting `docs/alm/last-test-site.json` after each stage's test run). The full categorized test report drives the new **Validation** tab in the rendered HTML. Shape per stage: +**`validationRuns` block** (PP Pipelines path only — initialize one entry per target stage with value `null`). plan-alm leaves these `null` at plan time; they are populated later by **`test-site`'s own final-phase refresh** (`refresh-alm-plan-data.js --phase test-site --stageName {stage}`) when the user runs `test-site` for each stage. The full categorized test report drives the **Validation** tab in the rendered HTML. Shape per stage: ```json { @@ -753,9 +743,9 @@ Build a `planData` object with all gathered strategy inputs: } ``` -The shape is identical to `docs/alm/last-test-site.json` written by `test-site` Phase 6.7a — `plan-alm` reads that file verbatim and assigns it to `validationRuns[stageName]`. The renderer maps `runOutcome` to green / yellow / red Outcome badges and produces a sub-tab per stage on the Validation tab. For the Manual path, omit `validationRuns` from planData. +The shape is identical to `docs/alm/last-test-site.json` written by `test-site` Phase 6.7a — the refresh helper reads that file verbatim and assigns it to `validationRuns[stageName]` when `test-site` runs. The renderer maps `runOutcome` to green / yellow / red Outcome badges and produces a sub-tab per stage on the Validation tab. For the Manual path, omit `validationRuns` from planData. -**`pipelineMeta` block** (PP Pipelines path only — read from `docs/alm/last-pipeline.json` and `docs/alm/last-deploy.json` at planData-build time. `null` on fresh plans where no pipeline is configured yet; populated after `setup-pipeline` and refreshed after each `deploy-pipeline` run via the post-deploy re-render in Phase 7). Highlights the pipeline that is actually moving configurations for this project. Shape: +**`pipelineMeta` block** (PP Pipelines path only — read from `docs/alm/last-pipeline.json` and `docs/alm/last-deploy.json` at planData-build time. `null` on fresh plans where no pipeline is configured yet. Later refreshed by `setup-pipeline`'s and `deploy-pipeline`'s own final-phase refresh when the user runs them). Highlights the pipeline that is actually moving configurations for this project. Shape: ```json { @@ -856,6 +846,17 @@ Populate `risks` based on gathered data: - If `HOST_RESOLUTION.status === "MultipleUnboundCustomHosts"`: `{ type: "info", message: HOST_RESOLUTION.candidates.existingCustomHosts.length + " existing Custom Hosts found in tenant. setup-pipeline will prompt for selection." }` - If `HOST_RESOLUTION.status === "PlatformHostExistsUnbound"`: `{ type: "info", message: "Tenant has a Platform Host. Reusing it is the lowest-friction option; creating a Custom Host instead provides better governance for separate-tenant or governed scenarios." }` - If `HOST_RESOLUTION.status === "CannotRedirect"`: `{ type: "warning", message: "CannotRedirect: source env ProjectHostEnvironmentId points at PE but tenant default custom host is set elsewhere. Resolution requires Power Platform admin." }` (Note: Phase 2 Q4 normally blocks plan generation in this state; this is a defensive entry in case the plan is somehow generated.) +- **Manual path** (always, when `STRATEGY = "manual"`): `{ type: "info", message: "Recommended sequence: run /power-pages:export-solution, review the produced zip, then run /power-pages:import-solution for each target. plan-alm does not perform the export/import itself." }` +- **Raw-discovery gaps (#9)** — for each of `rawDiscovery.estimate`, `rawDiscovery.splitPlan`, and (PP path) `rawDiscovery.hostResolution` that is `null` at planData-build time: `{ type: "warning", message: "Discovery for {X} did not run; the related size/split/host decisions in this plan are unverified." }` (substitute `{X}` = "solution size estimate" / "split analysis" / "pipeline host resolution"). + +**Plan completeness check (#10).** Before writing planData, verify the plan rests on real discovery: +- `sizeAnalysis.totalSizeMB` is non-null, +- `stages[]` is non-empty, +- `solutionContents` is populated when a solution is configured (`SOLUTION_DONE = true`). + +If any check fails, set `PLAN_QUALITY = "degraded"` and record which check failed. (This is in addition to the Phase 1 Step 6 auth-failure path that already sets degraded.) + +**Degraded-plan risk (#8)** — if `PLAN_QUALITY === "degraded"` (from the auth-failure path or the completeness check), prepend to `risks`: `{ type: "warning", message: "⚠ PLAN QUALITY: DEGRADED — discovery was incomplete ({cause}). Review carefully before executing; re-run /power-pages:plan-alm after fixing the cause (commonly dev-environment auth) to regenerate a complete plan." }`. Use `type: "warning"` (the renderer styles it). Substitute `{cause}` from the recorded reason(s). Write `planData` to `docs/.alm-plan-data.json` (create `docs/` if it doesn't exist). @@ -867,7 +868,7 @@ node "${CLAUDE_PLUGIN_ROOT}/skills/plan-alm/scripts/render-alm-plan.js" \ --data "/docs/.alm-plan-data.json" ``` -**Keep `docs/.alm-plan-data.json` on disk.** Phases 5 / 6 / 7 / 8 read this file to refresh `hostResolution`, `pipelineMeta`, `validationRuns`, `risks`, and the plan footer after each run step, then re-render `docs/alm-plan.html` so the rendered tabs reflect actual run state (not the pre-run plan). The file is also what `check-alm-plan.js` reads for the Phase 0 ALM-plan gate in `setup-pipeline` / `deploy-pipeline` / `setup-solution` / `export-solution` / `import-solution` / `configure-env-variables` — deleting it makes every downstream skill think no plan exists. Earlier guidance to delete this file after the initial render was incorrect and caused the Pipelines tab + risks list to stay frozen at pre-run state for the lifetime of the plan. +**Keep `docs/.alm-plan-data.json` on disk — never delete it.** Two consumers depend on it after `plan-alm` exits: (1) the **execution skills' final-phase refresh** (`refresh-alm-plan-data.js`) reads it to update `hostResolution`, `pipelineMeta`, `validationRuns`, `risks`, `steps[]` status, and the footer as each skill runs, then re-renders `docs/alm-plan.html` so the rendered tabs reflect actual run state (not the pre-run plan); (2) `check-alm-plan.js` reads it for the Phase 0 ALM-plan gate in `setup-pipeline` / `deploy-pipeline` / `setup-solution` / `export-solution` / `import-solution` / `configure-env-variables` — deleting it makes every downstream skill think no plan exists (and fire its no-plan gate). Earlier guidance to delete this file after the initial render was incorrect and caused the Pipelines tab + risks list to stay frozen at pre-run state for the lifetime of the plan. Write `docs/alm/alm-plan-context.json` (persists so `setup-solution` can read it): ```json @@ -930,9 +931,11 @@ Mark task 1 as `completed`. --- -## Phase 4 — Present Plan and Get Approval +## Phase 4 — Present Plan, Approve & Save -Mark task 2 ("Approve ALM plan") as `in_progress`. +Mark task 2 ("Approve & save ALM plan") as `in_progress`. + +**`plan-alm` stops here.** This phase saves the plan (approved or draft) and exits. It does **not** invoke `setup-solution`, `setup-pipeline`, `deploy-pipeline`, `export-solution`, or `import-solution`. The user runs those afterward. Present a concise inline Markdown summary: @@ -944,377 +947,79 @@ Present a concise inline Markdown summary: **Approval gates:** {description from PP_APPROVAL_MODE, or "N/A — manual path"} **Solution export:** {Managed / Unmanaged} **Pipeline host:** {hostEnvUrl} ({status}) — *(PP Pipelines path only; when `WILL_ENSURE_HOST = true`, render as `Will be ensured during setup-pipeline ({status})` instead)* +**Plan quality:** {complete / ⚠ DEGRADED — } + +**Decisions defaulted (please review):** +{For each DECISIONS_LOG entry, one line: "- {field}: {value} ({default} or {your pick})". Tag source:"default" as (default) and source:"explicit" as (your pick). If every decision was explicit, write "- (none — every choice was an explicit pick)".} -**Steps that will run:** -- [ ] Setup solution {(SKIP — already set up) if SOLUTION_DONE} -- [ ] Setup pipeline {(SKIP — already set up) if PIPELINE_DONE} {(PP path only)} -- [ ] Export solution {(manual path only)} -- [ ] Import to {targetLabel} × {N} {(manual path only)} -- [ ] Deploy via pipeline {(PP path only)} +**Recommended execution sequence (you run these after approval):** +- [ ] /power-pages:setup-solution {(SKIP — already set up) if SOLUTION_DONE} +- [ ] /power-pages:setup-pipeline {(SKIP — already set up) if PIPELINE_DONE} {(PP path only)} +- [ ] /power-pages:export-solution {(manual path only)} +- [ ] /power-pages:deploy-pipeline — per stage {(PP path only)} +- [ ] /power-pages:import-solution — per target {(manual path only)} Full plan written to: docs/alm-plan.html ``` -> 🚦 **Gate (plan · plan-alm:4.approve):** The capstone — user approves the rendered HTML plan before plan-alm dispatches any downstream skill. Save-for-later writes the plan but exits without execution. +> 🚦 **Gate (plan · plan-alm:4.approve):** The capstone — user saves the rendered HTML plan. `plan-alm` never executes; this gate only chooses whether the plan is saved **Approved** (ready for the user to run the execution skills) or **Draft**, or sends the user back to revise. No downstream skill is dispatched here. Ask via `AskUserQuestion`: -> "Does this ALM plan look correct?" +> "How would you like to save this ALM plan?" Options: -1. **Approve and execute the plan** -2. **Save plan but execute manually later** +1. **Save plan — approved, ready to execute** (saves as Approved; you run the execution skills next) +2. **Save plan as draft** (saves as Draft; re-run plan-alm to approve later) 3. **I want to change something** — go back to questions - **If option 3:** Re-run Phase 2 (ask which section to change, then re-gather those answers). Regenerate the plan (repeat Phase 3). Re-present for approval. -- **If option 2:** Capture the approver (see below), stamp `` and `` in the HTML, then update HTML plan footer `plan-status` span to "Approved — Deferred" via `Edit` tool. Commit `docs/alm-plan.html` with message `"Add ALM plan for {siteName} (deferred)"`. Show next steps for manual execution. Mark task 2 as `completed`. Exit the skill. -- **If option 1:** Capture the approver (see below), stamp the HTML, then update `` to "In Execution" via `Edit` tool. **Also update `docs/.alm-plan-data.json`** — set `PLAN_STATUS: "In Execution"` and write `LAST_INVOCATION_AT: ""` (now). This is the signal `check-alm-plan.js` reads to set `inExecution.status === "active"` so the downstream skills' Phase 0 gates skip silently. `check-alm-plan.js` will refresh `LAST_INVOCATION_AT` on every subsequent invocation that finds the plan in execution, so the chain stays alive even across multi-hour deploys. Mark task 2 as `completed`. +- **If option 1 (approved):** Capture the approver (see below). Stamp `` / `` in the HTML and set `` text to `Approved` via `Edit`. **Update `docs/.alm-plan-data.json`**: set `PLAN_STATUS: "Approved"` and `PLAN_MODE: "approved"`. Then run the finalize steps below (skill tracking + commit), print the next-steps guidance, mark task 2 `completed`, and **exit**. +- **If option 2 (draft):** Do **not** capture an approver. Set `` text to `Draft` via `Edit`. Update `docs/.alm-plan-data.json`: `PLAN_STATUS: "Draft"`, `PLAN_MODE: "draft"`. Run the finalize steps below (commit only — skip skill tracking or run it, your choice; commit message `"Add ALM plan for {siteName} (draft)"`), tell the user to re-run `/power-pages:plan-alm` when ready to approve, mark task 2 `completed`, and **exit**. -**Capturing the approver (both options 1 and 2):** +**Capturing the approver (option 1 only) — always interactive (#1):** -Capture the name silently using git, falling back to the OS user: +Never auto-apply a name silently. First compute a **prefill suggestion** from git / OS user: ```bash node -e "const {execSync}=require('child_process');let n='';try{n=execSync('git config user.name',{encoding:'utf8'}).trim();}catch{};if(!n){n=process.env.USER||process.env.USERNAME||'';}process.stdout.write(n);" ``` - + -Store the output as `APPROVER`. If `APPROVER` is empty (no git config, no USER env var), ask via `AskUserQuestion`: +Then **always** ask via `AskUserQuestion` (even when the suggestion is non-empty) so a human actively confirms the approver recorded in the audit trail: -> "Who is approving this plan? (needed for the audit trail in docs/alm-plan.html)" +> "Who is approving this plan? (recorded in the audit trail in docs/alm-plan.html)" > -> Options: 1. *{current system user from `whoami`}* · 2. Other (enter name) +> Options: 1. *{suggested name from git/OS, if any}* · 2. Other (enter name) -Once `APPROVER` is known, use `Edit` to replace the empty/placeholder value in `docs/alm-plan.html`: +If the command returned an empty string, present only option 2 (free-text). Store the confirmed result as `APPROVER`, then use `Edit` to replace the spans in `docs/alm-plan.html`: - Find `` (or `` / `__APPROVED_BY__`) and replace its inner text with `APPROVER`. - Find `` and replace its inner text with the current ISO timestamp. Both spans are guaranteed to exist in the template — there is exactly one of each in the "Execution Checklist" tab footer. ---- - -## Phase 5 — Execute: setup-solution (conditional) - -**If `SOLUTION_DONE = true`:** -Mark the "Setup solution" task as `completed` with description "Skipped — solution already configured". Update the HTML checklist step for "Setup solution" to `status-skipped` via `Edit` tool. Skip to Phase 6. - -**If `SOLUTION_DONE = false`:** -Mark the "Setup solution" task as `in_progress`. Update the HTML checklist step to `status-in-progress` via `Edit` tool. - -Invoke the skill: -``` -/power-pages:setup-solution -``` - -After completion: mark the task as `completed`. Update the HTML checklist step to `status-completed` via `Edit` tool. - ---- - -## Phase 6 — Execute: setup-pipeline OR export-solution - -### PP Pipelines path - -**If `PIPELINE_DONE = true`:** -Mark the "Setup pipeline" task as `completed` with description "Skipped — pipeline already configured". Update HTML checklist step to `status-skipped`. Skip to Phase 7. - -**If `PIPELINE_DONE = false`:** -Mark the "Setup pipeline" task as `in_progress`. Update HTML checklist step to `status-in-progress`. - -Invoke the skill: -``` -/power-pages:setup-pipeline -``` - -After completion: mark task as `completed`. Update HTML checklist step to `status-completed`. Then run the post-run plan refresh — this is **not optional**; without it the Pipelines tab stays at "Will be ensured during setup-pipeline" and the risks list keeps surfacing the pre-run NoHost warning even though the host now exists: - -```bash -node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ - --projectRoot "." \ - --phase setup-pipeline \ - --render -``` - -This (a) reads `docs/alm/last-host-check.json` and rewrites `planData.hostResolution` to the post-run state (status flips from `NoHost` to `AvailableUsingCustomHost`, all the `willEnsure*` / `willProvision*` / `chosenEnvUrl` flags clear), (b) reads `docs/alm/last-pipeline.json` and populates `planData.pipelineMeta` with the actual pipeline name + ID + host URL + stages (no `lastDeploy` yet — that fills in after Phase 7 Step A), (c) drops resolved entries from `planData.risks` (NoHost / *Unbound* / Platform-Host warnings), then (d) re-renders `docs/alm-plan.html`. If the helper exits with `ok:false`, surface the reason — the most likely cause is that `docs/.alm-plan-data.json` is missing (Phase 3 must have written it; the file should never be deleted between phases). - -### Manual path - -Mark the "Export solution" task as `in_progress`. Update HTML checklist step to `status-in-progress`. - -Invoke the skill: -``` -/power-pages:export-solution -``` - -After completion: mark task as `completed`. Update HTML checklist step to `status-completed`. - -**Refresh the plan after export.** Run the helper (export-solution self-refreshes too — this is belt-and-suspenders): - -```bash -node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ - --projectRoot "." \ - --phase export-solution \ - --render -``` - -The helper re-renders `docs/alm-plan.html` so the Export-step status update is visible to reviewers immediately. - - -> 🚦 **Gate (progress · plan-alm:7.manual-checkpoint):** Manual path pause between export and import — user reviews zip before import proceeds. Defer-now exits cleanly leaving the zip. - -**If `MANUAL_CHECKPOINT = true`:** Ask via `AskUserQuestion`: -> "Export complete. Review the solution zip at `{zipPath}` before importing. Ready to proceed with import?" - -Options: -1. Yes, proceed with import -2. Stop here — I'll import manually later - -If option 2: update HTML plan footer to "Approved — Deferred (paused after export)". Commit `docs/alm-plan.html`. Exit. - ---- - -## Phase 7 — Execute: Deploy - -### PP Pipelines path - -**For each target stage in `PP_STAGES` (e.g. Staging, then Production), run this loop:** - -**Step A — Deploy:** -Mark the "Deploy to {stageName}" task as `in_progress`. Update HTML checklist step to `status-in-progress`. - -Invoke the skill: -``` -/power-pages:deploy-pipeline -``` - -**Halt-on-failure check (Step A.1).** After `deploy-pipeline` returns, read `docs/alm/last-deploy.json` to see what actually happened. **Do NOT unconditionally mark the deploy task `completed`** — `deploy-pipeline` can halt at many points (validation failure, Phase 3.6 batch-validation-failed, blocked-attachments cancel, user-cancel at Phase 6.0 consent gate, mid-deploy `AttachmentBlocked`, etc.) and each writes a marker with a status field. Proceeding to Step B / the next stage on a failed deploy wastes time and produces confusing output (activate-site against a target that didn't receive the solution). - -```bash -node -e "try { const d = require('./docs/alm/last-deploy.json'); const gaps = Array.isArray(d.knownGaps) ? d.knownGaps : []; process.stdout.write(JSON.stringify({status: d.status, stageName: d.stageName, knownGaps: gaps, knownGapsCount: gaps.length})) } catch (e) { process.stdout.write(JSON.stringify({status: 'NoMarker', knownGaps: [], knownGapsCount: 0, error: e.message})) }" -``` - -The extractor returns `knownGaps: []` when the field is absent or malformed — the agent branches on `knownGapsCount > 0` rather than null/undefined checks, which avoids the ambiguity between "field missing" and "field present but empty array". - -Branch on the parsed status. **Check `knownGaps` presence first** because Phase 3.6.5's "deploy succeeded subset" path writes `status: "Succeeded"` alongside a populated `knownGaps[]` — a naive `status === "Succeeded"` branch would miss the gap signal: - -- **`knownGaps` is a non-empty array** (regardless of `status`): A subset of solutions deployed (Phase 3.6.5 "deploy succeeded subset" path) but at least one was intentionally skipped. Mark the deploy task `completed-with-gaps` (or `completed` with a note in the rendered plan). Show the user the recorded `knownGaps` and confirm they want to proceed to Step B for the partial deploy. Run the refresh-plan step regardless. -- **`status === "Succeeded"`** (and no `knownGaps`): Mark deploy task `completed`. Update HTML checklist step to `status-completed`. Proceed to the refresh-plan step below, then Step B. -- **Any other status** (`"Failed"`, `"ValidationFailed"`, `"Canceled"`, `"PendingApproval"` — user cancelled the approval pause, `"Unknown"` — poll timed out, `"NoMarker"` — the file didn't exist or was unparseable, or any unrecognised value): Mark deploy task `failed`. Update HTML checklist step to `status-failed`. Run the refresh-plan step so the rendered plan shows the failure surface. Then fire the deploy-failure gate below. The catch-all clause covers future deploy-pipeline status values without requiring this skill to be updated in lockstep. - - -> 🚦 **Gate (plan · plan-alm:7.deploy-failure):** deploy-pipeline halted before completing the deploy to `{stageName}`. Caller decides: retry this stage (re-invoke deploy-pipeline — Dataverse import idempotency makes already-succeeded solutions in a multi-solution loop a no-op), skip this stage and continue to the next (advanced — leaves a stage gap; the rendered plan will show it), or exit the orchestration. Cancel exits the loop without touching subsequent stages. - -Use `AskUserQuestion`: - - | Question | Header | Options | - |---|---|---| - | Deploy to `{stageName}` did not complete (`{status}`). What now? | Deploy failure | Retry this stage, Skip to next stage (advanced — leaves a gap), Exit orchestration | - - - **Retry**: re-invoke `/power-pages:deploy-pipeline` for this stage. Re-run the halt-on-failure check on the retry's marker. Loop up to the user's tolerance. - - **Skip to next stage**: leave the deploy task `failed`, do NOT run Step B / Step C for this stage, continue the outer `PP_STAGES` loop. The rendered plan shows a stage gap; the user owns the consequence. - - **Exit orchestration**: stop the skill. The rendered plan reflects current state (succeeded stages + the failed one). Re-invoking `/power-pages:plan-alm` later resumes from Phase 7 against the existing plan. - -**Refresh the plan after each stage's deploy.** Run the helper (regardless of success/failure — the refresh ingests both outcomes): - -```bash -node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ - --projectRoot "." \ - --phase deploy-pipeline \ - --render -``` - -The helper reads `docs/alm/last-deploy.json`, populates `planData.pipelineMeta.lastDeploy` (status, stageName, deployedAt, artifactVersion, componentCount, activationStatus, siteUrl), and re-renders `docs/alm-plan.html`. The Pipelines tab now shows the actual pipeline name + ACTIVE chip + last-run footer (Succeeded / Failed status + version + component count + stage label). Cheap; runs once per stage in the deploy loop. The `setStepStatus` cross-cutting behavior already maps failure-status markers to `failed` step status so the checklist surfaces what actually happened — Step A.1's halt-on-failure gate just adds the user-facing prompt that was previously missing. - -**Step B — Activate (immediately after deploy for this stage, only when Step A succeeded or completed-with-gaps):** -Mark the "Activate site in {stageName}" task as `in_progress`. Update HTML checklist step to `status-in-progress`. - -Read `docs/alm/last-deploy.json` to check whether activation already happened inside `deploy-pipeline`: -```bash -node -e "const d=require('./docs/alm/last-deploy.json'); process.stdout.write(JSON.stringify({activationStatus: d.activationStatus, siteUrl: d.siteUrl}))" -``` - -- `activationStatus === "Activated"`: site is live. Mark task `completed`. Update checklist step to `status-completed`. Show site URL. -- `activationStatus === "Pending"` or `null`: activation was deferred or didn't run inside `deploy-pipeline` Phase 7.7. Re-prompt the user. - - **Switch PAC CLI to the target environment first** — the activation flow reads the current env from `pac auth who` and looks up the site's record via `pac pages list` in that env. If PAC is still pointing at dev (or at the previous stage's env), activation would target the wrong environment. This mirrors the explicit switch pattern used by `deploy-pipeline` Phase 7.7 (the helper-and-switch-back pair around `check-activation-status.js`); plan-alm's Step C also switches PAC back at the end of the stage loop, but the forward switch BEFORE activate-site must happen here. - - ```bash - pac env select --environment "{stage.targetEnvironmentUrl}" - ``` - - Where `{stage.targetEnvironmentUrl}` is the current stage's target environment URL — pulled from `planData.stages[]` (matched by `stage.label === stageName`, then `stage.envUrl`) or equivalently from `docs/alm/last-pipeline.json` `stages[].targetEnvironmentUrl`. Do NOT skip this switch even when only one target stage exists — by the time Step B runs, PAC may already have been switched back to dev by the end of `deploy-pipeline`. - - - > 🚦 **Gate (plan · plan-alm:7.activate-step-b):** Per-stage post-deploy activation prompt — Step B second-chance when deploy-pipeline Phase 7.7 was skipped or errored. **Fires PER STAGE in the multi-stage execution loop.** Two stages (Staging + Production) where both need activation = two prompts. The "Yes, activate now" answer for Staging does NOT cover Production — each stage's activation is a distinct decision (different URLs, different audiences, different go-live timing). Do NOT batch. - - Then ask via `AskUserQuestion`: - - > "**{siteName}** was deployed to **{stageName}** successfully. The site is not yet activated (not publicly accessible). Activate it now?" - - Options: - 1. **Yes, activate now** — invoke `/power-pages:activate-site`. After it completes, mark task `completed`, update checklist step to `status-completed`. - 2. **No, skip for now** — mark task `skipped`, update checklist step to `status-skipped`. - - > **Why this gap mattered.** In the happy path, `deploy-pipeline` Phase 7.7 already activated the site (or got an explicit "No") and patched `activationStatus` into `docs/alm/last-deploy.json`. Step B sees `"Activated"` and skips. The gap shows up only when 7.7 was answered "No, I'll activate later" OR when 7.7 was skipped (e.g. activation-status check returned `error`) — in those cases Step B is the second-chance prompt, and it MUST do the PAC switch itself because 7.7's switch-back at the end of Phase 7.7 already returned PAC to dev. For a multi-stage plan this is especially important: on the Production iteration, PAC is back at dev after Staging's Step C; Step B for Production needs an explicit switch to the Production env URL, not Staging's. - -**Step C — Test site (immediately after activate for this stage):** -Mark the "Test site in {stageName}" task as `in_progress`. Update HTML checklist step to `status-in-progress`. - -Determine the URL to test: -- Prefer `siteUrl` from `docs/alm/last-deploy.json` (written by `deploy-pipeline`). -- If absent or empty, fall back to the URL returned by the most recent `activate-site` invocation for this stage. -- If both are unavailable (activation was skipped, no URL captured), mark the task `skipped` and set `validationRuns[stageName] = null`. Update checklist step to `status-skipped`. - -If a URL is available, invoke the skill (forwarding the URL as the argument): -``` -/power-pages:test-site --siteUrl {activatedUrl} -``` - -When `test-site` completes, ingest its `docs/alm/last-test-site.json` marker into `validationRuns[stageName]` and re-render via the same refresh helper used by other phases: - -```bash -node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ - --projectRoot "." \ - --phase test-site \ - --stageName "{stageName}" \ - --render -``` - -The helper reads `docs/alm/last-test-site.json`, populates `planData.validationRuns[{stageName}]` with the test outcome (runOutcome, summary counts, categories), and re-renders `docs/alm-plan.html` so the Validation tab updates immediately. - -`runOutcome` is set by `test-site` itself (see Phase 6.7a in the test-site skill): `"failed"` when any critical/high failure exists, `"passed-with-warnings"` for any non-critical failure or console errors, `"passed"` otherwise. Trust the value as written — do not re-compute. - -**Decision rule (non-blocking):** Regardless of `runOutcome`, mark the task `completed` and continue to the next stage. Failures are diagnostic, not gating — the plan does not abort. - -Update the HTML checklist step: -- `runOutcome === "failed"` → `status-warning` (NEW status — yellow). -- otherwise → `status-completed`. - -**Checklist substep rendering** (the renderer handles this automatically once `validationRuns` is populated and the planData re-rendered): every `Test site in {stageName}` step gets an inline substep showing the test-result badge (`PASSED` / `WARNINGS` / `FAILED`), the tested URL, the `pass / fail / skip` summary line, and a "View details →" link that jumps to the Validation tab. Every `Deploy via pipeline to {stageName}` and `Activate site in {stageName}` step also gets a `Target: ` substep so reviewers see the target env without leaving the Execution tab. The renderer derives env URLs from `data.stages[].envUrl` (matched by trailing stage label) — keep stage labels consistent across `data.stages` and `data.steps`. - -After handling activation and testing, switch PAC CLI back to the dev environment: -```bash -pac env select --environment "{devEnvUrl}" -``` - -**Then repeat Step A + B + C for the next stage** (if any). - -### Manual path (one import per target environment) - -For each entry in `MANUAL_TARGETS`: - -1. Mark the "Import to {targetLabel}" task as `in_progress`. Update the corresponding HTML checklist step to `status-in-progress`. - -2. Switch the PAC CLI context to the target environment: - ```bash - pac env select --environment "{targetEnvUrl}" - ``` - -3. Invoke the skill: - ``` - /power-pages:import-solution - ``` - -4. After completion: mark the task as `completed`. Update the HTML checklist step to `status-completed`. - -5. **Refresh the plan after the import.** Run the helper (import-solution self-refreshes too — this is belt-and-suspenders): +**Finalize (both save options):** +1. **Skill tracking** (option 1; optional for draft): + > Reference: `${CLAUDE_PLUGIN_ROOT}/references/skill-tracking-reference.md` ```bash - node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ - --projectRoot "." \ - --phase import-solution \ - --stageName "{targetLabel}" \ - --render + node "${CLAUDE_PLUGIN_ROOT}/scripts/update-skill-tracking.js" \ + --projectRoot "." --skillName "PlanAlm" --authoringTool "ClaudeCode" ``` - - `{targetLabel}` is the current iteration's target (e.g. `Staging`, `Production`). The helper reads `docs/alm/last-import.json`, writes a per-target entry into `planData.manualImports[targetLabel]` (status, version, component count, failures), and re-renders `docs/alm-plan.html`. The matching `Import to {targetLabel}` checklist step picks up an `IMPORTED` (or `FAILED`) badge with import details inline — same idiom as the test-site validation substep on PP-path Test steps. Always pass `--stageName` from the per-target loop so the helper doesn't have to fall back to URL matching. - -6. **Activate site in {targetLabel}** (optional) — mark the "Activate site in {targetLabel}" task as `in_progress`. Update HTML checklist step to `status-in-progress`. - - PAC CLI is already pointing to the target environment from step 2. Run the activation check: - ```bash - node "${CLAUDE_PLUGIN_ROOT}/scripts/check-activation-status.js" --projectRoot "." - ``` - - - **`activated: true`**: Site is already live. Mark task as `completed`. Update checklist step to `status-completed`. - - **`activated: false`**: Invoke `/power-pages:activate-site`. After completion, mark task as `completed`. Update checklist step to `status-completed`. - - **`error`**: Mark task as `skipped`. Note error in summary. - -7. **Refresh the plan after activation.** Run the helper (activate-site self-refreshes too — this is belt-and-suspenders): - +2. **Commit the plan:** ```bash - node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ - --projectRoot "." \ - --phase activate-site \ - --render + git add docs/alm-plan.html && git commit -m "Add ALM plan for {siteName}" ``` + (Use the `(draft)` suffix for option 2.) `docs/.alm-plan-data.json` stays on disk — it is read by `check-alm-plan.js` for every downstream skill's Phase 0 gate and refreshed by those skills as they run. **Never delete it.** -After all imports: switch PAC CLI back to the dev environment: -```bash -pac env select --environment "{devEnvUrl}" -``` - ---- - -## Phase 8 — Finalize - -Mark the "Finalize" task as `in_progress`. - -### 8.1 Update HTML plan status - -Run the post-run plan refresh in `finalize` mode and re-render. This sets `planData.PLAN_STATUS = "Completed"` and produces the final HTML with all post-run state (latest hostResolution, pipelineMeta + lastDeploy, validationRuns, status footer) consistent across every tab: - -```bash -node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ - --projectRoot "." \ - --phase finalize \ - --render -``` - -If you also need to surface a timestamp in the footer (e.g. plan completion time), apply that via `Edit` tool **after** the re-render — the renderer doesn't currently emit a completion timestamp. - -### 8.2 Run skill tracking - -> Reference: `${CLAUDE_PLUGIN_ROOT}/references/skill-tracking-reference.md` - -```bash -node "${CLAUDE_PLUGIN_ROOT}/scripts/update-skill-tracking.js" \ - --projectRoot "." \ - --skillName "PlanAlm" \ - --authoringTool "ClaudeCode" -``` - -### 8.3 Commit - -```bash -git add docs/alm-plan.html && git commit -m "Add ALM plan for {siteName}" -``` - -### 8.4 Present final summary - -Display a summary: - -``` -## ALM Complete: {siteName} - -**Strategy used:** {PP Pipelines / Manual export/import} -**Skills invoked:** {comma-separated list of skills that ran} +**Next-steps guidance (option 1 — print to the user, do NOT invoke):** -**Artifacts created:** -- docs/alm-plan.html — ALM plan document -- .solution-manifest.json — Solution configuration {(if newly created)} -- docs/alm/last-pipeline.json — Pipeline configuration {(PP path only, if newly created)} -- docs/alm/last-deploy.json — Last deployment record {(PP path only)} -- {solutionName}_{managed|unmanaged}.zip — Solution package {(manual path only)} - -**Site activation:** { - PP path: "Activation status per stage is in docs/alm/last-deploy.json and each deploy history file." - Manual path: list each target env and its activation status (Activated / Pending) -} -``` - -Mark the "Finalize" task as `completed`. +> "Plan approved and saved. **plan-alm doesn't deploy** — run these next, in order. Each detects this plan and proceeds without re-asking, then updates the plan as it completes: +> - **PP Pipelines path:** `/power-pages:setup-solution` → `/power-pages:setup-pipeline` → `/power-pages:deploy-pipeline` (once per target stage; activation + testing happen inside the deploy flow). +> - **Manual path:** `/power-pages:setup-solution` → `/power-pages:export-solution` → review the zip → `/power-pages:import-solution` (once per target). +> Skip any step marked *already set up*. You can re-open the plan any time at `docs/alm-plan.html`." --- @@ -1323,33 +1028,26 @@ Mark the "Finalize" task as `completed`. | Task subject | activeForm | Description | |---|---|---| | Generate ALM plan | Generating ALM plan | Gather strategy inputs, build planData, render docs/alm-plan.html | -| Approve ALM plan | Awaiting plan approval | Present inline summary + HTML plan path, get user confirmation | -| Setup solution | Setting up solution | Invoke setup-solution skill (skip if .solution-manifest.json exists) | -| Setup pipeline | Setting up pipeline | Invoke setup-pipeline skill — PP Pipelines path only (skip if docs/alm/last-pipeline.json exists). May delegate to ensure-pipelines-host internally to resolve or provision the host environment when `hostResolution.willEnsureDuringExecution` is true; that delegation is transparent to plan-alm and is not a separate top-level task. | -| Export solution | Exporting solution | Invoke export-solution skill — Manual path only | -| Deploy to {stageName} | Deploying to {stageName} | Invoke deploy-pipeline skill — PP Pipelines path, one task per target stage | -| Activate site in {stageName} | Activating site in {stageName} | Check activation status + invoke activate-site immediately after each stage deploys — one task per target stage | -| Test site in {stageName} | Testing site in {stageName} | Invoke /power-pages:test-site against the activated URL; capture pass/fail counts; non-blocking | -| Import to {targetEnv} | Importing solution | Switch PAC CLI context, invoke import-solution — Manual path, one task per target | -| Activate site in {targetEnv} | Activating site | Check activation status + invoke activate-site if needed — Manual path, one task per target | -| Finalize | Finalizing | Update HTML plan status, commit, run skill tracking, present summary | +| Approve & save ALM plan | Awaiting plan approval | Present inline summary (incl. defaulted-decisions + plan quality), capture approver, save Approved or Draft, commit, print next-steps | + +> `plan-alm` has exactly these two tasks. Setup / pipeline / deploy / export / import / activate / test are performed by the individual ALM skills the user runs **after** approval — they appear in the plan's `steps[]` (the recommended sequence) but are never `plan-alm` tasks. --- ## Key Decision Points (Wait for User) -1. **Phase 2, Q1**: Solution setup — confirm existing or include `setup-solution` in plan -2. **Phase 2, Q2**: Promotion strategy — PP Pipelines, Manual, or already set up -3. **Phase 2, Q3–Q6** (PP path): Stage count, host env, approval gates (managed auto-set) - **Phase 2, Q3–Q6** (Manual path): Target count, target env URLs, export type, checkpoint pause -4. **Phase 4**: Plan approval — execute, defer, or revise -5. **Phase 6, Manual**: Checkpoint pause after export (if Q6 = Yes) -6. **Phase 7 (delegated)**: Each invoked skill has its own approval gates +1. **Phase 1**: `.alm-deferred` marker handling; pre-plan completeness check (if a solution exists) +2. **Phase 2, Q1**: Solution setup — confirm existing or include `setup-solution` in plan +3. **Phase 2, Q1b**: Split recommendation + override confirmation (if recommended) +4. **Phase 2, Q2**: Promotion strategy — PP Pipelines, Manual, or already set up +5. **Phase 2, Q3–Q5** (PP path): Stage count, host env, approval gates (managed auto-set) + **Phase 2, Q3–Q5** (Manual path): Target count, target env URLs, export type +6. **Phase 4**: Save the plan — Approved, Draft, or revise. **This is the only "approval"; no execution follows.** ## Error Handling - No `powerpages.config.json`: stop, advise `/power-pages:create-site` - `pac env list` fails: skip ENV_LIST pre-filling; ask for environment URLs manually - `render-alm-plan.js` fails (non-zero exit): report error, show planData JSON as fallback, ask user whether to proceed -- Invoked skill fails: report the failure, mark the task as blocked, ask user whether to retry or exit +- Discovery/auth failure: set `PLAN_QUALITY = "degraded"`, surface a prominent risk, still produce the plan (the user fixes auth and re-runs to regenerate) - Plan approval = option 3 (change something): re-run Phase 2 fully, then regenerate plan — do not carry over stale answers diff --git a/plugins/power-pages/skills/plan-alm/scripts/render-alm-plan.js b/plugins/power-pages/skills/plan-alm/scripts/render-alm-plan.js index d6a36d1f2..33453328b 100644 --- a/plugins/power-pages/skills/plan-alm/scripts/render-alm-plan.js +++ b/plugins/power-pages/skills/plan-alm/scripts/render-alm-plan.js @@ -898,7 +898,7 @@ function buildValidationTab(d) { // Renders the full "Site Validation" tab body. One sub-tab per target stage. // Each sub-tab shows a summary grid + per-category test cards. // - // Data shape (from plan-alm Phase 7 Step C, ingesting test-site's docs/alm/last-test-site.json): + // Data shape (populated by test-site's own final-phase refresh, ingesting docs/alm/last-test-site.json): // data.validationRuns = { // "": null | { // url, runAt, durationSec, runOutcome, diff --git a/plugins/power-pages/skills/setup-pipeline/SKILL.md b/plugins/power-pages/skills/setup-pipeline/SKILL.md index 6adb631ca..288bc5d2d 100644 --- a/plugins/power-pages/skills/setup-pipeline/SKILL.md +++ b/plugins/power-pages/skills/setup-pipeline/SKILL.md @@ -84,7 +84,7 @@ The helper returns JSON with `{ exists, stale, staleness: { reason, detail }, ge |---|---|---| | Run `/power-pages:plan-alm` first? | ALM plan gate | Yes — run /power-pages:plan-alm now (Recommended), Continue without a plan (advanced — I know what I'm doing), Cancel | -- **Yes (Recommended)** → invoke `/power-pages:plan-alm`. plan-alm's Phase 7 dispatches back into this skill at the appropriate stage. +- **Yes (Recommended)** → invoke `/power-pages:plan-alm`. It builds the plan and returns — `plan-alm` is a planner and does not deploy. This skill then re-runs the Phase 0 check (now `exists:true`) and proceeds to Phase 1. - **Continue without a plan** → set `BYPASSED_PLAN_GATE = true` and proceed to Phase 1. - **Cancel** → exit cleanly. @@ -529,7 +529,9 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ --render ``` -The helper reads `docs/alm/last-host-check.json` + `docs/alm/last-pipeline.json`, refreshes `planData.hostResolution` and `planData.pipelineMeta`, drops pre-setup "no host detected" risks, and re-renders `docs/alm-plan.html`. When `docs/.alm-plan-data.json` is absent (standalone invocation, not via plan-alm), the helper returns `ok:false` as a soft no-op — safe to run unconditionally. +The helper reads `docs/alm/last-host-check.json` + `docs/alm/last-pipeline.json`, refreshes `planData.hostResolution` and `planData.pipelineMeta`, drops pre-setup "no host detected" risks, and re-renders `docs/alm-plan.html`. When `docs/.alm-plan-data.json` is absent (standalone invocation, not part of an ALM plan), the helper returns `ok:false` as a soft no-op — safe to run unconditionally. + +**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill } | null`. When non-null, tell the user: *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."* When `null` (all steps done) or the helper returned `ok:false` (no plan), say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. **7.6 Present summary:** diff --git a/plugins/power-pages/skills/setup-solution/SKILL.md b/plugins/power-pages/skills/setup-solution/SKILL.md index 0feb106e1..ac437cc7d 100644 --- a/plugins/power-pages/skills/setup-solution/SKILL.md +++ b/plugins/power-pages/skills/setup-solution/SKILL.md @@ -69,7 +69,7 @@ The helper returns JSON with `{ exists, deferred, stale, staleness: { reason, de |---|---|---| | Run `/power-pages:plan-alm` first? | ALM plan gate | Yes — run /power-pages:plan-alm now (Recommended), Continue without a plan (advanced — I know what I'm doing), Cancel | -- **Yes (Recommended)** → invoke `/power-pages:plan-alm`. plan-alm's Phase 7 dispatches back into this skill at the appropriate stage. +- **Yes (Recommended)** → invoke `/power-pages:plan-alm`. It builds the plan and returns — `plan-alm` is a planner and does not deploy. This skill then re-runs the Phase 0 check (now `exists:true`) and proceeds to Phase 1. - **Continue without a plan** → set `BYPASSED_PLAN_GATE = true` and proceed to Phase 1. - **Cancel** → exit cleanly. @@ -884,7 +884,13 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ --render ``` -The helper resets `planData.plannedEnvVarCount` to 0 (the planned env vars have either been created or skipped at the user's request) and re-renders `docs/alm-plan.html` so the Overview stat card and Env Variables tab reflect post-setup state. When `docs/.alm-plan-data.json` is absent (standalone invocation, not via plan-alm), the helper returns `ok:false` as a soft no-op — safe to run unconditionally. +The helper resets `planData.plannedEnvVarCount` to 0 (the planned env vars have either been created or skipped at the user's request) and re-renders `docs/alm-plan.html` so the Overview stat card and Env Variables tab reflect post-setup state. When `docs/.alm-plan-data.json` is absent (standalone invocation, not part of an ALM plan), the helper returns `ok:false` as a soft no-op — safe to run unconditionally. + +**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill } | null`. When it is non-null, tell the user: + +> "Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready." + +When `nextStep` is `null` (every planned step is done) or the helper returned `ok:false` (no plan on disk), say nothing about a next step. **Never auto-invoke the next skill** — `plan-alm` is a planner and the user drives execution one skill at a time. ## Key Decision Points (Wait for User) diff --git a/plugins/power-pages/skills/test-site/SKILL.md b/plugins/power-pages/skills/test-site/SKILL.md index 478f17de5..d48fc3141 100644 --- a/plugins/power-pages/skills/test-site/SKILL.md +++ b/plugins/power-pages/skills/test-site/SKILL.md @@ -730,7 +730,7 @@ EOF ``` or — when invoked from `plan-alm`, the orchestrator may supply the JSON inline. Either way, the marker file location is fixed: `docs/alm/last-test-site.json` (sibling to `docs/alm/last-deploy.json` and `docs/alm/last-pipeline.json`). -**Always include `stageName` in the marker when known.** The agent learns the stage label from the upstream context — plan-alm Phase 7's per-target loop, `docs/alm/last-deploy.json`'s `stageName`, or an explicit user mention. If the stage cannot be inferred (e.g. test-site invoked standalone against an arbitrary URL), set `stageName` to `null`; the refresh helper has fallback resolution paths but the explicit field is the most reliable signal. +**Always include `stageName` in the marker when known.** The agent learns the stage label from the upstream context — `docs/alm/last-deploy.json`'s `stageName`, the plan's `stages[]`, or an explicit user mention. If the stage cannot be inferred (e.g. test-site invoked standalone against an arbitrary URL), set `stageName` to `null`; the refresh helper has fallback resolution paths but the explicit field is the most reliable signal. #### 6.7b Refresh the ALM plan (if one exists) @@ -746,6 +746,8 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ The helper reads `docs/alm/last-test-site.json`, populates `planData.validationRuns[{resolvedStage}]` with the categorized test outcome, and re-renders `docs/alm-plan.html` so the Validation tab updates immediately. When `docs/.alm-plan-data.json` is absent (standalone invocation, no plan in the project), the helper returns `ok:false` as a soft no-op — safe to run unconditionally. +**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill } | null`. When non-null, tell the user: *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."* (Typically: deploy/activate/test the next stage.) When `null` (this was the last step) or the helper returned `ok:false`, say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. + #### 6.8 Suggest Next Steps Based on the test results, suggest relevant skills: From 4d50c9e3749dacbfcdbd0bb67d57347ef2b65fc1 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 14:25:23 +0530 Subject: [PATCH 02/38] Address Copilot review on #191 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - refresh-alm-plan-data.js reconcile(): per-phase refresh failures are no longer swallowed by an empty catch. Each failure is captured into result.failed = [{ phase, error }] and written to stderr, so a marker-schema break that makes a refresh throw is diagnosable instead of silently skipped. `reconciled` now lists only the phases that actually healed; `failed` is present (array) on every return path. Contract-guard test added. - setup-solution SKILL.md: the next-step guidance documented `nextStep: { name, skill }` but `skill` can be null for an internal step (e.g. Finalize) — the prose would tell the user to "run null". Contract corrected to `skill: string | null` and the guidance now branches: print the command only when skill is non-null, else name the step alone. - plan-alm SKILL.md: the finalize commit step always showed the Approved message with a vague "(use the (draft) suffix for option 2)" aside. Now shows two explicit commit commands — Approved vs Draft — so the Draft path can't be followed incorrectly. 1183 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../scripts/lib/refresh-alm-plan-data.js | 24 ++++++++++++----- .../tests/refresh-alm-plan-data.test.js | 26 +++++++++++++++++++ plugins/power-pages/skills/plan-alm/SKILL.md | 15 +++++++---- .../skills/setup-solution/SKILL.md | 7 ++--- 4 files changed, 57 insertions(+), 15 deletions(-) diff --git a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js index f58fc44c1..5887108ac 100644 --- a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js +++ b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js @@ -976,7 +976,9 @@ function mtimeMs(filePath) { // single loaded planData, writes once, and renders once. Idempotent — a marker // the plan already reflects (plan newer than marker) is skipped, so the steady // state is a cheap no-op. Honors the .alm-deferred opt-out and soft-no-ops when -// there is no plan. +// there is no plan. Returns { ok, reconciled:[phases healed], failed:[{phase,error}], +// rendered, nextStep } — `failed` is non-empty when a phase's refresh threw (e.g. a +// marker schema the refresh can't parse); the reconcile still heals the other phases. function reconcile({ projectRoot, render, rendererPath }) { if (!projectRoot) throw new Error('--projectRoot is required'); const dataPath = path.join(projectRoot, 'docs', '.alm-plan-data.json'); @@ -984,10 +986,10 @@ function reconcile({ projectRoot, render, rendererPath }) { // Respect the project-level ALM opt-out. if (fs.existsSync(path.join(projectRoot, '.alm-deferred'))) { - return { ok: true, reconciled: [], rendered: false, reason: 'deferred' }; + return { ok: true, reconciled: [], failed: [], rendered: false, reason: 'deferred' }; } if (!fs.existsSync(dataPath)) { - return { ok: false, reconciled: [], rendered: false, reason: 'no-plan' }; + return { ok: false, reconciled: [], failed: [], rendered: false, reason: 'no-plan' }; } const planMtime = mtimeMs(dataPath); @@ -1009,7 +1011,7 @@ function reconcile({ projectRoot, render, rendererPath }) { if (mtimeMs(almPath(projectRoot, 'lastEnvVars')) > planMtime) pending.add(envVarPhase); if (pending.size === 0) { - return { ok: true, reconciled: [], rendered: false }; + return { ok: true, reconciled: [], failed: [], rendered: false }; } // Deterministic application order (source schema first, host/pipeline, then @@ -1028,11 +1030,19 @@ function reconcile({ projectRoot, render, rendererPath }) { throw new Error('Could not parse docs/.alm-plan-data.json: ' + e.message); } + // A single phase failing must not abort the reconcile — keep healing the rest — + // but the failure must be visible (an empty catch makes a broken marker schema + // impossible to diagnose: reconcile would report success while silently skipping + // the phase). Collect failures and surface them in the result + on stderr. + const reconciled = []; + const failed = []; for (const phase of phases) { try { applyRefresh(planData, phase, projectRoot, null); - } catch { - // A single phase failing must not abort the reconcile — keep healing the rest. + reconciled.push(phase); + } catch (e) { + failed.push({ phase, error: e.message }); + process.stderr.write(`[refresh-alm-plan-data] reconcile phase "${phase}" failed: ${e.message}\n`); } } fs.writeFileSync(dataPath, JSON.stringify(planData, null, 2), 'utf8'); @@ -1043,7 +1053,7 @@ function reconcile({ projectRoot, render, rendererPath }) { rendered = true; } - return { ok: true, reconciled: phases, rendered, nextStep: computeNextStep(planData) }; + return { ok: true, reconciled, failed, rendered, nextStep: computeNextStep(planData) }; } function findRendererPath(rendererPath) { diff --git a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js index 65428d570..46261599e 100644 --- a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js +++ b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js @@ -1715,11 +1715,37 @@ test('reconcile: heals a skipped refresh — a newer last-deploy.json is ingeste const result = reconcile({ projectRoot: root, render: false }); assert.equal(result.ok, true); assert.deepEqual(result.reconciled, ['deploy-pipeline']); + // failed[] is part of the contract — empty when every phase heals cleanly. + assert.deepEqual(result.failed, []); const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); assert.equal(planData.pipelineMeta.lastDeploy.status, 'Succeeded'); assert.equal(planData.pipelineMeta.lastDeploy.componentCount, 118); }); +test('reconcile: failed[] is always present and is an array on every return path', (t) => { + // Contract guard for the per-phase error capture (Copilot review on #191): a + // phase whose refresh throws must NOT be swallowed silently — it lands in + // result.failed = [{ phase, error }] while the other phases still heal, and + // failed[] is present (empty) on the no-op / deferred / no-plan paths too. + // (A phase only throws on a genuine marker-schema break — readJson is + // defensive — so we assert the contract shape rather than fabricate a throw.) + const noPlan = reconcile({ projectRoot: makeProject(t), render: false }); + assert.ok(Array.isArray(noPlan.failed), 'no-plan path carries failed:[]'); + + const deferredRoot = makeProject(t); + writeJson(path.join(deferredRoot, 'docs', '.alm-plan-data.json'), { SITE_NAME: 'T' }); + fs.writeFileSync(path.join(deferredRoot, '.alm-deferred'), 'deferred'); + const deferred = reconcile({ projectRoot: deferredRoot, render: false }); + assert.deepEqual(deferred.failed, [], 'deferred path carries failed:[]'); + + const idleRoot = makeProject(t); + writeJson(path.join(idleRoot, 'docs', '.alm-plan-data.json'), { SITE_NAME: 'T' }); + const future = (Date.now() + 60 * 1000) / 1000; + fs.utimesSync(path.join(idleRoot, 'docs', '.alm-plan-data.json'), future, future); + const idle = reconcile({ projectRoot: idleRoot, render: false }); + assert.deepEqual(idle.failed, [], 'nothing-pending path carries failed:[]'); +}); + test('reconcile: idempotent no-op when the plan already reflects the markers', (t) => { const root = makeProject(t); writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { stageName: 'Staging', status: 'Succeeded' }); diff --git a/plugins/power-pages/skills/plan-alm/SKILL.md b/plugins/power-pages/skills/plan-alm/SKILL.md index 8f4edeb53..8e6fe301f 100644 --- a/plugins/power-pages/skills/plan-alm/SKILL.md +++ b/plugins/power-pages/skills/plan-alm/SKILL.md @@ -1008,11 +1008,16 @@ Both spans are guaranteed to exist in the template — there is exactly one of e node "${CLAUDE_PLUGIN_ROOT}/scripts/update-skill-tracking.js" \ --projectRoot "." --skillName "PlanAlm" --authoringTool "ClaudeCode" ``` -2. **Commit the plan:** - ```bash - git add docs/alm-plan.html && git commit -m "Add ALM plan for {siteName}" - ``` - (Use the `(draft)` suffix for option 2.) `docs/.alm-plan-data.json` stays on disk — it is read by `check-alm-plan.js` for every downstream skill's Phase 0 gate and refreshed by those skills as they run. **Never delete it.** +2. **Commit the plan** — pick the commit message for the save option the user chose: + - **Option 1 (Approved):** + ```bash + git add docs/alm-plan.html && git commit -m "Add ALM plan for {siteName}" + ``` + - **Option 2 (Draft):** + ```bash + git add docs/alm-plan.html && git commit -m "Add ALM plan for {siteName} (draft)" + ``` + `docs/.alm-plan-data.json` stays on disk — it is read by `check-alm-plan.js` for every downstream skill's Phase 0 gate and refreshed by those skills as they run. **Never delete it.** **Next-steps guidance (option 1 — print to the user, do NOT invoke):** diff --git a/plugins/power-pages/skills/setup-solution/SKILL.md b/plugins/power-pages/skills/setup-solution/SKILL.md index ac437cc7d..2c44aeb39 100644 --- a/plugins/power-pages/skills/setup-solution/SKILL.md +++ b/plugins/power-pages/skills/setup-solution/SKILL.md @@ -886,11 +886,12 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ The helper resets `planData.plannedEnvVarCount` to 0 (the planned env vars have either been created or skipped at the user's request) and re-renders `docs/alm-plan.html` so the Overview stat card and Env Variables tab reflect post-setup state. When `docs/.alm-plan-data.json` is absent (standalone invocation, not part of an ALM plan), the helper returns `ok:false` as a soft no-op — safe to run unconditionally. -**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill } | null`. When it is non-null, tell the user: +**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill: string | null } | null`. `skill` is `null` when the next pending step has no user-invocable command (e.g. an internal "Finalize" step) — so branch on it: -> "Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready." +- **`nextStep.skill` is non-null** → "Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready." +- **`nextStep.skill` is `null`** → name the step only, with no command: "Plan updated. Next in your plan: **{nextStep.name}**." Never print `run null`. -When `nextStep` is `null` (every planned step is done) or the helper returned `ok:false` (no plan on disk), say nothing about a next step. **Never auto-invoke the next skill** — `plan-alm` is a planner and the user drives execution one skill at a time. +When `nextStep` itself is `null` (every planned step is done) or the helper returned `ok:false` (no plan on disk), say nothing about a next step. **Never auto-invoke the next skill** — `plan-alm` is a planner and the user drives execution one skill at a time. ## Key Decision Points (Wait for User) From 2fd9aa6a21e919001ba37bdc134c46d1c57b234f Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 14:41:28 +0530 Subject: [PATCH 03/38] Self-review fixes: PP-path activate step, manual steps[], EDM error rule MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From a full review pass of the plan-only conversion: - refreshDeployPipeline now completes the "Activate site in {stage}" step when the deploy marker evidences activation (activationStatus set, deploy not failed). The PP deploy flow activates the site internally, so leaving the step pending made computeNextStep redundantly nudge the user to run /activate-site for work the deploy already did. Testing stays a separate step (test-site). When the marker has no activationStatus, the Activate step is left pending. - Next-steps guidance corrected: the PP path does activation inside the deploy flow but NOT testing — testing is the separate /power-pages:test-site step the plan already lists. Guidance now ends "... → /power-pages:test-site". - steps[] template: documented the MANUAL-path shape (Setup solution → Export solution → per-target Import/Activate/Test) alongside the PP-path example, so a manual-strategy plan emits step names that step-sync + computeNextStep match — previously only the PP shape was shown. - Error Handling: the "No powerpages.config.json: stop" rule contradicted Phase 1, which resolves data-model/EDM sites from .powerpages-site/website.yml (no config.json). Now stops only when BOTH markers are absent. 1185 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../scripts/lib/refresh-alm-plan-data.js | 15 ++++++ .../tests/refresh-alm-plan-data.test.js | 52 +++++++++++++++++++ plugins/power-pages/skills/plan-alm/SKILL.md | 15 +++++- 3 files changed, 80 insertions(+), 2 deletions(-) diff --git a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js index 5887108ac..1654a2deb 100644 --- a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js +++ b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js @@ -445,6 +445,21 @@ function refreshDeployPipeline(planData, projectRoot) { stage: deployMarker.stageName, status: failed ? 'failed' : 'completed', }); + // The PP deploy flow activates the site as part of deployment (deploy-pipeline + // Phase 7.7), recording it in the marker's `activationStatus`. When the deploy + // succeeded AND the marker evidences activation, complete the matching + // "Activate site in {stage}" step too — otherwise computeNextStep would point + // the user at /activate-site for work the deploy already did. Testing stays a + // separate step (the test-site skill flips it via refreshTestSite). When the + // marker carries no activationStatus, leave the Activate step pending — the + // user may still need to run /activate-site explicitly. + if (!failed && deployMarker.activationStatus) { + setStepStatus(planData, { + keyword: /\bactivate\b/i, + stage: deployMarker.stageName, + status: 'completed', + }); + } } return planData; } diff --git a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js index 46261599e..2be786a41 100644 --- a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js +++ b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js @@ -497,6 +497,58 @@ test('refresh deploy-pipeline ingests batchValidation block from last-deploy.jso }); }); +test('refresh deploy-pipeline completes the "Activate site" step when the marker evidences activation, but not Test', (t) => { + // The PP deploy flow activates the site internally (marker.activationStatus), + // so a successful deploy must also complete the matching "Activate site in {stage}" + // step — otherwise computeNextStep would redundantly point the user at + // /activate-site. Testing remains a separate step (test-site flips it). + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'TestSite', + STRATEGY: 'pipeline', + steps: [ + { name: 'Deploy via pipeline to Staging', status: 'pending' }, + { name: 'Activate site in Staging', status: 'pending' }, + { name: 'Test site in Staging', status: 'pending' }, + ], + stages: [{ label: 'Staging', envUrl: 'https://staging.crm.dynamics.com' }], + }); + writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { + stageRunId: 'srun-1', stageName: 'Staging', status: 'Succeeded', + deployedAt: '2026-06-16T00:00:00.000Z', activationStatus: 'Activated', + siteUrl: 'https://contoso.powerappsportals.com', + }); + + refresh({ projectRoot: root, phase: 'deploy-pipeline', render: false }); + const steps = readJson(path.join(root, 'docs', '.alm-plan-data.json')).steps; + const byName = (n) => steps.find((s) => s.name === n).status; + assert.equal(byName('Deploy via pipeline to Staging'), 'completed'); + assert.equal(byName('Activate site in Staging'), 'completed', 'deploy activates the site -> Activate step completes too'); + assert.equal(byName('Test site in Staging'), 'pending', 'Test stays pending — it is the separate test-site step'); +}); + +test('refresh deploy-pipeline leaves the "Activate site" step pending when the marker has no activationStatus', (t) => { + // No activation evidence in the marker -> don't fabricate completion; the user + // may still need to run /activate-site explicitly. + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'TestSite', + STRATEGY: 'pipeline', + steps: [ + { name: 'Deploy via pipeline to Staging', status: 'pending' }, + { name: 'Activate site in Staging', status: 'pending' }, + ], + stages: [{ label: 'Staging', envUrl: 'https://staging.crm.dynamics.com' }], + }); + writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { + stageRunId: 'srun-1', stageName: 'Staging', status: 'Succeeded', deployedAt: '2026-06-16T00:00:00.000Z', + }); + + refresh({ projectRoot: root, phase: 'deploy-pipeline', render: false }); + const steps = readJson(path.join(root, 'docs', '.alm-plan-data.json')).steps; + assert.equal(steps.find((s) => s.name === 'Activate site in Staging').status, 'pending'); +}); + test('refresh deploy-pipeline accepts legacy elapsedSecondsApprox name in batchValidation', (t) => { // Backward-compatibility: legacy SKILL.md prose (pre-v1.x) used // `elapsedSecondsApprox`. Markers written under that schema still in the diff --git a/plugins/power-pages/skills/plan-alm/SKILL.md b/plugins/power-pages/skills/plan-alm/SKILL.md index 8e6fe301f..3b1d8a037 100644 --- a/plugins/power-pages/skills/plan-alm/SKILL.md +++ b/plugins/power-pages/skills/plan-alm/SKILL.md @@ -630,6 +630,17 @@ Build a `planData` object with all gathered strategy inputs: { "name": "Activate site in Production", "status": "pending", "skip": false }, { "name": "Test site in Production", "status": "pending", "skip": false } ], + // The steps[] above is the PP Pipelines path. For the MANUAL path (STRATEGY = "manual") + // emit this shape instead — do NOT include "Setup pipeline"/"Deploy via pipeline": + // { "name": "Setup solution", ... }, + // { "name": "Export solution", ... }, + // then PER Manual target {targetLabel}: + // { "name": "Import to {targetLabel}", ... }, + // { "name": "Activate site in {targetLabel}", ... }, // import-solution does NOT activate + // { "name": "Test site in {targetLabel}", ... } + // These names are what refresh-alm-plan-data.js step-sync (export/import/activate/test) + // and computeNextStep match on, so the manual checklist and next-step nudges resolve + // to /power-pages:export-solution → /power-pages:import-solution → /power-pages:activate-site → /power-pages:test-site. "validationRuns": { "Staging": null, "Production": null @@ -1022,7 +1033,7 @@ Both spans are guaranteed to exist in the template — there is exactly one of e **Next-steps guidance (option 1 — print to the user, do NOT invoke):** > "Plan approved and saved. **plan-alm doesn't deploy** — run these next, in order. Each detects this plan and proceeds without re-asking, then updates the plan as it completes: -> - **PP Pipelines path:** `/power-pages:setup-solution` → `/power-pages:setup-pipeline` → `/power-pages:deploy-pipeline` (once per target stage; activation + testing happen inside the deploy flow). +> - **PP Pipelines path:** `/power-pages:setup-solution` → `/power-pages:setup-pipeline` → `/power-pages:deploy-pipeline` (once per target stage; the deploy flow activates the site) → `/power-pages:test-site` to validate each stage. > - **Manual path:** `/power-pages:setup-solution` → `/power-pages:export-solution` → review the zip → `/power-pages:import-solution` (once per target). > Skip any step marked *already set up*. You can re-open the plan any time at `docs/alm-plan.html`." @@ -1051,7 +1062,7 @@ Both spans are guaranteed to exist in the template — there is exactly one of e ## Error Handling -- No `powerpages.config.json`: stop, advise `/power-pages:create-site` +- No `.powerpages-site/website.yml` **and** no `powerpages.config.json`: stop, advise `/power-pages:create-site` (Phase 1 resolves site identity from either marker — a data-model/EDM site has only `website.yml`, so don't hard-stop on the missing config alone) - `pac env list` fails: skip ENV_LIST pre-filling; ask for environment URLs manually - `render-alm-plan.js` fails (non-zero exit): report error, show planData JSON as fallback, ask user whether to proceed - Discovery/auth failure: set `PLAN_QUALITY = "degraded"`, surface a prominent risk, still produce the plan (the user fixes auth and re-runs to regenerate) From 298fcf0ff3def4feaa437ec31a7cf0e07b539529 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 14:56:10 +0530 Subject: [PATCH 04/38] Address Copilot second-pass review on #191 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - refreshDeployPipeline: gate the Activate-step auto-completion on the explicit "Activated" outcome, not mere truthiness. deploy-pipeline writes activationStatus: "Pending" when the user DEFERS activation — that's truthy but means /power-pages:activate-site is still required, so the previous check would have wrongly completed the Activate step and dropped it from nextStep. Now any non-"Activated" value (Pending, null, a failure note) leaves the step pending. Regression test extended to cover undefined/null/"Pending"/"Failed". - nextStep skill:null guidance: the same "run {nextStep.skill}" guidance lived in 7 other execution skills (activate-site, configure-env-variables, deploy-pipeline, export-solution, import-solution, setup-pipeline, test-site) — the prior fix only updated setup-solution. All 7 now document `skill: string | null` and branch so an internal step (e.g. Finalize) names the step without ever printing `run null`. 1185 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../scripts/lib/refresh-alm-plan-data.js | 20 ++++---- .../tests/refresh-alm-plan-data.test.js | 48 +++++++++++-------- .../power-pages/skills/activate-site/SKILL.md | 2 +- .../skills/configure-env-variables/SKILL.md | 2 +- .../skills/deploy-pipeline/SKILL.md | 2 +- .../skills/export-solution/SKILL.md | 2 +- .../skills/import-solution/SKILL.md | 2 +- .../skills/setup-pipeline/SKILL.md | 2 +- plugins/power-pages/skills/test-site/SKILL.md | 2 +- 9 files changed, 46 insertions(+), 36 deletions(-) diff --git a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js index 1654a2deb..ba5788f83 100644 --- a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js +++ b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js @@ -445,15 +445,17 @@ function refreshDeployPipeline(planData, projectRoot) { stage: deployMarker.stageName, status: failed ? 'failed' : 'completed', }); - // The PP deploy flow activates the site as part of deployment (deploy-pipeline - // Phase 7.7), recording it in the marker's `activationStatus`. When the deploy - // succeeded AND the marker evidences activation, complete the matching - // "Activate site in {stage}" step too — otherwise computeNextStep would point - // the user at /activate-site for work the deploy already did. Testing stays a - // separate step (the test-site skill flips it via refreshTestSite). When the - // marker carries no activationStatus, leave the Activate step pending — the - // user may still need to run /activate-site explicitly. - if (!failed && deployMarker.activationStatus) { + // The PP deploy flow can activate the site as part of deployment (deploy-pipeline + // Phase 7.7), recording the outcome in the marker's `activationStatus`. When the + // deploy succeeded AND the marker shows the site was actually activated, complete + // the matching "Activate site in {stage}" step too — otherwise computeNextStep + // would point the user at /activate-site for work the deploy already did. Gate on + // the explicit "Activated" outcome, NOT mere truthiness: deploy-pipeline writes + // activationStatus: "Pending" when the user DEFERS activation, which is truthy but + // means the Activate step is still REQUIRED. Any non-"Activated" value (Pending, + // null, a failure note) leaves the Activate step pending. Testing stays a separate + // step (the test-site skill flips it via refreshTestSite). + if (!failed && /^activated$/i.test(String(deployMarker.activationStatus || ''))) { setStepStatus(planData, { keyword: /\bactivate\b/i, stage: deployMarker.stageName, diff --git a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js index 2be786a41..c6cfdb919 100644 --- a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js +++ b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js @@ -527,26 +527,34 @@ test('refresh deploy-pipeline completes the "Activate site" step when the marker assert.equal(byName('Test site in Staging'), 'pending', 'Test stays pending — it is the separate test-site step'); }); -test('refresh deploy-pipeline leaves the "Activate site" step pending when the marker has no activationStatus', (t) => { - // No activation evidence in the marker -> don't fabricate completion; the user - // may still need to run /activate-site explicitly. - const root = makeProject(t); - writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { - SITE_NAME: 'TestSite', - STRATEGY: 'pipeline', - steps: [ - { name: 'Deploy via pipeline to Staging', status: 'pending' }, - { name: 'Activate site in Staging', status: 'pending' }, - ], - stages: [{ label: 'Staging', envUrl: 'https://staging.crm.dynamics.com' }], - }); - writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { - stageRunId: 'srun-1', stageName: 'Staging', status: 'Succeeded', deployedAt: '2026-06-16T00:00:00.000Z', - }); - - refresh({ projectRoot: root, phase: 'deploy-pipeline', render: false }); - const steps = readJson(path.join(root, 'docs', '.alm-plan-data.json')).steps; - assert.equal(steps.find((s) => s.name === 'Activate site in Staging').status, 'pending'); +test('refresh deploy-pipeline leaves the "Activate site" step pending when activation was not done (null or deferred "Pending")', (t) => { + // Only an explicit "Activated" outcome completes the Activate step. A missing + // activationStatus (null) OR a deferred activation (deploy-pipeline writes + // "Pending" when the user defers) must leave the step pending — the user still + // needs to run /activate-site, and nextStep must keep surfacing it. + for (const activationStatus of [undefined, null, 'Pending', 'Failed']) { + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'TestSite', + STRATEGY: 'pipeline', + steps: [ + { name: 'Deploy via pipeline to Staging', status: 'pending' }, + { name: 'Activate site in Staging', status: 'pending' }, + ], + stages: [{ label: 'Staging', envUrl: 'https://staging.crm.dynamics.com' }], + }); + const marker = { stageRunId: 'srun-1', stageName: 'Staging', status: 'Succeeded', deployedAt: '2026-06-16T00:00:00.000Z' }; + if (activationStatus !== undefined) marker.activationStatus = activationStatus; + writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), marker); + + refresh({ projectRoot: root, phase: 'deploy-pipeline', render: false }); + const steps = readJson(path.join(root, 'docs', '.alm-plan-data.json')).steps; + assert.equal( + steps.find((s) => s.name === 'Activate site in Staging').status, + 'pending', + `activationStatus=${JSON.stringify(activationStatus)} must NOT complete the Activate step`, + ); + } }); test('refresh deploy-pipeline accepts legacy elapsedSecondsApprox name in batchValidation', (t) => { diff --git a/plugins/power-pages/skills/activate-site/SKILL.md b/plugins/power-pages/skills/activate-site/SKILL.md index 7476cf187..6c388ceee 100644 --- a/plugins/power-pages/skills/activate-site/SKILL.md +++ b/plugins/power-pages/skills/activate-site/SKILL.md @@ -308,7 +308,7 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ The helper reads `docs/alm/last-activate.json`, writes a per-target entry into `planData.activations[stageName]` (siteUrl, status, activatedAt), and re-renders `docs/alm-plan.html` so the matching `Activate site in {stageName}` checklist step shows an `ACTIVATED` badge with the live site URL inline. When `docs/.alm-plan-data.json` is absent (standalone, not part of an ALM plan), the helper returns `ok:false` as a soft no-op. -**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill } | null`. When non-null, tell the user: *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."* When `null` or the helper returned `ok:false`, say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. +**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill: string | null } | null`. When non-null, branch on `skill`: when `skill` is non-null, tell the user *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."*; when `skill` is `null` (an internal step such as Finalize, no user command), name the step only — *"Plan updated. Next in your plan: **{nextStep.name}**."* — and never print `run null`. When `null` or the helper returned `ok:false`, say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. #### 5.3 Suggest Next Steps diff --git a/plugins/power-pages/skills/configure-env-variables/SKILL.md b/plugins/power-pages/skills/configure-env-variables/SKILL.md index 683a3d62d..81a577967 100644 --- a/plugins/power-pages/skills/configure-env-variables/SKILL.md +++ b/plugins/power-pages/skills/configure-env-variables/SKILL.md @@ -493,7 +493,7 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ The helper re-reads `docs/alm/last-env-vars.json` so newly-created definitions appear in `planData.envVars[]`, backfills per-stage values from `deployment-settings.json` into the "Values by Environment" matrix, zeroes `plannedEnvVarCount`, stamps `LAST_SYNC_AT`, and re-renders `docs/alm-plan.html`. When `docs/.alm-plan-data.json` is absent (standalone invocation, not part of an ALM plan), the helper returns `ok:false` as a soft no-op — safe to run unconditionally. -**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill } | null`. When non-null, tell the user: *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."* When `null` or the helper returned `ok:false`, say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. +**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill: string | null } | null`. When non-null, branch on `skill`: when `skill` is non-null, tell the user *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."*; when `skill` is `null` (an internal step such as Finalize, no user command), name the step only — *"Plan updated. Next in your plan: **{nextStep.name}**."* — and never print `run null`. When `null` or the helper returned `ok:false`, say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. ## Key Decision Points (Wait for User) diff --git a/plugins/power-pages/skills/deploy-pipeline/SKILL.md b/plugins/power-pages/skills/deploy-pipeline/SKILL.md index a686c1d1a..7465cc0c9 100644 --- a/plugins/power-pages/skills/deploy-pipeline/SKILL.md +++ b/plugins/power-pages/skills/deploy-pipeline/SKILL.md @@ -946,7 +946,7 @@ The helper reads the `docs/alm/last-deploy.json` you just wrote, ingests it into This step is what keeps the rendered plan current — `plan-alm` is a planner and does not refresh the plan itself, so each execution skill owns its own post-run refresh. Running it more than once is idempotent (same input → same output). -**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill } | null`. When non-null, tell the user: *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."* (For a multi-stage pipeline this is typically the next stage's deploy, or the next stage's activate/test if those are separate steps.) When `null` (all steps done) or the helper returned `ok:false` (no plan), say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. +**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill: string | null } | null`. When non-null, branch on `skill`: when `skill` is non-null, tell the user *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."*; when `skill` is `null` (an internal step such as Finalize, no user command), name the step only — *"Plan updated. Next in your plan: **{nextStep.name}**."* — and never print `run null`. (For a multi-stage pipeline this is typically the next stage's deploy, or the next stage's activate/test if those are separate steps.) When `null` (all steps done) or the helper returned `ok:false` (no plan), say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. **7.6 Present summary:** diff --git a/plugins/power-pages/skills/export-solution/SKILL.md b/plugins/power-pages/skills/export-solution/SKILL.md index 5b34bcfb9..489796e7d 100644 --- a/plugins/power-pages/skills/export-solution/SKILL.md +++ b/plugins/power-pages/skills/export-solution/SKILL.md @@ -379,7 +379,7 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ Re-renders `docs/alm-plan.html` so any step-status updates the agent made during this skill (`Export solution` → `status-completed`) flow through. When `docs/.alm-plan-data.json` is absent (standalone invocation, not part of an ALM plan), the helper returns `ok:false` as a soft no-op — safe to run unconditionally. -**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill } | null`. When non-null, tell the user: *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."* (Typically: review the exported zip, then run `/power-pages:import-solution` for the first target.) When `null` or the helper returned `ok:false`, say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. +**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill: string | null } | null`. When non-null, branch on `skill`: when `skill` is non-null, tell the user *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."*; when `skill` is `null` (an internal step such as Finalize, no user command), name the step only — *"Plan updated. Next in your plan: **{nextStep.name}**."* — and never print `run null`. (Typically: review the exported zip, then run `/power-pages:import-solution` for the first target.) When `null` or the helper returned `ok:false`, say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. ## Key Decision Points (Wait for User) diff --git a/plugins/power-pages/skills/import-solution/SKILL.md b/plugins/power-pages/skills/import-solution/SKILL.md index f3d538b05..35234ed34 100644 --- a/plugins/power-pages/skills/import-solution/SKILL.md +++ b/plugins/power-pages/skills/import-solution/SKILL.md @@ -548,7 +548,7 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ If `--stageName` is omitted the helper falls back to matching `docs/alm/last-import.json`'s `targetEnvironment` URL against `planData.stages[].envUrl`. When the match fails (rare — usually a stage-label/env-URL mismatch in planData), the import is captured under a synthetic key so it isn't silently lost; pass `--stageName` explicitly to keep the rendered plan clean. When `docs/.alm-plan-data.json` is absent, the helper returns `ok:false` as a soft no-op. -**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill } | null`. When non-null, tell the user: *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."* (Typically: import to the next target, or activate the site in this target.) When `null` or the helper returned `ok:false`, say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. +**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill: string | null } | null`. When non-null, branch on `skill`: when `skill` is non-null, tell the user *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."*; when `skill` is `null` (an internal step such as Finalize, no user command), name the step only — *"Plan updated. Next in your plan: **{nextStep.name}**."* — and never print `run null`. (Typically: import to the next target, or activate the site in this target.) When `null` or the helper returned `ok:false`, say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. ## Key Decision Points (Wait for User) diff --git a/plugins/power-pages/skills/setup-pipeline/SKILL.md b/plugins/power-pages/skills/setup-pipeline/SKILL.md index 288bc5d2d..710a8810e 100644 --- a/plugins/power-pages/skills/setup-pipeline/SKILL.md +++ b/plugins/power-pages/skills/setup-pipeline/SKILL.md @@ -531,7 +531,7 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ The helper reads `docs/alm/last-host-check.json` + `docs/alm/last-pipeline.json`, refreshes `planData.hostResolution` and `planData.pipelineMeta`, drops pre-setup "no host detected" risks, and re-renders `docs/alm-plan.html`. When `docs/.alm-plan-data.json` is absent (standalone invocation, not part of an ALM plan), the helper returns `ok:false` as a soft no-op — safe to run unconditionally. -**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill } | null`. When non-null, tell the user: *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."* When `null` (all steps done) or the helper returned `ok:false` (no plan), say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. +**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill: string | null } | null`. When non-null, branch on `skill`: when `skill` is non-null, tell the user *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."*; when `skill` is `null` (an internal step such as Finalize, no user command), name the step only — *"Plan updated. Next in your plan: **{nextStep.name}**."* — and never print `run null`. When `null` (all steps done) or the helper returned `ok:false` (no plan), say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. **7.6 Present summary:** diff --git a/plugins/power-pages/skills/test-site/SKILL.md b/plugins/power-pages/skills/test-site/SKILL.md index d48fc3141..c1b3df680 100644 --- a/plugins/power-pages/skills/test-site/SKILL.md +++ b/plugins/power-pages/skills/test-site/SKILL.md @@ -746,7 +746,7 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ The helper reads `docs/alm/last-test-site.json`, populates `planData.validationRuns[{resolvedStage}]` with the categorized test outcome, and re-renders `docs/alm-plan.html` so the Validation tab updates immediately. When `docs/.alm-plan-data.json` is absent (standalone invocation, no plan in the project), the helper returns `ok:false` as a soft no-op — safe to run unconditionally. -**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill } | null`. When non-null, tell the user: *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."* (Typically: deploy/activate/test the next stage.) When `null` (this was the last step) or the helper returned `ok:false`, say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. +**Point the user at the next step (user-driven sequencing).** The helper's stdout JSON includes `nextStep: { name, skill: string | null } | null`. When non-null, branch on `skill`: when `skill` is non-null, tell the user *"Plan updated. Next in your plan: **{nextStep.name}** → run `{nextStep.skill}` when you're ready."*; when `skill` is `null` (an internal step such as Finalize, no user command), name the step only — *"Plan updated. Next in your plan: **{nextStep.name}**."* — and never print `run null`. (Typically: deploy/activate/test the next stage.) When `null` (this was the last step) or the helper returned `ok:false`, say nothing about a next step. **Never auto-invoke the next skill** — the user drives execution. #### 6.8 Suggest Next Steps From 9f66e35cb3e607e85e76adb8465c2e7a9520c6d7 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 17:51:11 +0530 Subject: [PATCH 05/38] Fix step-sync stage matching: marker "Deploy to {label}" vs plan step label MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit priyanshu92 review on #191: a finished PP deploy never advanced its plan step. setup-pipeline names pipeline stages "Deploy to {targetLabel}", and deploy-pipeline writes that verbatim as last-deploy.json's stageName ("Deploy to Staging"). The plan step is "Deploy via pipeline to Staging", so setStepStatus's substring match of "deploy to staging" against the step name failed and the step (and the Activate step my earlier change keys off the same stage) stayed pending forever — defeating the self-maintaining-plan goal. Every existing fixture used the bare "Staging", so the suite gave false confidence. setStepStatus now strips a leading "Deploy to " from the stage filter to recover the bare label before matching; callers that already pass the label (e.g. test-site --stageName "Staging") are unaffected (no-op strip). Added regression tests using the REAL "Deploy to Staging" marker shape (step + activate flip; Production untouched) plus a direct setStepStatus normalization test. Also removed a duplicate inline comment on the 'ensure-pipelines-host' PHASES entry (per review — the explanation already lives at the refreshEnsurePipelinesHost def). 1187 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../scripts/lib/refresh-alm-plan-data.js | 18 ++++--- .../tests/refresh-alm-plan-data.test.js | 47 +++++++++++++++++++ 2 files changed, 59 insertions(+), 6 deletions(-) diff --git a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js index ba5788f83..3fde3f0ba 100644 --- a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js +++ b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js @@ -75,11 +75,6 @@ const PHASES = new Set([ 'import-solution', 'activate-site', 'test-site', - // ensure-pipelines-host: host-only update from docs/alm/last-host-check.json. - // Distinct from 'setup-pipeline' (which also ingests last-pipeline.json + flips - // the "Setup pipeline" step) — this runs when the host was resolved/provisioned - // but the pipeline does not exist yet (e.g. the host install crossed a session - // boundary before setup-pipeline ran). 'ensure-pipelines-host', 'finalize', ]); @@ -247,7 +242,18 @@ function backfillEnvVarValuesFromSettings(planData, projectRoot) { function setStepStatus(planData, { keyword, stage, status }) { if (!Array.isArray(planData.steps)) return 0; if (!keyword) return 0; - const targetStage = (typeof stage === 'string' && stage.length > 0) ? stage.toLowerCase() : null; + // The stage filter must match the bare target LABEL embedded in the plan step + // names ("Deploy via pipeline to Staging", "Activate site in Staging", "Test + // site in Staging" — all carry "Staging"). But the run markers carry the + // pipeline STAGE name, which setup-pipeline creates as "Deploy to {label}" + // (e.g. last-deploy.json's stageName = SELECTED_STAGE.name = "Deploy to + // Staging"). A raw substring match of "deploy to staging" against "deploy via + // pipeline to staging" FAILS, so a finished deploy would never flip its step. + // Strip a leading "Deploy to " to recover the label before matching. Callers + // that already pass the bare label (e.g. test-site --stageName "Staging") are + // unaffected — the strip is a no-op when the prefix isn't present. + const rawStage = (typeof stage === 'string' && stage.length > 0) ? stage.toLowerCase() : null; + const targetStage = rawStage ? rawStage.replace(/^deploy\s+to\s+/, '').trim() : null; let flipped = 0; for (const step of planData.steps) { if (!step || typeof step.name !== 'string') continue; diff --git a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js index c6cfdb919..5d6a2a7ca 100644 --- a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js +++ b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js @@ -1211,6 +1211,53 @@ test('step-sync: deploy-pipeline flips ONLY the matching stage step to completed 'Production deploy step must stay pending — stage filter disambiguates "Deploy" steps'); }); +test('step-sync: deploy-pipeline matches the REAL marker stageName "Deploy to {label}" (not just bare "Staging")', (t) => { + // Regression for the stage-name mismatch: setup-pipeline names pipeline stages + // "Deploy to {targetLabel}", and deploy-pipeline writes that verbatim as + // last-deploy.json's stageName ("Deploy to Staging"). The plan step is + // "Deploy via pipeline to Staging", so a raw substring match of the marker's + // "deploy to staging" against the step name FAILS and the step would never + // flip. setStepStatus must strip the leading "Deploy to " to recover "Staging". + const root = makeProject(t); + writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { + stageRunId: 'sr-real', stageName: 'Deploy to Staging', status: 'Succeeded', + deployedAt: '2026-06-16T10:00:00Z', activationStatus: 'Activated', + siteUrl: 'https://contoso.powerappsportals.com', + }); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'TestSite', + steps: [ + { name: 'Deploy via pipeline to Staging', status: 'pending' }, + { name: 'Activate site in Staging', status: 'pending' }, + { name: 'Test site in Staging', status: 'pending' }, + { name: 'Deploy via pipeline to Production', status: 'pending' }, + ], + }); + + refresh({ projectRoot: root, phase: 'deploy-pipeline', render: false }); + + const steps = readJson(path.join(root, 'docs', '.alm-plan-data.json')).steps; + const st = (n) => steps.find((s) => s.name === n).status; + assert.equal(st('Deploy via pipeline to Staging'), 'completed', 'real "Deploy to Staging" marker must flip the Staging deploy step'); + assert.equal(st('Activate site in Staging'), 'completed', 'activation evidence + normalized stage must flip the Activate step'); + assert.equal(st('Test site in Staging'), 'pending', 'Test stays the separate test-site step'); + assert.equal(st('Deploy via pipeline to Production'), 'pending', 'Production must NOT be touched by a "Deploy to Staging" marker'); +}); + +test('setStepStatus: exported helper normalizes a "Deploy to {label}" stage to the bare label', () => { + const { setStepStatus } = require('../lib/refresh-alm-plan-data'); + const planData = { + steps: [ + { name: 'Deploy via pipeline to Staging', status: 'pending' }, + { name: 'Deploy via pipeline to Production', status: 'pending' }, + ], + }; + const flipped = setStepStatus(planData, { keyword: /\bdeploy\b/i, stage: 'Deploy to Staging', status: 'completed' }); + assert.equal(flipped, 1); + assert.equal(planData.steps[0].status, 'completed'); + assert.equal(planData.steps[1].status, 'pending'); +}); + test('step-sync: deploy-pipeline records "failed" status when marker shows failure', (t) => { const root = makeProject(t); writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { From 16f3d3241ee0b1f04be5b9954484471c896ed533 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 17:59:15 +0530 Subject: [PATCH 06/38] Address Copilot third-pass: stable reconcile() nextStep contract reconcile() documents a `nextStep` field but three return paths omitted it, forcing callers to special-case a missing property: - .alm-deferred early return -> now nextStep: null (nothing to guide toward) - no-plan early return -> now nextStep: null (no plan to compute from) - nothing-pending return -> the plan is current with all markers but may still have unfinished checklist steps, so load the plan and return the computed nextStep instead of dropping it. The plan is now parsed once before the pending-size check and reused by both the no-op and heal paths. Contract-guard test extended to assert failed[] AND nextStep are present on every return path (null for deferred/no-plan; the next unfinished step for nothing-pending). 1187 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../scripts/lib/refresh-alm-plan-data.js | 28 +++++++++++-------- .../tests/refresh-alm-plan-data.test.js | 26 ++++++++++++----- 2 files changed, 36 insertions(+), 18 deletions(-) diff --git a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js index 3fde3f0ba..c78d47166 100644 --- a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js +++ b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js @@ -1007,12 +1007,14 @@ function reconcile({ projectRoot, render, rendererPath }) { const dataPath = path.join(projectRoot, 'docs', '.alm-plan-data.json'); const htmlPath = path.join(projectRoot, 'docs', 'alm-plan.html'); - // Respect the project-level ALM opt-out. + // Respect the project-level ALM opt-out. nextStep is null on these early paths + // (there's nothing to guide toward), kept on the return for a stable contract so + // callers never have to special-case a missing property. if (fs.existsSync(path.join(projectRoot, '.alm-deferred'))) { - return { ok: true, reconciled: [], failed: [], rendered: false, reason: 'deferred' }; + return { ok: true, reconciled: [], failed: [], rendered: false, nextStep: null, reason: 'deferred' }; } if (!fs.existsSync(dataPath)) { - return { ok: false, reconciled: [], failed: [], rendered: false, reason: 'no-plan' }; + return { ok: false, reconciled: [], failed: [], rendered: false, nextStep: null, reason: 'no-plan' }; } const planMtime = mtimeMs(dataPath); @@ -1033,8 +1035,19 @@ function reconcile({ projectRoot, render, rendererPath }) { if (pending.has('setup-pipeline')) pending.delete('ensure-pipelines-host'); if (mtimeMs(almPath(projectRoot, 'lastEnvVars')) > planMtime) pending.add(envVarPhase); + // Load the plan once. We parse it even when nothing is pending so the return + // can still carry nextStep — the plan may be current with all markers yet have + // unfinished checklist steps, and callers shouldn't lose "what to run next" (or + // have to special-case a missing property) just because no heal was needed. + let planData; + try { + planData = JSON.parse(fs.readFileSync(dataPath, 'utf8')); + } catch (e) { + throw new Error('Could not parse docs/.alm-plan-data.json: ' + e.message); + } + if (pending.size === 0) { - return { ok: true, reconciled: [], failed: [], rendered: false }; + return { ok: true, reconciled: [], failed: [], rendered: false, nextStep: computeNextStep(planData) }; } // Deterministic application order (source schema first, host/pipeline, then @@ -1046,13 +1059,6 @@ function reconcile({ projectRoot, render, rendererPath }) { ]; const phases = ORDER.filter((p) => pending.has(p)); - let planData; - try { - planData = JSON.parse(fs.readFileSync(dataPath, 'utf8')); - } catch (e) { - throw new Error('Could not parse docs/.alm-plan-data.json: ' + e.message); - } - // A single phase failing must not abort the reconcile — keep healing the rest — // but the failure must be visible (an empty catch makes a broken marker schema // impossible to diagnose: reconcile would report success while silently skipping diff --git a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js index 5d6a2a7ca..3092a5f7a 100644 --- a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js +++ b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js @@ -1829,28 +1829,40 @@ test('reconcile: heals a skipped refresh — a newer last-deploy.json is ingeste assert.equal(planData.pipelineMeta.lastDeploy.componentCount, 118); }); -test('reconcile: failed[] is always present and is an array on every return path', (t) => { - // Contract guard for the per-phase error capture (Copilot review on #191): a - // phase whose refresh throws must NOT be swallowed silently — it lands in - // result.failed = [{ phase, error }] while the other phases still heal, and - // failed[] is present (empty) on the no-op / deferred / no-plan paths too. - // (A phase only throws on a genuine marker-schema break — readJson is +test('reconcile: failed[] and nextStep are present on EVERY return path (stable contract)', (t) => { + // Contract guard (Copilot review on #191): a phase whose refresh throws must NOT + // be swallowed silently — it lands in result.failed = [{ phase, error }] while the + // other phases still heal, and BOTH failed[] and nextStep are present on the + // no-op / deferred / no-plan paths too so callers never special-case a missing + // property. (A phase only throws on a genuine marker-schema break — readJson is // defensive — so we assert the contract shape rather than fabricate a throw.) const noPlan = reconcile({ projectRoot: makeProject(t), render: false }); assert.ok(Array.isArray(noPlan.failed), 'no-plan path carries failed:[]'); + assert.ok('nextStep' in noPlan && noPlan.nextStep === null, 'no-plan path carries nextStep:null'); const deferredRoot = makeProject(t); writeJson(path.join(deferredRoot, 'docs', '.alm-plan-data.json'), { SITE_NAME: 'T' }); fs.writeFileSync(path.join(deferredRoot, '.alm-deferred'), 'deferred'); const deferred = reconcile({ projectRoot: deferredRoot, render: false }); assert.deepEqual(deferred.failed, [], 'deferred path carries failed:[]'); + assert.ok('nextStep' in deferred && deferred.nextStep === null, 'deferred path carries nextStep:null'); + // Nothing-pending but the plan still has an unfinished step -> reconcile must + // compute and return that step (not omit nextStep just because no heal ran). const idleRoot = makeProject(t); - writeJson(path.join(idleRoot, 'docs', '.alm-plan-data.json'), { SITE_NAME: 'T' }); + writeJson(path.join(idleRoot, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'T', + steps: [ + { name: 'Setup solution', status: 'completed' }, + { name: 'Setup pipeline', status: 'pending' }, + ], + }); const future = (Date.now() + 60 * 1000) / 1000; fs.utimesSync(path.join(idleRoot, 'docs', '.alm-plan-data.json'), future, future); const idle = reconcile({ projectRoot: idleRoot, render: false }); assert.deepEqual(idle.failed, [], 'nothing-pending path carries failed:[]'); + assert.deepEqual(idle.nextStep, { name: 'Setup pipeline', skill: '/power-pages:setup-pipeline' }, + 'nothing-pending path still surfaces the next unfinished step'); }); test('reconcile: idempotent no-op when the plan already reflects the markers', (t) => { From be368a6b1cb843173170d095b45ca23dc615b6d4 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 18:51:26 +0530 Subject: [PATCH 07/38] Harden stage matching: shared normalizeStageLabel for keys too MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follow-up to the step-sync fix. Factored the "strip leading 'Deploy to '" logic into a shared normalizeStageLabel() helper and routed BOTH setStepStatus AND the per-stage object-key paths (validationRuns / manualImports / activations) through it. Previously only setStepStatus stripped the prefix; the key paths used the resolved stage verbatim, so if a marker ever carried "Deploy to {label}" the renderer (which keys by the bare label) would silently miss the run. Markers there currently emit the bare label so this is defensive — it closes the whole class of mismatch in one place rather than one code path. Added a test that a "Deploy to Staging" stage keys validationRuns["Staging"]. 1188 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../scripts/lib/refresh-alm-plan-data.js | 35 ++++++++++++------- .../tests/refresh-alm-plan-data.test.js | 21 +++++++++++ 2 files changed, 44 insertions(+), 12 deletions(-) diff --git a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js index c78d47166..5cc84c8ca 100644 --- a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js +++ b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js @@ -239,21 +239,29 @@ function backfillEnvVarValuesFromSettings(planData, projectRoot) { // - Skip steps already in a terminal state UNLESS we're setting `failed` // (a retry that succeeded after a failure is recorded as completed by a // subsequent invocation; a fresh failure overrides any prior status). +// Recover the bare target label from a pipeline stage name. setup-pipeline names +// pipeline stages "Deploy to {targetLabel}" and the deploy marker carries that +// verbatim (last-deploy.json's stageName = SELECTED_STAGE.name = "Deploy to +// Staging"), but plan step names ("Deploy via pipeline to Staging", "Activate site +// in Staging") AND the per-stage object keys (validationRuns/manualImports/ +// activations, which the renderer looks up by the bare label) all use just +// "Staging". Strip a leading "Deploy to " so the step matcher and the keys line up. +// Case-preserving + trimmed (callers lowercase if they need a case-insensitive +// compare); a no-op when the prefix isn't present (a caller already passing the +// bare label, e.g. test-site --stageName "Staging"). Centralizing this here keeps +// every stage consumer consistent so the "Deploy to {label}" mismatch can't recur +// in just one code path. +function normalizeStageLabel(stage) { + if (typeof stage !== 'string') return stage; + return stage.replace(/^\s*deploy\s+to\s+/i, '').trim(); +} + function setStepStatus(planData, { keyword, stage, status }) { if (!Array.isArray(planData.steps)) return 0; if (!keyword) return 0; - // The stage filter must match the bare target LABEL embedded in the plan step - // names ("Deploy via pipeline to Staging", "Activate site in Staging", "Test - // site in Staging" — all carry "Staging"). But the run markers carry the - // pipeline STAGE name, which setup-pipeline creates as "Deploy to {label}" - // (e.g. last-deploy.json's stageName = SELECTED_STAGE.name = "Deploy to - // Staging"). A raw substring match of "deploy to staging" against "deploy via - // pipeline to staging" FAILS, so a finished deploy would never flip its step. - // Strip a leading "Deploy to " to recover the label before matching. Callers - // that already pass the bare label (e.g. test-site --stageName "Staging") are - // unaffected — the strip is a no-op when the prefix isn't present. - const rawStage = (typeof stage === 'string' && stage.length > 0) ? stage.toLowerCase() : null; - const targetStage = rawStage ? rawStage.replace(/^deploy\s+to\s+/, '').trim() : null; + const targetStage = (typeof stage === 'string' && stage.length > 0) + ? normalizeStageLabel(stage).toLowerCase() + : null; let flipped = 0; for (const step of planData.steps) { if (!step || typeof step.name !== 'string') continue; @@ -489,6 +497,7 @@ function refreshTestSite(planData, projectRoot, stageName) { if (targets.length === 1 && targets[0].label) resolvedStage = targets[0].label; } if (!resolvedStage) return planData; + resolvedStage = normalizeStageLabel(resolvedStage); planData.validationRuns = planData.validationRuns || {}; planData.validationRuns[resolvedStage] = { @@ -768,6 +777,7 @@ function refreshImportSolution(planData, projectRoot, stageName) { if (hit && hit.label) resolvedStage = hit.label; } } + if (resolvedStage) resolvedStage = normalizeStageLabel(resolvedStage); if (!resolvedStage) { // Defensive — write to a synthetic key so subsequent imports for resolvable // stages don't clobber it. Caller should pass --stageName explicitly. @@ -830,6 +840,7 @@ function refreshActivateSite(planData, projectRoot, stageName) { if (hit && hit.label) resolvedStage = hit.label; } } + if (resolvedStage) resolvedStage = normalizeStageLabel(resolvedStage); if (!resolvedStage) { resolvedStage = `unresolved-${marker.environmentUrl || 'unknown'}`; } diff --git a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js index 3092a5f7a..15cbf79bb 100644 --- a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js +++ b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js @@ -234,6 +234,27 @@ test('refresh test-site populates validationRuns[stage] from docs/alm/last-test- assert.equal(planData.validationRuns.Staging.summary.total, 12); }); +test('refresh test-site keys validationRuns by the BARE label even if given a "Deploy to {label}" stage', (t) => { + // Hardening (defense against the "Deploy to {label}" mismatch class): if a caller + // ever passes the pipeline stage NAME ("Deploy to Staging") instead of the bare + // label, the per-stage object key must still be "Staging" so the renderer (which + // looks up validationRuns by the bare label from the step name) finds it. + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'TestSite', + validationRuns: { Staging: null }, + steps: [{ name: 'Test site in Staging', status: 'pending' }], + }); + writeJson(path.join(root, 'docs', 'alm', 'last-test-site.json'), { runOutcome: 'passed', runAt: '2026-06-16T00:00:00.000Z' }); + + refresh({ projectRoot: root, phase: 'test-site', render: false, stageName: 'Deploy to Staging' }); + + const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); + assert.ok(planData.validationRuns.Staging, 'keyed by bare "Staging"'); + assert.equal(planData.validationRuns['Deploy to Staging'], undefined, 'must NOT key by the "Deploy to" stage name'); + assert.equal(planData.steps[0].status, 'completed', 'and the matching Test step still flips'); +}); + test('refresh test-site is a no-op when stageName is omitted', (t) => { const root = makeProject(t); writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { From 776356d5c7cabea5308cb19097e2290c91a4df31 Mon Sep 17 00:00:00 2001 From: T-Nid Date: Wed, 17 Jun 2026 19:06:06 +0530 Subject: [PATCH 08/38] Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- plugins/power-pages/scripts/lib/refresh-alm-plan-data.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js index 5cc84c8ca..564aa925f 100644 --- a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js +++ b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js @@ -36,7 +36,7 @@ // finalize: // - PLAN_STATUS = "Completed" // -// stdout JSON includes `nextStep: { name, skill } | null` — the first +// stdout JSON includes `nextStep: { name, skill: string | null } | null` (when ok:true) — the first // still-pending checklist step and the slash command that runs it. Execution // skills echo this so the user knows the next step to invoke (user-driven // sequencing — never auto-fired). null when every step is complete. From e59db35fe5f7a983a3b292ba97c3646c1e880d6f Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 19:20:30 +0530 Subject: [PATCH 09/38] Address Copilot fourth-pass on #191 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - refresh-alm-plan-data.js header comment: document the nextStep stdout shape as `{ name, skill: string | null } | null` (skill is null for internal/unmapped steps) — the SKILL.md guidance + the JSDoc already branch on this; the file header was the last spot still showing the old non-nullable shape. - test fixtures: align STRATEGY to the canonical schema value `pp-pipelines` (SKILL.md planData + render-alm-plan.js both use it; `pipeline` was non-schema). All 5 fixtures updated. refresh-alm-plan-data.js doesn't branch on STRATEGY, so this is correctness/representativeness only — no behavior change. - plan-alm next-steps guidance: the PP-path line implied deploy always activates; clarified that the deploy flow activates the site but if you DEFER activation (activationStatus "Pending"), run /power-pages:activate-site. 1188 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../scripts/tests/refresh-alm-plan-data.test.js | 10 +++++----- plugins/power-pages/skills/plan-alm/SKILL.md | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js index 15cbf79bb..ab23aecea 100644 --- a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js +++ b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js @@ -468,7 +468,7 @@ test('refresh deploy-pipeline ingests batchValidation block from last-deploy.jso const root = makeProject(t); writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { SITE_NAME: 'TestSite', - STRATEGY: 'pipeline', + STRATEGY: 'pp-pipelines', steps: [{ name: 'Deploy via pipeline to Staging', status: 'in_progress' }], stages: [{ label: 'Staging', envUrl: 'https://staging.crm.dynamics.com' }], }); @@ -526,7 +526,7 @@ test('refresh deploy-pipeline completes the "Activate site" step when the marker const root = makeProject(t); writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { SITE_NAME: 'TestSite', - STRATEGY: 'pipeline', + STRATEGY: 'pp-pipelines', steps: [ { name: 'Deploy via pipeline to Staging', status: 'pending' }, { name: 'Activate site in Staging', status: 'pending' }, @@ -557,7 +557,7 @@ test('refresh deploy-pipeline leaves the "Activate site" step pending when activ const root = makeProject(t); writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { SITE_NAME: 'TestSite', - STRATEGY: 'pipeline', + STRATEGY: 'pp-pipelines', steps: [ { name: 'Deploy via pipeline to Staging', status: 'pending' }, { name: 'Activate site in Staging', status: 'pending' }, @@ -586,7 +586,7 @@ test('refresh deploy-pipeline accepts legacy elapsedSecondsApprox name in batchV const root = makeProject(t); writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { SITE_NAME: 'TestSite', - STRATEGY: 'pipeline', + STRATEGY: 'pp-pipelines', steps: [{ name: 'Deploy via pipeline to Staging', status: 'in_progress' }], stages: [{ label: 'Staging', envUrl: 'https://staging.crm.dynamics.com' }], }); @@ -623,7 +623,7 @@ test('refresh deploy-pipeline sets batchValidation to null when marker omits the const root = makeProject(t); writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { SITE_NAME: 'TestSite', - STRATEGY: 'pipeline', + STRATEGY: 'pp-pipelines', steps: [{ name: 'Deploy via pipeline to Staging', status: 'in_progress' }], stages: [{ label: 'Staging', envUrl: 'https://staging.crm.dynamics.com' }], }); diff --git a/plugins/power-pages/skills/plan-alm/SKILL.md b/plugins/power-pages/skills/plan-alm/SKILL.md index 3b1d8a037..c36b2dd1d 100644 --- a/plugins/power-pages/skills/plan-alm/SKILL.md +++ b/plugins/power-pages/skills/plan-alm/SKILL.md @@ -1033,7 +1033,7 @@ Both spans are guaranteed to exist in the template — there is exactly one of e **Next-steps guidance (option 1 — print to the user, do NOT invoke):** > "Plan approved and saved. **plan-alm doesn't deploy** — run these next, in order. Each detects this plan and proceeds without re-asking, then updates the plan as it completes: -> - **PP Pipelines path:** `/power-pages:setup-solution` → `/power-pages:setup-pipeline` → `/power-pages:deploy-pipeline` (once per target stage; the deploy flow activates the site) → `/power-pages:test-site` to validate each stage. +> - **PP Pipelines path:** `/power-pages:setup-solution` → `/power-pages:setup-pipeline` → `/power-pages:deploy-pipeline` (once per target stage; the deploy flow activates the site for you — but if you defer activation, run `/power-pages:activate-site`) → `/power-pages:test-site` to validate each stage. > - **Manual path:** `/power-pages:setup-solution` → `/power-pages:export-solution` → review the zip → `/power-pages:import-solution` (once per target). > Skip any step marked *already set up*. You can re-open the plan any time at `docs/alm-plan.html`." From ed2ad59119e345e898542f419f95af32b842883e Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 12:30:30 +0530 Subject: [PATCH 10/38] EDM (enhanced/standard data-model) Power Pages site support in ALM discovery MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Enhanced data-model sites are downloaded with `pac pages download` (not download-code-site): they have NO `powerpages.config.json` and no SPA build output — just `.powerpages-site/` with a config tree + `website.yml`. The shared discovery helpers hard-required `powerpages.config.json`, so every ALM skill broke on EDM sites. - `findProjectRoot` (validation-helpers.js): treats a `.powerpages-site/` directory as a project-root marker, not just `powerpages.config.json`. - `detect-project-context.js`: falls back to `.powerpages-site/website.yml` (`id`→websiteRecordId, `name`→siteName) when no `powerpages.config.json`; returns a new `siteType` ("code" | "data-model"); exits 1 only when neither marker exists. - `check-activation-status.js`: same fallback (verified live against an EDM site → resolves identity + activation status instead of erroring). Backward-compatible: `powerpages.config.json` stays the primary signal for code sites. PR 2 of 4 (stacked, on plan-alm-plan-only). Co-Authored-By: Claude Opus 4.8 (1M context) --- .../power-pages/.claude-plugin/plugin.json | 2 +- plugins/power-pages/AGENTS.md | 2 +- .../scripts/check-activation-status.js | 41 ++++--- .../scripts/lib/detect-project-context.js | 106 ++++++++++++++---- .../scripts/lib/validation-helpers.js | 20 +++- .../tests/detect-project-context.test.js | 69 +++++++++++- .../scripts/tests/validation-helpers.test.js | 40 +++++++ 7 files changed, 235 insertions(+), 45 deletions(-) diff --git a/plugins/power-pages/.claude-plugin/plugin.json b/plugins/power-pages/.claude-plugin/plugin.json index 3021445a0..25996f2b7 100644 --- a/plugins/power-pages/.claude-plugin/plugin.json +++ b/plugins/power-pages/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "power-pages", - "version": "2.2.0", + "version": "2.3.0", "description": "Create and deploy Power Pages sites using modern development approaches. Supports code sites (SPAs) with React, Angular, Vue, or Astro. Includes ALM orchestration (plan-alm) with a solution-splitting decision tree, per-solution pipelines, Azure Blob asset advisory, manifest schema v2 for multi-solution deployments, and force-link remediation for cross-host pipeline migrations.", "author": { "name": "Microsoft", diff --git a/plugins/power-pages/AGENTS.md b/plugins/power-pages/AGENTS.md index 9d9ecc10d..959684efa 100644 --- a/plugins/power-pages/AGENTS.md +++ b/plugins/power-pages/AGENTS.md @@ -196,7 +196,7 @@ Shared lib modules live at `scripts/lib/` and are imported by other scripts via #### ALM Prerequisites & Context - `scripts/lib/verify-alm-prerequisites.js`: Verifies all prerequisites for ALM skills — PAC CLI installed + authenticated (`pac env who`), Azure CLI installed + logged in, Dataverse API reachable (`WhoAmI`). Args: `--envUrl` (opt, overrides env from PAC CLI), `--require-manifest` (fails if `.solution-manifest.json` not found). Output: `{ envUrl, token, userId, organizationId, tenantId }`. Exit 0 on success, exit 1 on any failure. Used by `setup-solution`, `export-solution`, `import-solution`, `setup-pipeline`, `deploy-pipeline`, `plan-alm`. -- `scripts/lib/detect-project-context.js`: Reads Power Pages project context files from the project root — `powerpages.config.json`, `.solution-manifest.json`, and `.datamodel-manifest.json`. Args: `--projectRoot` (opt, auto-discovered from cwd if omitted). Output: `{ projectRoot, siteName, websiteRecordId, environmentUrl, solutionManifest, datamodelManifest }`. Exit 0 on success, exit 1 if `powerpages.config.json` not found. +- `scripts/lib/detect-project-context.js`: Reads Power Pages project context from the project root. Resolves site identity in order: (1) `powerpages.config.json` → `siteType: "code"` (SPA sites); (2) `.powerpages-site/website.yml` → `siteType: "data-model"` (standard/enhanced data-model "EDM" sites, which have **no** `powerpages.config.json` — `id`→`websiteRecordId`, `name`→`siteName`, `environmentUrl: null` since the local files carry no env URL). Also reads `.solution-manifest.json` and `.datamodel-manifest.json`. Args: `--projectRoot` (opt, auto-discovered from cwd if omitted). Output: `{ projectRoot, siteType, siteName, websiteRecordId, environmentUrl, solutionManifest, datamodelManifest }`. Exit 0 on success, exit 1 only if **neither** `powerpages.config.json` nor `.powerpages-site/website.yml` is found. Note: `findProjectRoot` (in `validation-helpers.js`) likewise treats a `.powerpages-site/` directory as a project-root marker, not just `powerpages.config.json`, so data-model sites are discoverable. - `scripts/lib/alm-paths.js`: Single source of truth for ALM artifact paths. Exports `ALM_DIR` (always `docs/alm`), `FILE_NAMES` (frozen object mapping logical key → filename for all 14 ALM artifacts), `almDir(projectRoot) → path`, `almPath(projectRoot, key) → path`, `ensureAlmDir(projectRoot) → path` (mkdir -p idempotent). Every ALM-only state file (5 plan/decision JSONs + 9 `last-*.json` skill-run markers including `last-export.json`) writes under `/docs/alm/`. **Always resolve through this helper** — never inline a raw `docs/alm/...` path in a script. Files intentionally NOT moved here (and not in `FILE_NAMES`): `.solution-manifest.json`, `.datamodel-manifest.json`, `.alm-config.json`, `.alm-deferred`, `deployment-settings.json`. Adding a new ALM marker means adding its key + filename to `FILE_NAMES` first; `almPath` throws on unknown keys to catch typos at call-site. - `scripts/lib/check-alm-plan.js`: Phase 0 gate helper used by every ALM skill to detect (a) whether an ALM plan exists for this project, (b) whether the user has explicitly deferred ALM via the `.alm-deferred` marker, and (c) whether an existing plan is stale (the source solution was modified after the plan was generated). Args: `--projectRoot`, `--envUrl` (opt — required for staleness check), `--token` (opt), `--solutionId` (opt — required for staleness check). Output: `{ exists, deferred, deferral, planPath, htmlPath, stale, staleness: { reason, detail }, generatedAt, planStatus, solution: {...} }`. Without env/solution context the helper does an existence-only check; with them it queries Dataverse for `solutions(solutionId)?$select=modifiedon` and compares against `planData.generatedAt`. Used by `setup-solution`, `setup-pipeline`, `deploy-pipeline`, `export-solution`, `import-solution`, `configure-env-variables`, `ensure-pipelines-host`, `force-link-environment` Phase 0 gates — the "fail closed when no plan" pattern. - `scripts/lib/resolve-target-solution.js`: Resolves "which solution should this new Dataverse record land in?" Implements the strict 3-step order from the ALM-aware-by-default principle: (1) explicit `--solutionUniqueName` (or equivalent caller arg) wins; (2) `.solution-manifest.json` in the project root; (3) neither → throw `NoSolutionConfiguredError`. **The module NEVER auto-picks from Dataverse** — interactive prompt UX is the caller's responsibility (catch the error, present an `AskUserQuestion` list, re-invoke with `explicit` populated). Callers that need to confirm the solution still exists in Dataverse can pass `verifyExists: true`; the module then enriches the result with `{ solutionId, version, ismanaged }`. Component-creation scripts must require this helper and pass through `--solutionUniqueName` so records land in the user's solution instead of `Default`. diff --git a/plugins/power-pages/scripts/check-activation-status.js b/plugins/power-pages/scripts/check-activation-status.js index f49537186..4210f58ba 100644 --- a/plugins/power-pages/scripts/check-activation-status.js +++ b/plugins/power-pages/scripts/check-activation-status.js @@ -16,6 +16,7 @@ const fs = require('fs'); const path = require('path'); const { execSync } = require('child_process'); const { findPath, getPacAuthInfo, getAuthToken, makeRequest, CLOUD_TO_API } = require('./lib/validation-helpers'); +const { readWebsiteYml } = require('./lib/detect-project-context'); function output(obj) { process.stdout.write(JSON.stringify(obj)); @@ -27,26 +28,36 @@ const args = process.argv.slice(2); const rootIdx = args.indexOf('--projectRoot'); const projectRoot = rootIdx !== -1 ? args[rootIdx + 1] : process.cwd(); -// --- Read siteName from powerpages.config.json --- -const configPath = findPath(projectRoot, 'powerpages.config.json'); -if (!configPath) { - output({ error: 'powerpages.config.json not found' }); -} - +// --- Read site identity from powerpages.config.json (code/SPA sites) OR +// .powerpages-site/website.yml (data-model / enhanced data model "EDM" sites, +// which have no powerpages.config.json). website.yml carries both the site +// name and the website GUID, so EDM sites skip the pac-pages-list lookup below. --- let siteName; -try { - const config = JSON.parse(fs.readFileSync(configPath, 'utf8')); - siteName = config.siteName; -} catch { - output({ error: 'Failed to parse powerpages.config.json' }); +let websiteRecordId = null; +const configPath = findPath(projectRoot, 'powerpages.config.json'); +if (configPath) { + try { + const config = JSON.parse(fs.readFileSync(configPath, 'utf8')); + siteName = config.siteName; + websiteRecordId = config.websiteRecordId || null; + } catch { + output({ error: 'Failed to parse powerpages.config.json' }); + } +} else { + const websiteYmlPath = findPath(projectRoot, path.join('.powerpages-site', 'website.yml')); + const site = websiteYmlPath ? readWebsiteYml(websiteYmlPath) : null; + if (site) { + siteName = site.name; + websiteRecordId = site.id || null; + } } if (!siteName) { - output({ error: 'siteName not found in powerpages.config.json' }); + output({ error: 'Site name not found — looked in powerpages.config.json and .powerpages-site/website.yml' }); } -// --- Get websiteRecordId from pac pages list --- -let websiteRecordId = null; -try { +// --- Get websiteRecordId from pac pages list (only when not already known, +// e.g. a code site whose config omitted it). EDM sites already have it from website.yml. --- +if (!websiteRecordId) try { const pacOutput = execSync('pac pages list', { encoding: 'utf8', timeout: 15000 }); // pac pages list outputs a table with columns. Find the row matching siteName. // Column headers vary but Website Record ID is always a GUID column. diff --git a/plugins/power-pages/scripts/lib/detect-project-context.js b/plugins/power-pages/scripts/lib/detect-project-context.js index 2d74329fb..26e101476 100644 --- a/plugins/power-pages/scripts/lib/detect-project-context.js +++ b/plugins/power-pages/scripts/lib/detect-project-context.js @@ -1,7 +1,17 @@ #!/usr/bin/env node // Reads Power Pages project context files from the project root. -// Locates powerpages.config.json, .solution-manifest.json, and .datamodel-manifest.json. +// Locates powerpages.config.json (code/SPA sites) OR .powerpages-site/website.yml +// (data-model config sites, standard and enhanced data model), plus +// .solution-manifest.json and .datamodel-manifest.json. +// +// Site identity resolution order (first match wins): +// 1. powerpages.config.json -> siteType "code" (SPA sites; has siteName, +// websiteRecordId, environmentUrl) +// 2. .powerpages-site/website.yml -> siteType "data-model" (enhanced/standard +// data-model sites from `pac pages download`; the +// YAML carries `id` and `name`, but no environment URL — +// callers re-confirm the env via `pac env who`) // // Usage: node detect-project-context.js [--projectRoot ] // @@ -11,14 +21,16 @@ // Output (JSON to stdout): // { // "projectRoot": "...", +// "siteType": "code" | "data-model", // "siteName": "...", // "websiteRecordId": "...", -// "environmentUrl": "...", +// "environmentUrl": "..." | null, // "solutionManifest": { ... } | null, // "datamodelManifest": { ... } | null // } // -// Exit 0 on success, exit 1 if powerpages.config.json not found. +// Exit 0 on success, exit 1 if neither powerpages.config.json nor +// .powerpages-site/website.yml is found (not a Power Pages project). 'use strict'; @@ -45,6 +57,32 @@ function readJsonFile(filePath) { } } +// Minimal reader for the flat `.powerpages-site/website.yml` (a simple `key: value` +// per line — no nesting). Returns { id, name } or null. Zero-dependency by design +// (the plugin ships no YAML library); only the two identity keys are needed here. +function readWebsiteYml(filePath) { + let raw; + try { + raw = fs.readFileSync(filePath, 'utf8'); + } catch { + return null; + } + const out = {}; + for (const line of raw.split(/\r?\n/)) { + const m = /^([A-Za-z0-9_]+):\s*(.*)$/.exec(line); + if (!m) continue; + const key = m[1]; + let value = m[2].trim(); + // Strip surrounding quotes a YAML writer may add. + if ((value.startsWith('"') && value.endsWith('"')) || + (value.startsWith("'") && value.endsWith("'"))) { + value = value.slice(1, -1); + } + if (key === 'id' || key === 'name') out[key] = value; + } + return (out.id || out.name) ? out : null; +} + function detectProjectContext(options = {}) { const startDir = options.projectRoot || process.cwd(); const projectRoot = options.projectRoot @@ -53,31 +91,57 @@ function detectProjectContext(options = {}) { if (!projectRoot) { throw new Error( - 'powerpages.config.json not found. Run this command from a Power Pages project directory.' + 'No Power Pages project found. Run this command from a site project directory ' + + '(one containing powerpages.config.json for a code site, or .powerpages-site/ ' + + 'for a data-model site).' ); } + const solutionManifest = readJsonFile(path.join(projectRoot, '.solution-manifest.json')); + const datamodelManifest = readJsonFile(path.join(projectRoot, '.datamodel-manifest.json')); + + // 1. Code/SPA site — powerpages.config.json is the source of truth. const configPath = path.join(projectRoot, 'powerpages.config.json'); - if (!fs.existsSync(configPath)) { - throw new Error(`powerpages.config.json not found at: ${configPath}`); + if (fs.existsSync(configPath)) { + const config = readJsonFile(configPath); + if (!config) { + throw new Error(`Failed to parse powerpages.config.json at: ${configPath}`); + } + return { + projectRoot, + siteType: 'code', + siteName: config.siteName || null, + websiteRecordId: config.websiteRecordId || null, + environmentUrl: config.environmentUrl || null, + solutionManifest, + datamodelManifest, + }; } - const config = readJsonFile(configPath); - if (!config) { - throw new Error(`Failed to parse powerpages.config.json at: ${configPath}`); + // 2. Data-model (standard/enhanced) site — identity comes from + // .powerpages-site/website.yml (`id` -> websiteRecordId, `name` -> siteName). + // There is no environment URL in the local files; callers re-confirm via `pac env who`. + const websiteYmlPath = path.join(projectRoot, '.powerpages-site', 'website.yml'); + if (fs.existsSync(websiteYmlPath)) { + const site = readWebsiteYml(websiteYmlPath); + if (!site) { + throw new Error(`Could not read site id/name from: ${websiteYmlPath}`); + } + return { + projectRoot, + siteType: 'data-model', + siteName: site.name || null, + websiteRecordId: site.id || null, + environmentUrl: null, + solutionManifest, + datamodelManifest, + }; } - const solutionManifest = readJsonFile(path.join(projectRoot, '.solution-manifest.json')); - const datamodelManifest = readJsonFile(path.join(projectRoot, '.datamodel-manifest.json')); - - return { - projectRoot, - siteName: config.siteName || null, - websiteRecordId: config.websiteRecordId || null, - environmentUrl: config.environmentUrl || null, - solutionManifest, - datamodelManifest, - }; + throw new Error( + `No site identity found at ${projectRoot}: neither powerpages.config.json nor ` + + '.powerpages-site/website.yml is present.' + ); } // CLI entry point @@ -94,4 +158,4 @@ if (require.main === module) { } } -module.exports = { detectProjectContext }; +module.exports = { detectProjectContext, readWebsiteYml }; diff --git a/plugins/power-pages/scripts/lib/validation-helpers.js b/plugins/power-pages/scripts/lib/validation-helpers.js index c36d89e81..5667795c5 100644 --- a/plugins/power-pages/scripts/lib/validation-helpers.js +++ b/plugins/power-pages/scripts/lib/validation-helpers.js @@ -89,14 +89,23 @@ function findPath(dir, target) { } /** - * Finds the project root directory (containing powerpages.config.json). + * Finds the project root directory of a Power Pages site. + * + * A project root is marked by EITHER: + * - `powerpages.config.json` — code/SPA sites (`pac pages download-code-site`), OR + * - a `.powerpages-site/` directory — data-model config sites (`pac pages download`, + * standard or enhanced data model). These have NO `powerpages.config.json`. + * + * Code sites have both markers; data-model (e.g. enhanced data model) sites have only + * `.powerpages-site/`. Checking for either makes root discovery work for both site types. + * * @returns {string|null} Project root path, or null */ function findProjectRoot(dir) { let current = path.resolve(dir); while (true) { - const configPath = path.join(current, 'powerpages.config.json'); - if (fs.existsSync(configPath)) { + if (fs.existsSync(path.join(current, 'powerpages.config.json')) || + fs.existsSync(path.join(current, '.powerpages-site'))) { return current; } @@ -107,8 +116,11 @@ function findProjectRoot(dir) { current = parent; } + // Fallback: search subdirectories for either marker (config first, then .powerpages-site/). const fallbackConfigPath = findPath(dir, 'powerpages.config.json'); - return fallbackConfigPath ? path.dirname(fallbackConfigPath) : null; + if (fallbackConfigPath) return path.dirname(fallbackConfigPath); + const fallbackSiteDir = findPath(dir, '.powerpages-site'); + return fallbackSiteDir ? path.dirname(fallbackSiteDir) : null; } /** diff --git a/plugins/power-pages/scripts/tests/detect-project-context.test.js b/plugins/power-pages/scripts/tests/detect-project-context.test.js index be5279751..b550e964d 100644 --- a/plugins/power-pages/scripts/tests/detect-project-context.test.js +++ b/plugins/power-pages/scripts/tests/detect-project-context.test.js @@ -4,19 +4,82 @@ const fs = require('fs'); const os = require('os'); const path = require('path'); -const { detectProjectContext } = require('../lib/detect-project-context'); +const { detectProjectContext, readWebsiteYml } = require('../lib/detect-project-context'); const { createTempProject, writeProjectFile } = require('./test-utils'); -test('detectProjectContext throws when powerpages.config.json is missing', (t) => { +test('detectProjectContext throws when neither config nor .powerpages-site/website.yml exists', (t) => { const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ctx-test-')); t.after(() => fs.rmSync(dir, { recursive: true, force: true })); assert.throws( () => detectProjectContext({ projectRoot: dir }), - /powerpages.config.json not found/ + /neither powerpages\.config\.json nor/ ); }); +test('detectProjectContext: code site (powerpages.config.json) reports siteType "code"', (t) => { + const projectRoot = createTempProject(t); + writeProjectFile(projectRoot, 'powerpages.config.json', JSON.stringify({ + siteName: 'Code Site', + websiteRecordId: 'aabbccdd-1234-5678-abcd-00000000000c', + environmentUrl: 'https://org.crm.dynamics.com', + })); + + const result = detectProjectContext({ projectRoot }); + assert.equal(result.siteType, 'code'); + assert.equal(result.siteName, 'Code Site'); +}); + +test('detectProjectContext: enhanced data-model site resolves identity from .powerpages-site/website.yml', (t) => { + const projectRoot = createTempProject(t); + // No powerpages.config.json — this is an EDM / data-model config site. + writeProjectFile( + projectRoot, + '.powerpages-site/website.yml', + [ + 'defaultlanguage: 32cc32f6-8665-f111-a826-000d3a5a7777', + 'id: 2ecc32f6-8665-f111-a826-000d3a5a7777', + 'name: Application processing EDM site - permitapplication-elyyn', + 'statecode: 0', + '', + ].join('\n') + ); + + const result = detectProjectContext({ projectRoot }); + assert.equal(result.siteType, 'data-model'); + assert.equal(result.websiteRecordId, '2ecc32f6-8665-f111-a826-000d3a5a7777'); + assert.equal(result.siteName, 'Application processing EDM site - permitapplication-elyyn'); + // Data-model sites carry no environment URL locally — callers re-confirm via `pac env who`. + assert.equal(result.environmentUrl, null); +}); + +test('detectProjectContext: config site wins over website.yml when both exist (code-site precedence)', (t) => { + const projectRoot = createTempProject(t); + writeProjectFile(projectRoot, 'powerpages.config.json', JSON.stringify({ + siteName: 'Code Wins', + websiteRecordId: 'config-guid', + environmentUrl: 'https://org.crm.dynamics.com', + })); + writeProjectFile(projectRoot, '.powerpages-site/website.yml', 'id: yml-guid\nname: YAML Name\n'); + + const result = detectProjectContext({ projectRoot }); + assert.equal(result.siteType, 'code'); + assert.equal(result.websiteRecordId, 'config-guid'); +}); + +test('readWebsiteYml: extracts id + name, strips quotes, ignores other keys', () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'yml-test-')); + const p = path.join(dir, 'website.yml'); + fs.writeFileSync(p, 'id: "abc-123"\nname: \'Quoted Site\'\nstatecode: 0\n'); + try { + const site = readWebsiteYml(p); + assert.deepEqual(site, { id: 'abc-123', name: 'Quoted Site' }); + assert.equal(readWebsiteYml(path.join(dir, 'missing.yml')), null); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } +}); + test('detectProjectContext returns siteName and websiteRecordId from config', (t) => { const projectRoot = createTempProject(t); writeProjectFile(projectRoot, 'powerpages.config.json', JSON.stringify({ diff --git a/plugins/power-pages/scripts/tests/validation-helpers.test.js b/plugins/power-pages/scripts/tests/validation-helpers.test.js index 620e3a107..9b7e2dc87 100644 --- a/plugins/power-pages/scripts/tests/validation-helpers.test.js +++ b/plugins/power-pages/scripts/tests/validation-helpers.test.js @@ -33,3 +33,43 @@ test('getAuthToken calls az account get-access-token without --allow-no-subscrip ); assert.match(capturedCommand, /--resource "https:\/\/example\.crm\.dynamics\.com"/); }); + +// --- findProjectRoot: EDM / data-model site awareness ------------------------ + +test('findProjectRoot: recognizes a .powerpages-site/ directory as a project root (data-model/EDM sites)', (t) => { + const fs = require('fs'); + const os = require('os'); + const { findProjectRoot } = require(helpersPath); + + // EDM/data-model site: .powerpages-site/ present, NO powerpages.config.json. + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'fpr-edm-')); + t.after(() => fs.rmSync(root, { recursive: true, force: true })); + fs.mkdirSync(path.join(root, '.powerpages-site'), { recursive: true }); + fs.writeFileSync(path.join(root, '.powerpages-site', 'website.yml'), 'id: x\nname: y\n'); + + assert.equal(findProjectRoot(root), path.resolve(root)); +}); + +test('findProjectRoot: still recognizes powerpages.config.json (code sites)', (t) => { + const fs = require('fs'); + const os = require('os'); + const { findProjectRoot } = require(helpersPath); + + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'fpr-code-')); + t.after(() => fs.rmSync(root, { recursive: true, force: true })); + fs.writeFileSync(path.join(root, 'powerpages.config.json'), '{}'); + + assert.equal(findProjectRoot(root), path.resolve(root)); +}); + +test('findProjectRoot: returns null when neither marker is present', (t) => { + const fs = require('fs'); + const os = require('os'); + const { findProjectRoot } = require(helpersPath); + + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'fpr-none-')); + t.after(() => fs.rmSync(root, { recursive: true, force: true })); + + assert.equal(findProjectRoot(root), null); +}); + From c5db65401167f12922e6fd94142acbbb678bf84d Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 12:37:31 +0530 Subject: [PATCH 11/38] Site-referenced table discovery + dependency-aware solution splitting MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replaces publisher-prefix table discovery (which over-counts catastrophically with a shared/default publisher — a 6-table site matched 22 unrelated tables — and misses real tables from a different prefix) with site-referenced scoping: the custom tables the site's `.powerpages-site/table-permissions/` (+ datamodel manifest) actually reference, intersected with the env's custom-unmanaged tables. SME-confirmed: table permissions are the complete signal. Replaces the old one-solution-per-table-name-stem split heuristic (which produced ~one solution per table — e.g. a 21-solution split) with a dependency-aware, capacity-bounded packer: union-find connected-component clusters over table relationships (lookups + N:N), then first-fit-decreasing bin-packing of whole clusters into the fewest solutions under maxTableCount/maxSchemaAttrs, capped at maxSchemaSplitSolutions (8). The split trigger + thresholds are unchanged — only the packing. New shared libs: - resolve-site-tables.js — site-referenced table scoping (single source of truth) - query-metadata.js — consolidated custom-unmanaged-table query - query-table-relationships.js — relationship edges (lib; audit-permissions CLI is now a thin wrapper) - validation-helpers.js — odataGet/odataGetAll shared paginator estimate-solution-size.js now emits tableCountScope + tableRelationships[]; compute-split-plan.js consumes the edges. setup-solution Phase 5.2.D uses the shared discovery helper. 1209 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../power-pages/.claude-plugin/plugin.json | 2 +- plugins/power-pages/AGENTS.md | 8 +- .../power-pages/scripts/lib/alm-thresholds.js | 6 + .../scripts/lib/compute-split-plan.js | 127 +++++++++++++++--- .../scripts/lib/discover-site-components.js | 62 +++++---- .../scripts/lib/estimate-solution-size.js | 118 +++++++++++----- .../power-pages/scripts/lib/query-metadata.js | 44 ++++++ .../scripts/lib/query-table-relationships.js | 65 +++++++++ .../scripts/lib/resolve-site-tables.js | 103 ++++++++++++++ .../scripts/lib/validation-helpers.js | 49 +++++++ .../scripts/tests/compute-split-plan.test.js | 101 +++++++++++++- .../tests/discover-site-components.test.js | 40 +++++- .../tests/estimate-solution-size.test.js | 84 ++++++++++++ .../integration/discover-integration.test.js | 24 ++++ .../scripts/tests/query-metadata.test.js | 44 ++++++ .../tests/query-table-relationships.test.js | 52 +++++++ .../scripts/tests/resolve-site-tables.test.js | 98 ++++++++++++++ .../scripts/tests/validation-helpers.test.js | 25 ++++ .../scripts/query-table-relationships.js | 35 ++--- .../skills/setup-solution/SKILL.md | 22 +-- 20 files changed, 984 insertions(+), 125 deletions(-) create mode 100644 plugins/power-pages/scripts/lib/query-metadata.js create mode 100644 plugins/power-pages/scripts/lib/query-table-relationships.js create mode 100644 plugins/power-pages/scripts/lib/resolve-site-tables.js create mode 100644 plugins/power-pages/scripts/tests/query-metadata.test.js create mode 100644 plugins/power-pages/scripts/tests/query-table-relationships.test.js create mode 100644 plugins/power-pages/scripts/tests/resolve-site-tables.test.js diff --git a/plugins/power-pages/.claude-plugin/plugin.json b/plugins/power-pages/.claude-plugin/plugin.json index 25996f2b7..13763c7f2 100644 --- a/plugins/power-pages/.claude-plugin/plugin.json +++ b/plugins/power-pages/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "power-pages", - "version": "2.3.0", + "version": "2.4.0", "description": "Create and deploy Power Pages sites using modern development approaches. Supports code sites (SPAs) with React, Angular, Vue, or Astro. Includes ALM orchestration (plan-alm) with a solution-splitting decision tree, per-solution pipelines, Azure Blob asset advisory, manifest schema v2 for multi-solution deployments, and force-link remediation for cross-host pipeline migrations.", "author": { "name": "Microsoft", diff --git a/plugins/power-pages/AGENTS.md b/plugins/power-pages/AGENTS.md index 959684efa..64555df68 100644 --- a/plugins/power-pages/AGENTS.md +++ b/plugins/power-pages/AGENTS.md @@ -204,8 +204,12 @@ Shared lib modules live at `scripts/lib/` and are imported by other scripts via #### Solution Splitting Decision Tree (v1.3.0+) - `scripts/lib/alm-thresholds.js`: Central default threshold constants for the split decision tree. Loads optional `.alm-config.json` from project root and merges over defaults. Exports `DEFAULTS`, `DEFAULT_CONFIG`, `loadConfig(projectRoot)`, `classifyTier(value, greenUpperExclusive, yellowUpperExclusive)`, `deepMerge(target, source)`. Used by `estimate-solution-size.js` and `compute-split-plan.js`. -- `scripts/lib/estimate-solution-size.js`: Estimates solution size + component counts by querying Dataverse. Args: `--envUrl`, `--websiteRecordId`, `--token` (opt), `--publisherPrefix` (opt), `--siteName` (opt), `--solutionId` (opt — scopes env var count to the target solution; without it falls back to a publisher-prefix tenant-wide query that overcounts when prefix is shared), `--datamodelManifest` (opt), `--projectRoot` (opt — enables disk cross-check: walks the local build-output directory (`dist/`, `public-output/`, `build/`, `.output/`) and surfaces the byte total). Output: `{ totalSizeMB, componentCountSiteTotal, componentCountSiteActionable, componentCountInSolution, orphansOnSite, tableCount, schemaAttrCount, webFilesAggregateMB, webFilesIndividual[], webFileCount, webFileSampleSize, webFilesDiskMeasuredMB, webFilesDiskMeasuredPath, webFilesDiskFileCount, cloudFlowCount, botCount, envVarCount, envVarCountScope, envVarCountTenantWide, mediaRatio, siteType, tables[], breakdown, estimationMethod, estimationAccuracyPct, truncationSuspected, truncationWarnings, ppcGroundTruthCount }`. Metadata-based estimation with ±15% caveat. Web-file size is measured via stratified sample (first 50 + middle 50 + last 50, cap 150) scaled to full count. Disk fields are null unless `--projectRoot` was passed AND a build-output directory was found. Truncation canaries fire when Dataverse pagination disagrees with `@odata.count`, when ppcs land on a page-size boundary, when sampled average bytes/file < 1 KB at scale, or when the disk total exceeds the Dataverse total by >2× — any signal flips `truncationSuspected: true` with a per-cause `truncationWarnings[]` entry. Used by `plan-alm` Phase 1 Step 10. -- `scripts/lib/compute-split-plan.js`: Runs the split decision tree against a size-estimate blob. Args: `--estimate `, `--projectRoot` (opt — for `.alm-config.json` overrides), `--siteName` (opt), `--publisherPrefix` (opt). Output: `{ sizeAnalysis, assetAdvisory, splitStrategy, appliedStrategies, compositeSubPartitioned, proposedSolutions[], recommendations[], truncationSuspected, truncationWarnings }`. Evaluates strategies in priority order: Strategy 3 (Schema Segmentation) → Strategy 1 (Layer Split) → Strategy 2 (Change-Frequency) → Strategy 4 (Config Isolation). Strategy 4 stacks additively. Strategy 1 also runs a composite sub-partition pass: when Core still exceeds the size OR component-count cap after Web Assets are peeled off, Core is replaced with change-frequency-shaped sub-children (`_Foundation`/`_Config`/`_Content`, plus `_Integration` whenever the parent had any flows or bots — coverage takes priority over the `changeFreqMinFlows` heuristic, which governs only the TOP-LEVEL strategy choice). When additive Strategy 4 is firing concurrently (a top-level `_EnvVars` solution), `_Config` drops `Environment Variable` from its componentTypes to avoid double-claim; when it isn't, `_Config` absorbs env vars so they have an owner. Sub-partitioning sets `compositeSubPartitioned: true` and appends `composite-sub-partition` to `appliedStrategies`. `validateSplits` checks BOTH the size AND component-count cap per split (skipping `isFutureBuffer` solutions). Supports `.alm-config.json` overrides including `strategyOverride` to bypass the tree. See `solution-splitting-logic.md` spec in design docs for full logic. +- `scripts/lib/estimate-solution-size.js`: Estimates solution size + component counts by querying Dataverse. Args: `--envUrl`, `--websiteRecordId`, `--token` (opt), `--publisherPrefix` (opt), `--siteName` (opt), `--solutionId` (opt — scopes env var count to the target solution; without it falls back to a publisher-prefix tenant-wide query that overcounts when prefix is shared), `--datamodelManifest` (opt), `--projectRoot` (opt — enables disk cross-check: walks the local build-output directory (`dist/`, `public-output/`, `build/`, `.output/`) and surfaces the byte total). Output: `{ totalSizeMB, componentCountSiteTotal, componentCountSiteActionable, componentCountInSolution, orphansOnSite, tableCount, tableCountScope, schemaAttrCount, webFilesAggregateMB, webFilesIndividual[], webFileCount, webFileSampleSize, webFilesDiskMeasuredMB, webFilesDiskMeasuredPath, webFilesDiskFileCount, cloudFlowCount, botCount, envVarCount, envVarCountScope, envVarCountTenantWide, mediaRatio, siteType, tables[], tableRelationships[], breakdown, estimationMethod, estimationAccuracyPct, truncationSuspected, truncationWarnings, ppcGroundTruthCount }`. **Table discovery is site-referenced, NOT publisher-prefix:** `tableCount`/`tables[]` are scoped to the custom tables the site actually references — its `.powerpages-site/table-permissions/` (+ datamodel manifest) intersected with the env's custom-unmanaged tables (via `resolve-site-tables.js` + `query-metadata.js`). `tableCountScope` ∈ `"site-referenced" | "manifest-only" | "unavailable"` (the last → 0 tables, never an env-wide prefix dump). `--publisherPrefix` now scopes ONLY the env var count, not tables. `tableRelationships[]` are `[a,b]` dependency edges (lookups + N:N, via `query-table-relationships.js`) among the scoped tables, consumed by `compute-split-plan.js` to cluster related tables into the same solution. Metadata-based estimation with ±15% caveat. Web-file size is measured via stratified sample (first 50 + middle 50 + last 50, cap 150) scaled to full count. Disk fields are null unless `--projectRoot` was passed AND a build-output directory was found. Truncation canaries fire when Dataverse pagination disagrees with `@odata.count`, when ppcs land on a page-size boundary, when sampled average bytes/file < 1 KB at scale, or when the disk total exceeds the Dataverse total by >2× — any signal flips `truncationSuspected: true` with a per-cause `truncationWarnings[]` entry. Used by `plan-alm` Phase 1 Step 10. +- `scripts/lib/compute-split-plan.js`: Runs the split decision tree against a size-estimate blob. Args: `--estimate `, `--projectRoot` (opt — for `.alm-config.json` overrides), `--siteName` (opt), `--publisherPrefix` (opt). Output: `{ sizeAnalysis, assetAdvisory, splitStrategy, appliedStrategies, compositeSubPartitioned, proposedSolutions[], recommendations[], truncationSuspected, truncationWarnings }`. Evaluates strategies in priority order: Strategy 3 (Schema Segmentation) → Strategy 1 (Layer Split) → Strategy 2 (Change-Frequency) → Strategy 4 (Config Isolation). **Schema Segmentation is dependency-aware + capacity-bounded:** it builds connected-component clusters from `estimate.tableRelationships` (union-find), then bin-packs whole clusters (never splitting a relationship) into the fewest solutions that keep each under `maxTableCount`/`maxSchemaAttrs`, capped at `maxSchemaSplitSolutions` (default 8). This replaced the old one-solution-per-table-name-stem heuristic that produced ~one solution per table. An indivisible cluster over the cap stays whole and raises an oversized-cluster `recommendations[]` warning. The split trigger + thresholds are unchanged — only the packing. Strategy 4 stacks additively. Strategy 1 also runs a composite sub-partition pass: when Core still exceeds the size OR component-count cap after Web Assets are peeled off, Core is replaced with change-frequency-shaped sub-children (`_Foundation`/`_Config`/`_Content`, plus `_Integration` whenever the parent had any flows or bots — coverage takes priority over the `changeFreqMinFlows` heuristic, which governs only the TOP-LEVEL strategy choice). When additive Strategy 4 is firing concurrently (a top-level `_EnvVars` solution), `_Config` drops `Environment Variable` from its componentTypes to avoid double-claim; when it isn't, `_Config` absorbs env vars so they have an owner. Sub-partitioning sets `compositeSubPartitioned: true` and appends `composite-sub-partition` to `appliedStrategies`. `validateSplits` checks BOTH the size AND component-count cap per split (skipping `isFutureBuffer` solutions). Supports `.alm-config.json` overrides including `strategyOverride` to bypass the tree. See `solution-splitting-logic.md` spec in design docs for full logic. +- `scripts/lib/resolve-site-tables.js`: Single source of truth for "which custom tables does this site actually use." `collectReferencedEntityNames({ projectRoot, datamodelManifestPath })` reads `.powerpages-site/table-permissions/*.tablepermission.yml` (`entitylogicalname`, via `powerpages-config.js → loadTablePermissions`) + the datamodel manifest → `{ names:Set, available, sources }`. `scopeCustomTables(referencedNames, customUnmanagedTables)` intersects that set with the env's custom-unmanaged tables. SME-confirmed: table permissions are the complete signal ("if a table is used in the site there will be permissions for it"), so forms/lists are NOT scanned. Used by `estimate-solution-size.js` and `discover-site-components.js` to replace the publisher-prefix table dump. +- `scripts/lib/query-metadata.js`: `queryCustomUnmanagedTables(envUrl, token, makeRequest?)` → `[{ logicalName, metadataId, schemaName, displayName }]` (the single `EntityDefinitions?$filter=IsCustomEntity` query, `IsManaged===false` filtered). Consolidates the formerly-triplicated custom-table query (estimator, discover-site-components, setup-solution). Reuses `odataGetAll` from `validation-helpers.js`. +- `scripts/lib/query-table-relationships.js`: `fetchTableRelationships(envUrl, table, token, makeRequest?)` → `{ oneToMany[], manyToMany[] }`. Extracted from `skills/audit-permissions/scripts/query-table-relationships.js` (now a thin CLI wrapper over this lib) and extended with ManyToMany. OneToMany errors propagate; ManyToMany is best-effort. Used by the estimator to build `tableRelationships[]` and by audit-permissions for relationship-scope validation. +- `scripts/lib/validation-helpers.js` also exports `odataGet(url, token, makeRequest?)` + `odataGetAll(url, token, makeRequest?, maxPages?)` — the shared, injectable OData GET + `@odata.nextLink` pagination used by the new metadata/relationship helpers (avoids each lib rolling its own paginator). #### Solution Management diff --git a/plugins/power-pages/scripts/lib/alm-thresholds.js b/plugins/power-pages/scripts/lib/alm-thresholds.js index 3b57f030c..4df94e50b 100644 --- a/plugins/power-pages/scripts/lib/alm-thresholds.js +++ b/plugins/power-pages/scripts/lib/alm-thresholds.js @@ -21,6 +21,12 @@ const DEFAULTS = Object.freeze({ hardFlagComponentCount: 10000, maxSchemaAttrs: 15000, maxTableCount: 20, + // Safety ceiling on the number of auto-derived schema-split solutions. The + // schema-segmentation packing keeps each solution under maxTableCount / + // maxSchemaAttrs, but caps the COUNT here so a pathological schema can't + // explode into dozens of solutions — beyond this, the hardFlagComponentCount + // recommendation tells the user to archive/consolidate instead. + maxSchemaSplitSolutions: 8, maxAggregateWebFilesMB: 40, maxSingleFileMB: 2, maxEnvVarCount: 500, diff --git a/plugins/power-pages/scripts/lib/compute-split-plan.js b/plugins/power-pages/scripts/lib/compute-split-plan.js index e4291b371..f59028400 100644 --- a/plugins/power-pages/scripts/lib/compute-split-plan.js +++ b/plugins/power-pages/scripts/lib/compute-split-plan.js @@ -299,7 +299,7 @@ function partitionByChangeFrequency(estimate, meta) { function partitionBySchema(estimate, meta, config) { const explicitDomains = Array.isArray(config.domains) && config.domains.length > 0 ? config.domains - : deriveDomainsFromPrefix(estimate); + : deriveDomainsByCapacity(estimate, config.thresholds); // Derive domain vs site size shares from the estimator's breakdown when available, // falling back to a 50/50 heuristic only if breakdown is absent. @@ -319,9 +319,12 @@ function partitionBySchema(estimate, meta, config) { componentTypes: ['Table'], description: `Schema domain: ${dom.name}. Tables: ${(dom.tableLogicalNames || []).join(', ') || '(derived)'}${domainDescSuffix}`, sizeMB: round(sizePerDomain), - componentCount: Math.ceil( - (estimate.schemaAttrCount || 0) / domainCount, - ), + // A Table domain's component count IS its table count when known (each table + // is one Entity solution component). Falls back to an even attr-share split + // only for explicit domains that didn't list their tables. + componentCount: (dom.tableLogicalNames && dom.tableLogicalNames.length > 0) + ? dom.tableLogicalNames.length + : Math.ceil((estimate.schemaAttrCount || 0) / domainCount), components: [], tableLogicalNames: dom.tableLogicalNames || [], })); @@ -345,23 +348,99 @@ function partitionBySchema(estimate, meta, config) { return [...domainSolutions, siteSolution]; } -function deriveDomainsFromPrefix(estimate) { - const tables = estimate.tables || []; - if (tables.length === 0) return [{ name: 'All', tableLogicalNames: [] }]; +// --- Dependency-aware schema packing --------------------------------------- +// +// Replaces the old "one solution per table-name stem" heuristic (which produced +// ~one solution per table for any distinctly-named schema). Tables connected by +// a relationship MUST ship together, so we: +// 1. Group tables into connected components (union-find over the estimator's +// `tableRelationships` edges). Because components have no edges between +// them, packing whole components into separate solutions never cuts a +// relationship — so there are no cross-/circular-solution table deps and +// import order among the table solutions is irrelevant. +// 2. Bin-pack the components into the FEWEST solutions that keep each under the +// per-solution caps (maxTableCount tables AND maxSchemaAttrs columns), +// capped at maxSchemaSplitSolutions. + +function normalizeTables(estimate) { + return (estimate.tables || []) + .map((t) => ({ + logicalName: (t && (t.logicalName || t)).toString(), + attributeCount: (t && t.attributeCount) || 0, + })) + .filter((t) => t.logicalName); +} +// Union-find over tables + relationship edges -> array of clusters (each a list +// of table objects). A table with no edges is its own singleton cluster. +function buildTableClusters(tables, edges) { + const idx = new Map(); + tables.forEach((t, i) => idx.set(t.logicalName.toLowerCase(), i)); + const parent = tables.map((_, i) => i); + const find = (x) => { while (parent[x] !== x) { parent[x] = parent[parent[x]]; x = parent[x]; } return x; }; + const union = (a, b) => { const ra = find(a), rb = find(b); if (ra !== rb) parent[ra] = rb; }; + for (const e of edges || []) { + if (!Array.isArray(e) || e.length < 2) continue; + const ia = idx.get(String(e[0]).toLowerCase()); + const ib = idx.get(String(e[1]).toLowerCase()); + if (ia != null && ib != null) union(ia, ib); + } const groups = new Map(); - for (const t of tables) { - const name = (t.logicalName || t).toString(); - const afterPrefix = name.includes('_') ? name.split('_').slice(1).join('_') : name; - const stem = afterPrefix.split(/[_]/)[0] || 'misc'; - const key = stem.charAt(0).toUpperCase() + stem.slice(1); - if (!groups.has(key)) groups.set(key, []); - groups.get(key).push(name); + tables.forEach((t, i) => { + const r = find(i); + if (!groups.has(r)) groups.set(r, []); + groups.get(r).push(t); + }); + return [...groups.values()]; +} + +function clusterAttrs(cluster) { + return cluster.reduce((s, t) => s + (t.attributeCount || 0), 0); +} + +// First-fit-decreasing pack of whole clusters into `n` buckets, respecting the +// per-solution table + attribute caps. A cluster that fits nowhere under the +// caps (oversized, or n too small) goes to the least-loaded bucket — that bucket +// then exceeds a cap and is surfaced by the oversized-cluster recommendation. +function packClusters(clusters, n, thresholds) { + const sorted = [...clusters].sort((a, b) => (clusterAttrs(b) - clusterAttrs(a)) || (b.length - a.length)); + const buckets = Array.from({ length: Math.max(n, 1) }, () => ({ tables: [], attrs: 0 })); + for (const cluster of sorted) { + const cAttrs = clusterAttrs(cluster); + let target = buckets.findIndex( + (b) => b.tables.length + cluster.length <= thresholds.maxTableCount && + b.attrs + cAttrs <= thresholds.maxSchemaAttrs, + ); + if (target === -1) { + target = buckets.reduce((best, b, i) => (b.attrs < buckets[best].attrs ? i : best), 0); + } + buckets[target].tables.push(...cluster); + buckets[target].attrs += cAttrs; } + return buckets.filter((b) => b.tables.length > 0); +} - return Array.from(groups.entries()).map(([name, tableLogicalNames]) => ({ - name, - tableLogicalNames, +// Returns capacity-bounded "domains" (one per packed bucket) in the same shape +// the schema partitioner consumes: { name, tableLogicalNames }. +function deriveDomainsByCapacity(estimate, thresholds) { + const tables = normalizeTables(estimate); + if (tables.length === 0) return [{ name: 'Tables', tableLogicalNames: [] }]; + + const clusters = buildTableClusters(tables, estimate.tableRelationships || []); + const totalAttrs = tables.reduce((s, t) => s + t.attributeCount, 0); + const ceiling = (thresholds && thresholds.maxSchemaSplitSolutions) || 8; + let n = Math.max( + 1, + Math.ceil(tables.length / thresholds.maxTableCount), + Math.ceil(totalAttrs / Math.max(thresholds.maxSchemaAttrs, 1)), + ); + n = Math.min(n, ceiling, clusters.length); + + const buckets = packClusters(clusters, n, thresholds); + const multi = buckets.length > 1; + return buckets.map((b, i) => ({ + name: multi ? `Tables ${i + 1}` : 'Tables', + tableLogicalNames: b.tables.map((t) => t.logicalName), })); } @@ -704,6 +783,17 @@ function computeSplitPlan({ estimate, config, meta }) { proposedSolutions = appendFutureBuffer(proposedSolutions, meta); const splitWarnings = validateSplits(proposedSolutions, config.thresholds); + // Oversized-cluster guard: a Table solution holding more tables than the + // per-solution cap means a single connected dependency cluster couldn't be + // split without cutting a relationship. Name it so the user can decide whether + // to denormalize the schema or raise the cap — we never silently split a cluster. + const oversizedClusterWarnings = proposedSolutions + .filter((s) => Array.isArray(s.tableLogicalNames) && + s.tableLogicalNames.length > config.thresholds.maxTableCount) + .map((s) => ({ + type: 'warning', + message: `Solution ${s.uniqueName} holds ${s.tableLogicalNames.length} related tables — above the ${config.thresholds.maxTableCount}-per-solution cap — because they form one dependency cluster that cannot be split without breaking a relationship. Consider denormalizing the schema or raising maxTableCount in .alm-config.json.`, + })); // Surface estimator-side truncation warnings as `recommendations[]` entries // so the rendered plan shows them inline. These get the `error` type because // a truncated input is more dangerous than a normal split-decision warning @@ -715,7 +805,8 @@ function computeSplitPlan({ estimate, config, meta }) { })); const recommendations = truncationRecs .concat(buildRecommendations(estimate, strategy, config)) - .concat(splitWarnings); + .concat(splitWarnings) + .concat(oversizedClusterWarnings); const appliedStrategies = [strategy.primary]; if (strategy.additive) appliedStrategies.push('strategy-4-config-isolation'); diff --git a/plugins/power-pages/scripts/lib/discover-site-components.js b/plugins/power-pages/scripts/lib/discover-site-components.js index fdd1cf421..c774e14b5 100644 --- a/plugins/power-pages/scripts/lib/discover-site-components.js +++ b/plugins/power-pages/scripts/lib/discover-site-components.js @@ -53,6 +53,8 @@ 'use strict'; const helpers = require('./validation-helpers'); +const { queryCustomUnmanagedTables } = require('./query-metadata'); +const { collectReferencedEntityNames, scopeCustomTables } = require('./resolve-site-tables'); /** Authoritative picklist labels for powerpagecomponenttype. */ const PPC_TYPE_LABELS = Object.freeze({ @@ -107,6 +109,8 @@ function parseArgs(argv) { siteId: null, publisherPrefix: null, solutionId: null, + projectRoot: null, + datamodelManifestPath: null, }; for (let i = 0; i < args.length; i++) { if (args[i] === '--envUrl' && args[i + 1]) out.envUrl = args[++i]; @@ -114,6 +118,8 @@ function parseArgs(argv) { else if (args[i] === '--siteId' && args[i + 1]) out.siteId = args[++i]; else if (args[i] === '--publisherPrefix' && args[i + 1]) out.publisherPrefix = args[++i]; else if (args[i] === '--solutionId' && args[i + 1]) out.solutionId = args[++i]; + else if (args[i] === '--projectRoot' && args[i + 1]) out.projectRoot = args[++i]; + else if (args[i] === '--datamodelManifest' && args[i + 1]) out.datamodelManifestPath = args[++i]; } return out; } @@ -166,6 +172,8 @@ async function discoverSiteComponents({ siteId, publisherPrefix = null, solutionId = null, + projectRoot = null, + datamodelManifestPath = null, makeRequest = helpers.makeRequest, } = {}) { if (!envUrl) throw new Error('--envUrl is required'); @@ -232,10 +240,13 @@ async function discoverSiteComponents({ ? await discoverEnvVars({ baseUrl, token, publisherPrefix, makeRequest }) : []; - // 5) Custom tables filtered by publisher prefix (optional) - const customTables = publisherPrefix - ? await discoverCustomTables({ baseUrl, token, publisherPrefix, makeRequest }) - : []; + // 5) Custom tables the SITE references (table permissions + datamodel manifest), + // intersected with the env's custom-unmanaged tables. NOT a publisher-prefix + // dump — that over-counted unrelated tables sharing the prefix (the + // new_/default-publisher bug). Empty when no local signal is available. + const customTables = await discoverCustomTables({ + baseUrl, token, projectRoot, datamodelManifestPath, makeRequest, + }); const result = { siteId, @@ -382,28 +393,27 @@ async function discoverEnvVars({ baseUrl, token, publisherPrefix, makeRequest }) })); } -async function discoverCustomTables({ baseUrl, token, publisherPrefix, makeRequest }) { - // The $metadata/EntityDefinitions endpoint doesn't support `startswith` (0x8006088a), - // so we fetch all custom tables and filter client-side. Custom-entity sets are small - // enough that a single request is fine. MetadataId is included so callers can diff - // against solutioncomponents.objectid (componenttype 1 = Entity). - // publisherPrefix validated at the entry point of discoverSiteComponents. - const prefixLower = String(publisherPrefix).trim().toLowerCase(); - const url = - `${baseUrl}/api/data/v9.2/EntityDefinitions` + - `?$filter=IsCustomEntity eq true` + - `&$select=LogicalName,SchemaName,DisplayName,MetadataId`; - const rows = await odataGetAll(url, token, makeRequest); - return rows - .filter((r) => (r.LogicalName || '').toLowerCase().startsWith(`${prefixLower}_`)) - .map((r) => ({ - id: r.MetadataId, - logicalName: r.LogicalName, - schemaName: r.SchemaName, - displayName: - (r.DisplayName && r.DisplayName.UserLocalizedLabel && r.DisplayName.UserLocalizedLabel.Label) || - r.SchemaName, - })); +async function discoverCustomTables({ baseUrl, token, projectRoot, datamodelManifestPath, makeRequest }) { + // Scope to the tables the SITE references (its table permissions + datamodel + // manifest — SME-confirmed complete), intersected with the env's custom-unmanaged + // tables. Replaces the old publisher-prefix dump that returned every table + // sharing the prefix (catastrophic with `new_`/default publishers). MetadataId + // is returned as `id` so callers can diff against solutioncomponents.objectid + // (componenttype 1 = Entity). + const { names, available } = collectReferencedEntityNames({ projectRoot, datamodelManifestPath }); + if (!available) return []; // no local signal — empty, NEVER a prefix dump + let customUnmanaged = []; + try { + customUnmanaged = await queryCustomUnmanagedTables(baseUrl, token, makeRequest); + } catch { + customUnmanaged = []; + } + return scopeCustomTables(names, customUnmanaged).map((t) => ({ + id: t.metadataId, + logicalName: t.logicalName, + schemaName: t.schemaName, + displayName: t.displayName, + })); } if (require.main === module) { diff --git a/plugins/power-pages/scripts/lib/estimate-solution-size.js b/plugins/power-pages/scripts/lib/estimate-solution-size.js index f38a6e844..911146e07 100644 --- a/plugins/power-pages/scripts/lib/estimate-solution-size.js +++ b/plugins/power-pages/scripts/lib/estimate-solution-size.js @@ -27,6 +27,9 @@ const helpers = require('./validation-helpers'); const { getAuthToken } = helpers; +const { queryCustomUnmanagedTables } = require('./query-metadata'); +const { fetchTableRelationships } = require('./query-table-relationships'); +const { collectReferencedEntityNames, scopeCustomTables } = require('./resolve-site-tables'); // `makeRequest` is accessed via `helpers.makeRequest` (not destructured) so // tests can inject a mock by mutating `helpers.makeRequest` before calling // the top-level `estimateSolutionSize`. See estimate-solution-size.test.js for @@ -285,44 +288,78 @@ async function discoverPowerPageSiteLanguages(envUrl, websiteRecordId, token) { } } -async function discoverTables(envUrl, publisherPrefix, token, manifestPath) { - // Try manifest first - const fs = require('fs'); - let manifestTables = []; - if (manifestPath && fs.existsSync(manifestPath)) { - try { - const man = JSON.parse(fs.readFileSync(manifestPath, 'utf8')); - const entries = man.entities || man.tables || []; - manifestTables = entries.map((e) => ({ - logicalName: e.logicalName || e.LogicalName || e.name, - metadataId: e.metadataId || e.MetadataId, - })); - } catch {} +// Discovers the custom tables the SITE actually references — NOT every table +// sharing the publisher prefix. The old prefix-wide enumeration over-counted +// catastrophically with a shared/default publisher (`new_`, env default): a +// 6-table site reported 22 tables, which cascaded into absurd schema splits. +// +// Source of truth = the site's table permissions (+ datamodel manifest), per +// SME: "If a table is used in the site there will be permissions for it." +// We intersect those referenced names with the env's custom-unmanaged tables so +// standard tables (contact/annotation) and managed template tables drop out. +// +// Returns `{ tables, tableCountScope }` where scope ∈ +// "site-referenced" | "manifest-only" | "unavailable". +// On no local signal we return zero tables (NEVER a prefix dump) so a missing +// `.powerpages-site/` degrades safe instead of inflating the plan. +async function discoverTables(envUrl, token, { projectRoot, datamodelManifestPath } = {}) { + let customUnmanaged = []; + try { + customUnmanaged = await queryCustomUnmanagedTables(envUrl, token); + } catch { + customUnmanaged = []; } - // Query EntityDefinitions for custom unmanaged tables. - // Verified 2026-04-22 against org1e98cc97 (v9.2): EntityDefinitions does NOT - // support `$top` (returns 400 "The query parameter $top is not supported"). - // We filter server-side to IsCustomEntity=true to keep the payload bounded — - // there's still no client-side pagination needed for typical tenants. - const path = - `EntityDefinitions` + - `?$filter=IsCustomEntity eq true` + - `&$select=LogicalName,MetadataId,IsManaged,IsCustomEntity`; - const all = await collectPaginated(envUrl, path, token, 10); - const custom = all.filter((e) => e.IsCustomEntity === true && e.IsManaged === false); - const matchingPrefix = publisherPrefix - ? custom.filter((e) => (e.LogicalName || '').toLowerCase().startsWith(`${publisherPrefix.toLowerCase()}_`)) - : custom; + const { names, available, sources } = collectReferencedEntityNames({ projectRoot, datamodelManifestPath }); + + let scope; + let scoped = []; + if (!available) { + scope = 'unavailable'; + } else { + scoped = scopeCustomTables(names, customUnmanaged); + scope = sources.tablePermissions > 0 ? 'site-referenced' : 'manifest-only'; + } const byName = new Map(); - for (const t of [...manifestTables, ...matchingPrefix.map((e) => ({ - logicalName: e.LogicalName, - metadataId: e.MetadataId, - }))]) { - if (t.logicalName && !byName.has(t.logicalName)) byName.set(t.logicalName, t); + for (const t of scoped) { + if (t.logicalName && !byName.has(t.logicalName)) { + byName.set(t.logicalName, { logicalName: t.logicalName, metadataId: t.metadataId }); + } } - return Array.from(byName.values()); + return { tables: Array.from(byName.values()), tableCountScope: scope }; +} + +// Build the deduped, scoped dependency-edge list among the site's tables. +// Each edge `[a, b]` (lowercased logical names, a (t.logicalName || '').toLowerCase()).filter(Boolean)); + const seen = new Set(); + const edges = []; + const addEdge = (x, y) => { + const a = String(x || '').toLowerCase(); + const b = String(y || '').toLowerCase(); + if (!a || !b || a === b || !inSet.has(a) || !inSet.has(b)) return; + const key = a < b ? `${a}|${b}` : `${b}|${a}`; + if (seen.has(key)) return; + seen.add(key); + edges.push(a < b ? [a, b] : [b, a]); + }; + for (const t of tables) { + let rel; + try { + rel = await fetchTableRelationships(envUrl, t.logicalName, token); + } catch { + continue; // inaccessible table — skip its edges + } + for (const e of rel.oneToMany) addEdge(e.referencedEntity, e.referencingEntity); + for (const e of rel.manyToMany) addEdge(e.entity1, e.entity2); + } + return edges; } async function countAttributesForTables(envUrl, tables, token) { @@ -712,8 +749,14 @@ async function estimateSolutionSize({ envUrl, websiteRecordId, token, publisherP // (which includes them under componenttype 10428). const siteLanguages = await discoverPowerPageSiteLanguages(envUrl, websiteRecordId, resolved); - const tables = await discoverTables(envUrl, publisherPrefix, resolved, datamodelManifest); + const { tables, tableCountScope } = await discoverTables(envUrl, resolved, { + projectRoot, + datamodelManifestPath: datamodelManifest, + }); const schemaAttrCount = await countAttributesForTables(envUrl, tables, resolved); + // Dependency edges among the scoped tables — drives the schema-split clustering + // so related tables ship in the same solution (never split a relationship). + const tableRelationships = await discoverTableRelationships(envUrl, tables, resolved); // Tenant-wide env var defs matching the publisher prefix. This is the // fallback used when no solution is set up yet (fresh project); for sites @@ -1024,6 +1067,10 @@ async function estimateSolutionSize({ envUrl, websiteRecordId, token, publisherP } : null, tableCount: tables.length, + // How the table set was scoped: "site-referenced" (table permissions), + // "manifest-only" (datamodel manifest, no permissions), or "unavailable" + // (no local signal — tableCount reflects 0, NOT a publisher-prefix dump). + tableCountScope, schemaAttrCount, webFilesAggregateMB: round1(webFilesAggregateBytes / (1024 * 1024)), webFilesIndividual: webMeasure.individual, @@ -1059,6 +1106,9 @@ async function estimateSolutionSize({ envUrl, websiteRecordId, token, publisherP mediaRatio: Math.round(webMeasure.mediaRatio * 100) / 100, siteType: 'code-site', tables: tables.map((t) => ({ logicalName: t.logicalName, attributeCount: t.attributeCount || 0 })), + // Dependency edges among the scoped tables ([a,b], lowercased, a} + */ +async function queryCustomUnmanagedTables(envUrl, token, request = helpers.makeRequest) { + const base = String(envUrl).replace(/\/+$/, ''); + const url = + `${base}/api/data/v9.2/EntityDefinitions` + + `?$filter=IsCustomEntity eq true` + + `&$select=LogicalName,MetadataId,SchemaName,DisplayName,IsManaged,IsCustomEntity`; + const rows = await helpers.odataGetAll(url, token, request); + return rows + .filter((e) => e && e.IsCustomEntity === true && e.IsManaged === false) + .map((e) => ({ + logicalName: e.LogicalName, + metadataId: e.MetadataId, + schemaName: e.SchemaName, + displayName: + (e.DisplayName && e.DisplayName.UserLocalizedLabel && e.DisplayName.UserLocalizedLabel.Label) || + e.SchemaName, + })); +} + +module.exports = { queryCustomUnmanagedTables }; diff --git a/plugins/power-pages/scripts/lib/query-table-relationships.js b/plugins/power-pages/scripts/lib/query-table-relationships.js new file mode 100644 index 000000000..11850f171 --- /dev/null +++ b/plugins/power-pages/scripts/lib/query-table-relationships.js @@ -0,0 +1,65 @@ +#!/usr/bin/env node + +// Shared Dataverse relationship queries for a table: lookups (OneToMany) + N:N +// (ManyToMany). Extracted from skills/audit-permissions/scripts/query-table-relationships.js +// so it can be require()d (that file is a self-executing CLI). The CLI is now a +// thin wrapper over this module. Used by: +// - estimate-solution-size.js — to build the dependency graph for the +// schema-split clustering (so related tables ship in the same solution). +// - audit-permissions — to validate contact/account/parent relationship scopes. + +'use strict'; + +const helpers = require('./validation-helpers'); + +/** + * Fetches OneToMany (lookup-backed) and ManyToMany relationships for a table. + * + * OneToMany errors propagate (a genuinely missing/inaccessible table is a real + * error the CLI surfaces via exit 1; the estimator wraps the call per-table for + * resilience). ManyToMany is best-effort — many tables/envs have no N:N and the + * navigation property can be finicky — so its errors are swallowed to `[]`. + * + * @param {string} envUrl - environment base URL + * @param {string} table - table logical name + * @param {string} token - bearer token + * @param {Function} [request=helpers.makeRequest] - injectable for tests + * @returns {Promise<{ + * oneToMany: { schemaName, referencedEntity, referencingEntity, referencingAttribute }[], + * manyToMany: { schemaName, entity1, entity2 }[] + * }>} + */ +async function fetchTableRelationships(envUrl, table, token, request = helpers.makeRequest) { + const base = String(envUrl).replace(/\/+$/, ''); + const safe = String(table).replace(/'/g, "''"); + + const o2mUrl = + `${base}/api/data/v9.2/EntityDefinitions(LogicalName='${safe}')/OneToManyRelationships` + + `?$select=SchemaName,ReferencedEntity,ReferencingEntity,ReferencingAttribute`; + const o2mRows = await helpers.odataGetAll(o2mUrl, token, request); + const oneToMany = o2mRows.map((r) => ({ + schemaName: r.SchemaName, + referencedEntity: r.ReferencedEntity, + referencingEntity: r.ReferencingEntity, + referencingAttribute: r.ReferencingAttribute, + })); + + let manyToMany = []; + try { + const m2mUrl = + `${base}/api/data/v9.2/EntityDefinitions(LogicalName='${safe}')/ManyToManyRelationships` + + `?$select=SchemaName,Entity1LogicalName,Entity2LogicalName`; + const m2mRows = await helpers.odataGetAll(m2mUrl, token, request); + manyToMany = m2mRows.map((r) => ({ + schemaName: r.SchemaName, + entity1: r.Entity1LogicalName, + entity2: r.Entity2LogicalName, + })); + } catch { + // N:N unavailable for this table/env — best-effort, leave empty. + } + + return { oneToMany, manyToMany }; +} + +module.exports = { fetchTableRelationships }; diff --git a/plugins/power-pages/scripts/lib/resolve-site-tables.js b/plugins/power-pages/scripts/lib/resolve-site-tables.js new file mode 100644 index 000000000..043f35322 --- /dev/null +++ b/plugins/power-pages/scripts/lib/resolve-site-tables.js @@ -0,0 +1,103 @@ +#!/usr/bin/env node + +// Resolves the custom Dataverse tables a Power Pages site ACTUALLY references, +// so ALM table discovery stops scooping up every table that merely shares the +// publisher prefix (the `new_` / default-publisher over-count bug). +// +// SME-confirmed source of truth: "We rely on Table permissions from the site. +// If a table is used in the site that means there will be permissions for it." +// So the site's table permissions (+ the datamodel manifest) are the complete +// list of tables the site uses — no need to also scan forms/lists. +// +// Two steps, kept separate so the Dataverse query (custom-unmanaged tables) can +// be supplied by the caller (estimate-solution-size.js / discover-site-components.js): +// 1. collectReferencedEntityNames({ projectRoot, datamodelManifestPath }) +// -> the set of entity logical names the site references (local read). +// 2. scopeCustomTables(referencedNames, customUnmanagedTables) +// -> the caller's custom-unmanaged table list, intersected with that set. +// +// Intersecting with custom-UNMANAGED tables drops standard tables (contact, +// annotation) and managed template tables — leaving exactly the tables the +// user's solution would own. + +'use strict'; + +const fs = require('fs'); +const path = require('path'); +const { loadTablePermissions } = require('./powerpages-config'); + +/** + * Collects the entity logical names referenced by the site's table permissions + * and its datamodel manifest (both local reads; no Dataverse). + * + * @param {object} opts + * @param {string} [opts.projectRoot] - site project root (contains .powerpages-site/) + * @param {string} [opts.datamodelManifestPath] - explicit manifest path (defaults to + * `/.datamodel-manifest.json`) + * @returns {{ names: Set, available: boolean, sources: { tablePermissions: number, manifest: number } }} + * `names` are lowercased. `available` is false only when neither a + * `.powerpages-site/table-permissions/` directory nor a manifest was found. + */ +function collectReferencedEntityNames({ projectRoot, datamodelManifestPath } = {}) { + const names = new Set(); + const sources = { tablePermissions: 0, manifest: 0 }; + let sawTablePermissionsDir = false; + let sawManifest = false; + + // 1. Table permissions — `entitylogicalname` per `*.tablepermission.yml`. + if (projectRoot) { + const dir = path.join(projectRoot, '.powerpages-site', 'table-permissions'); + if (fs.existsSync(dir)) { + sawTablePermissionsDir = true; + let records = []; + try { records = loadTablePermissions(dir); } catch { records = []; } + for (const r of records) { + const name = r && r.entitylogicalname; // NOT entityname (that's the display label) + if (typeof name === 'string' && name.trim()) { + names.add(name.trim().toLowerCase()); + sources.tablePermissions += 1; + } + } + } + } + + // 2. Datamodel manifest — tables created for this site by setup-datamodel. + const manifestPath = datamodelManifestPath || + (projectRoot ? path.join(projectRoot, '.datamodel-manifest.json') : null); + if (manifestPath && fs.existsSync(manifestPath)) { + sawManifest = true; + try { + const man = JSON.parse(fs.readFileSync(manifestPath, 'utf8')); + const entries = man.entities || man.tables || []; + for (const e of entries) { + const name = e && (e.logicalName || e.LogicalName || e.name); + if (typeof name === 'string' && name.trim()) { + names.add(name.trim().toLowerCase()); + sources.manifest += 1; + } + } + } catch { + // Malformed manifest — ignore its contents but still count it as a signal. + } + } + + return { names, available: sawTablePermissionsDir || sawManifest, sources }; +} + +/** + * Intersects the caller's custom-unmanaged table list with the referenced-name + * set. Returns the tables the site actually uses (and that the user's solution + * would own). + * + * @param {Set} referencedNames - lowercased logical names (from collectReferencedEntityNames) + * @param {{ logicalName: string }[]} customUnmanagedTables + * @returns {{ logicalName: string }[]} + */ +function scopeCustomTables(referencedNames, customUnmanagedTables) { + if (!referencedNames || referencedNames.size === 0) return []; + return (customUnmanagedTables || []).filter( + (t) => t && typeof t.logicalName === 'string' && referencedNames.has(t.logicalName.toLowerCase()), + ); +} + +module.exports = { collectReferencedEntityNames, scopeCustomTables }; diff --git a/plugins/power-pages/scripts/lib/validation-helpers.js b/plugins/power-pages/scripts/lib/validation-helpers.js index 5667795c5..da8232707 100644 --- a/plugins/power-pages/scripts/lib/validation-helpers.js +++ b/plugins/power-pages/scripts/lib/validation-helpers.js @@ -235,6 +235,53 @@ function makeRequest({ url, method = 'GET', headers = {}, body = null, includeHe }); } +/** + * Single Dataverse OData GET (v9.2 headers, `Prefer: odata.maxpagesize=5000`), + * throws on non-2xx. `url` is absolute — pass an `@odata.nextLink` straight back in. + * @param {string} url - absolute URL + * @param {string} token - bearer token + * @param {Function} [request=makeRequest] - injectable for tests + * @returns {Promise} parsed JSON body + */ +async function odataGet(url, token, request = makeRequest) { + const res = await request({ + url, + headers: { + Authorization: `Bearer ${token}`, + Accept: 'application/json', + 'OData-MaxVersion': '4.0', + 'OData-Version': '4.0', + Prefer: 'odata.maxpagesize=5000', + }, + timeout: 30000, + }); + if (res.error) throw new Error(`OData request failed: ${res.error}`); + if (res.statusCode < 200 || res.statusCode >= 300) { + throw new Error(`HTTP ${res.statusCode} from ${url}: ${(res.body || '').slice(0, 400)}`); + } + return JSON.parse(res.body); +} + +/** + * Follows `@odata.nextLink`, aggregating every page's `value[]` into one array. + * `maxPages` is a runaway-loop safety cap (100 × 5000 ≈ 500K rows). + * @param {string} url - absolute starting URL + * @param {string} token - bearer token + * @param {Function} [request=makeRequest] - injectable for tests + * @param {number} [maxPages=100] + * @returns {Promise} + */ +async function odataGetAll(url, token, request = makeRequest, maxPages = 100) { + const out = []; + let next = url; + for (let p = 0; p < maxPages && next; p++) { + const page = await odataGet(next, token, request); + if (Array.isArray(page.value)) out.push(...page.value); + next = page['@odata.nextLink'] || null; + } + return out; +} + /** Cloud → Power Platform API base URL mapping */ const CLOUD_TO_API = { 'Public': 'https://api.powerplatform.com', @@ -264,6 +311,8 @@ module.exports = { UUID_REGEX, getAuthToken, makeRequest, + odataGet, + odataGetAll, getEnvironmentUrl, getPacAuthInfo, CLOUD_TO_API, diff --git a/plugins/power-pages/scripts/tests/compute-split-plan.test.js b/plugins/power-pages/scripts/tests/compute-split-plan.test.js index 8fda1569b..48c92c03a 100644 --- a/plugins/power-pages/scripts/tests/compute-split-plan.test.js +++ b/plugins/power-pages/scripts/tests/compute-split-plan.test.js @@ -183,23 +183,26 @@ test('computeSplitPlan Strategy 3 uses explicit config.domains when present', () assert.equal(result.proposedSolutions[3].isFutureBuffer, true); }); -test('computeSplitPlan Strategy 3 falls back to prefix heuristic when no domains configured', () => { +test('computeSplitPlan Strategy 3 packs tables by capacity (no domains configured) — bounded, not per-table', () => { const result = computeSplitPlan({ estimate: baseEstimate({ tableCount: 22, - schemaAttrCount: 16000, + schemaAttrCount: 16000, // > maxSchemaAttrs(15000) -> 2 buckets by attrs tables: [ - { logicalName: 'tst_product' }, - { logicalName: 'tst_productVariant' }, - { logicalName: 'tst_order' }, - { logicalName: 'tst_orderLine' }, + { logicalName: 'tst_product', attributeCount: 4000 }, + { logicalName: 'tst_productVariant', attributeCount: 4000 }, + { logicalName: 'tst_order', attributeCount: 4000 }, + { logicalName: 'tst_orderLine', attributeCount: 4000 }, ], }), config: baseConfig(), meta: { baseName: 'Test', siteName: 'Test Site' }, }); assert.equal(result.splitStrategy, 'strategy-3-schema-segmentation'); - assert.ok(result.proposedSolutions.length >= 2); + const tableSolutions = result.proposedSolutions.filter( + (s) => Array.isArray(s.componentTypes) && s.componentTypes.length === 1 && s.componentTypes[0] === 'Table', + ); + assert.equal(tableSolutions.length, 2, '16000 attrs / 15000 cap -> 2 Table solutions, not one-per-table'); }); test('computeSplitPlan additive Strategy 4 prepends EnvVars solution', () => { @@ -601,3 +604,87 @@ test('partitionBySchema uses breakdown.tables to size domain solutions', () => { // Site solution absorbs the remainder assert.equal(solutions[2].sizeMB, 60); }); + +// --- dependency-aware capacity packing (the "21 solutions" fix) -------------- + +function makeTables(n, attrsEach, prefix = 'tbl') { + return Array.from({ length: n }, (_, i) => ({ logicalName: `${prefix}_${i}`, attributeCount: attrsEach })); +} + +test('Strategy 3: 34 distinct tables / 32.3k cols -> a HANDFUL of solutions, never ~34', () => { + const tables = makeTables(34, 950); // 34 * 950 = 32300 + const result = computeSplitPlan({ + estimate: baseEstimate({ totalSizeMB: 68, tableCount: 34, schemaAttrCount: 32300, componentCountSiteTotal: 3000, tables, tableRelationships: [] }), + config: baseConfig(), + meta: { baseName: 'Big', siteName: 'Big Site' }, + }); + assert.equal(result.splitStrategy, 'strategy-3-schema-segmentation'); + const tableSolutions = result.proposedSolutions.filter((s) => s.componentTypes && s.componentTypes[0] === 'Table' && s.componentTypes.length === 1); + assert.ok(tableSolutions.length >= 2 && tableSolutions.length <= 8, `expected a handful of Table solutions, got ${tableSolutions.length}`); + // Every Table solution stays under the per-solution table cap. + for (const s of tableSolutions) { + assert.ok(s.tableLogicalNames.length <= 20, `Table solution ${s.uniqueName} has ${s.tableLogicalNames.length} tables (> cap)`); + } + // All 34 tables are placed exactly once across the Table solutions. + const placed = tableSolutions.flatMap((s) => s.tableLogicalNames); + assert.equal(placed.length, 34); + assert.equal(new Set(placed).size, 34); +}); + +test('Strategy 3: 22 tables with low cols -> 2 Table solutions (count-driven), not 22', () => { + const tables = makeTables(22, 50); + const result = computeSplitPlan({ + estimate: baseEstimate({ tableCount: 22, schemaAttrCount: 1100, tables, tableRelationships: [] }), + config: baseConfig(), + meta: { baseName: 'M', siteName: 'M' }, + }); + assert.equal(result.splitStrategy, 'strategy-3-schema-segmentation'); + const tableSolutions = result.proposedSolutions.filter((s) => s.componentTypes && s.componentTypes[0] === 'Table' && s.componentTypes.length === 1); + assert.equal(tableSolutions.length, 2, '22 tables / 20-per-solution -> 2 Table solutions'); +}); + +test('Strategy 3 does NOT trigger for <=20 tables with low cols -> single', () => { + const tables = makeTables(18, 50); + const { primary } = selectStrategy(baseEstimate({ tableCount: 18, schemaAttrCount: 900, tables }), baseConfig()); + assert.equal(primary, 'single'); +}); + +test('Strategy 3: dependency clusters are never split across solutions', () => { + // Cluster A (4 tables) + Cluster B (2 tables) + 20 standalone = 26 tables, low cols. + const clusterA = ['rel_a0', 'rel_a1', 'rel_a2', 'rel_a3']; + const clusterB = ['rel_b0', 'rel_b1']; + const standalone = makeTables(20, 50, 'solo').map((t) => t.logicalName); + const tables = [...clusterA, ...clusterB, ...standalone].map((n) => ({ logicalName: n, attributeCount: 50 })); + const edges = [ + ['rel_a0', 'rel_a1'], ['rel_a1', 'rel_a2'], ['rel_a2', 'rel_a3'], // A connected + ['rel_b0', 'rel_b1'], // B connected + ]; + const result = computeSplitPlan({ + estimate: baseEstimate({ tableCount: 26, schemaAttrCount: 1300, tables, tableRelationships: edges }), + config: baseConfig(), + meta: { baseName: 'Dep', siteName: 'Dep' }, + }); + assert.equal(result.splitStrategy, 'strategy-3-schema-segmentation'); + const tableSolutions = result.proposedSolutions.filter((s) => s.componentTypes && s.componentTypes[0] === 'Table' && s.componentTypes.length === 1); + const home = (name) => tableSolutions.findIndex((s) => s.tableLogicalNames.includes(name)); + // Every table in cluster A shares one solution; same for B. + assert.ok(home('rel_a0') !== -1); + assert.ok(clusterA.every((n) => home(n) === home('rel_a0')), 'cluster A must not be split across solutions'); + assert.ok(clusterB.every((n) => home(n) === home('rel_b0')), 'cluster B must not be split across solutions'); +}); + +test('Strategy 3: an oversized single cluster (>cap) stays whole + raises a warning', () => { + // 25 tables all chained into ONE connected cluster -> cannot be split. + const names = makeTables(25, 50, 'big').map((t) => t.logicalName); + const tables = names.map((n) => ({ logicalName: n, attributeCount: 50 })); + const edges = names.slice(1).map((n, i) => [names[i], n]); // chain a0-a1-a2-...-a24 + const result = computeSplitPlan({ + estimate: baseEstimate({ tableCount: 25, schemaAttrCount: 1250, tables, tableRelationships: edges }), + config: baseConfig(), + meta: { baseName: 'Mega', siteName: 'Mega' }, + }); + const tableSolutions = result.proposedSolutions.filter((s) => s.componentTypes && s.componentTypes[0] === 'Table' && s.componentTypes.length === 1); + assert.equal(tableSolutions.length, 1, 'one indivisible cluster -> one Table solution'); + assert.equal(tableSolutions[0].tableLogicalNames.length, 25); + assert.ok(result.recommendations.some((r) => /dependency cluster that cannot be split/.test(r.message)), 'oversized-cluster warning must fire'); +}); diff --git a/plugins/power-pages/scripts/tests/discover-site-components.test.js b/plugins/power-pages/scripts/tests/discover-site-components.test.js index 320c89289..47ea7328e 100644 --- a/plugins/power-pages/scripts/tests/discover-site-components.test.js +++ b/plugins/power-pages/scripts/tests/discover-site-components.test.js @@ -2,6 +2,9 @@ const test = require('node:test'); const assert = require('node:assert/strict'); +const fs = require('fs'); +const os = require('os'); +const path = require('path'); const { discoverSiteComponents, @@ -9,6 +12,21 @@ const { PPC_DEFAULT_INCLUDE, } = require('../lib/discover-site-components'); +// Creates a temp site root whose `.powerpages-site/table-permissions/` references +// the given entity logical names — the SME-confirmed signal for "tables the site +// uses" that now scopes custom-table discovery (replacing the publisher-prefix dump). +function tempSiteWithTablePermissions(t, entityLogicalNames) { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'dsc-site-')); + t.after(() => fs.rmSync(root, { recursive: true, force: true })); + const dir = path.join(root, '.powerpages-site', 'table-permissions'); + fs.mkdirSync(dir, { recursive: true }); + entityLogicalNames.forEach((entity, i) => { + fs.writeFileSync(path.join(dir, `perm-${i}.tablepermission.yml`), + `adx_entitypermission_webrole:\n- ad89f5ee-8665-f111-a826-6045bd00fdda\nentitylogicalname: ${entity}\nentityname: Perm ${i}\nid: f03fefed-8665-f111-a826-000d3a597e6a\nscope: 756150000\n`); + }); + return root; +} + /** * Creates a fake `makeRequest` that matches URL fragments to response bodies. * Each entry is `[urlFragment, responseObject]`; the first matching entry wins. @@ -169,7 +187,7 @@ test('computes missing[] diff against an existing solution', async () => { ); }); -test('diffs custom tables by MetadataId against solutioncomponents.objectid', async () => { +test('diffs custom tables by MetadataId against solutioncomponents.objectid', async (t) => { const makeRequest = fakeRequest([ ['/powerpagecomponents?', { value: [] }], ['/workflows?', { value: [] }], @@ -183,12 +201,16 @@ test('diffs custom tables by MetadataId against solutioncomponents.objectid', as LogicalName: 'crd50_already', SchemaName: 'crd50_Already', DisplayName: { UserLocalizedLabel: { Label: 'Already' } }, + IsCustomEntity: true, + IsManaged: false, }, { MetadataId: 'meta-missing', LogicalName: 'crd50_missing', SchemaName: 'crd50_Missing', DisplayName: { UserLocalizedLabel: { Label: 'Missing' } }, + IsCustomEntity: true, + IsManaged: false, }, ], }, @@ -199,12 +221,14 @@ test('diffs custom tables by MetadataId against solutioncomponents.objectid', as ], ]); + const projectRoot = tempSiteWithTablePermissions(t, ['crd50_already', 'crd50_missing']); const result = await discoverSiteComponents({ envUrl: 'https://example.crm.dynamics.com', token: 'tok', siteId: 'site-guid', publisherPrefix: 'crd50', solutionId: 'sol-guid', + projectRoot, makeRequest, }); @@ -258,7 +282,7 @@ test('matching is case-insensitive on solution object IDs', async () => { assert.equal(result.missing.powerpagecomponents.length, 0); }); -test('discovers env vars and custom tables when publisherPrefix is passed', async () => { +test('discovers env vars and scopes custom tables to site references (not publisher prefix)', async (t) => { const makeRequest = fakeRequest([ ['/powerpagecomponents?', { value: [] }], ['/workflows?', { value: [] }], @@ -288,24 +312,34 @@ test('discovers env vars and custom tables when publisherPrefix is passed', asyn LogicalName: 'crd50_invoice', SchemaName: 'crd50_Invoice', DisplayName: { UserLocalizedLabel: { Label: 'Invoice' } }, + IsCustomEntity: true, + IsManaged: false, }, { MetadataId: 'meta-other-widget', - // A custom table from a different publisher — must be filtered out client-side. + // A custom table the site does NOT reference — dropped by the site-reference + // scoping (not by prefix). Even shares no prefix concern: it's simply unused. LogicalName: 'other_widget', SchemaName: 'other_Widget', DisplayName: { UserLocalizedLabel: { Label: 'Widget' } }, + IsCustomEntity: true, + IsManaged: false, }, ], }, ], ]); + // Site references ONLY crd50_invoice (not other_widget). The scoping must keep + // crd50_invoice and drop other_widget — because it isn't referenced, NOT because + // of its prefix (the whole point of the fix). + const projectRoot = tempSiteWithTablePermissions(t, ['crd50_invoice']); const result = await discoverSiteComponents({ envUrl: 'https://example.crm.dynamics.com', token: 'tok', siteId: 'site-guid', publisherPrefix: 'crd50', + projectRoot, makeRequest, }); diff --git a/plugins/power-pages/scripts/tests/estimate-solution-size.test.js b/plugins/power-pages/scripts/tests/estimate-solution-size.test.js index 8432c5240..78055b750 100644 --- a/plugins/power-pages/scripts/tests/estimate-solution-size.test.js +++ b/plugins/power-pages/scripts/tests/estimate-solution-size.test.js @@ -705,3 +705,87 @@ test('disk-measurement gracefully no-ops when projectRoot has no build-output di 'disk-vs-dataverse canary must not fire when no build dir found', ); }); + +// --- site-referenced table scoping + dependency edges (the prefix-overcount fix) --- + +test('estimateSolutionSize scopes tables to site references (not publisher prefix) + emits relationships', async (t) => { + const fs = require('fs'); + const os = require('os'); + const path = require('path'); + + // Temp site root with table permissions referencing only bp_permit + bp_permitstep. + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'est-scope-')); + t.after(() => fs.rmSync(root, { recursive: true, force: true })); + const tpDir = path.join(root, '.powerpages-site', 'table-permissions'); + fs.mkdirSync(tpDir, { recursive: true }); + for (const [base, entity] of [['Permit', 'bp_permit'], ['Step', 'bp_permitstep'], ['Notes', 'annotation']]) { + fs.writeFileSync(path.join(tpDir, `${base}.tablepermission.yml`), + `adx_entitypermission_webrole:\n- ad89f5ee-8665-f111-a826-6045bd00fdda\nentitylogicalname: ${entity}\nentityname: ${base}\nid: f03fefed-8665-f111-a826-000d3a597e6a\nscope: 756150000\n`); + } + + withMockedMakeRequest(t, async ({ url }) => { + if (url.includes('OneToManyRelationships')) { + // bp_permit has a lookup to bp_permitstep (both in scope) + one to contact (out of scope). + if (url.includes("LogicalName='bp_permit'")) { + return { statusCode: 200, body: JSON.stringify({ value: [ + { SchemaName: 'bp_permit_step', ReferencedEntity: 'bp_permit', ReferencingEntity: 'bp_permitstep', ReferencingAttribute: 'bp_permitid' }, + { SchemaName: 'contact_permit', ReferencedEntity: 'contact', ReferencingEntity: 'bp_permit', ReferencingAttribute: 'bp_contactid' }, + ] }) }; + } + return { statusCode: 200, body: JSON.stringify({ value: [] }) }; + } + if (url.includes('ManyToManyRelationships')) return { statusCode: 200, body: JSON.stringify({ value: [] }) }; + if (url.includes('/Attributes')) return { statusCode: 200, body: JSON.stringify({ value: [{ LogicalName: 'c1' }, { LogicalName: 'c2' }] }) }; + if (url.includes('EntityDefinitions') && url.includes('IsCustomEntity')) { + // Env has 4 custom tables; only bp_* are referenced. new_* are the prefix-noise. + return { statusCode: 200, body: JSON.stringify({ value: [ + { LogicalName: 'bp_permit', MetadataId: 'm1', IsCustomEntity: true, IsManaged: false }, + { LogicalName: 'bp_permitstep', MetadataId: 'm2', IsCustomEntity: true, IsManaged: false }, + { LogicalName: 'new_unrelated1', MetadataId: 'm3', IsCustomEntity: true, IsManaged: false }, + { LogicalName: 'new_unrelated2', MetadataId: 'm4', IsCustomEntity: true, IsManaged: false }, + ] }) }; + } + if (url.includes('powerpagecomponents') && url.includes('$count')) { + return { statusCode: 200, body: JSON.stringify({ '@odata.count': 0, value: [] }) }; + } + return { statusCode: 200, body: JSON.stringify({ value: [] }) }; + }); + + const result = await estimateSolutionSize({ + envUrl: 'https://test.crm.dynamics.com', + websiteRecordId: '00000000-0000-0000-0000-000000000001', + publisherPrefix: 'new', // the (now-irrelevant for tables) shared prefix + projectRoot: root, + token: 'fake-token', + }); + + assert.equal(result.tableCount, 2, 'only the 2 site-referenced bp_* tables — NOT the 2 new_* prefix matches'); + assert.equal(result.tableCountScope, 'site-referenced'); + assert.deepEqual(result.tables.map((x) => x.logicalName).sort(), ['bp_permit', 'bp_permitstep']); + // Edge between the two scoped tables; the contact edge is dropped (out of scope). + assert.deepEqual(result.tableRelationships, [['bp_permit', 'bp_permitstep']]); +}); + +test('estimateSolutionSize tableCountScope is "unavailable" with no local signal (never a prefix dump)', async (t) => { + withMockedMakeRequest(t, async ({ url }) => { + if (url.includes('EntityDefinitions') && url.includes('IsCustomEntity')) { + return { statusCode: 200, body: JSON.stringify({ value: [ + { LogicalName: 'new_a', MetadataId: 'm1', IsCustomEntity: true, IsManaged: false }, + { LogicalName: 'new_b', MetadataId: 'm2', IsCustomEntity: true, IsManaged: false }, + ] }) }; + } + if (url.includes('powerpagecomponents') && url.includes('$count')) { + return { statusCode: 200, body: JSON.stringify({ '@odata.count': 0, value: [] }) }; + } + return { statusCode: 200, body: JSON.stringify({ value: [] }) }; + }); + const result = await estimateSolutionSize({ + envUrl: 'https://test.crm.dynamics.com', + websiteRecordId: '00000000-0000-0000-0000-000000000001', + publisherPrefix: 'new', + token: 'fake-token', // no projectRoot + }); + assert.equal(result.tableCount, 0, 'no .powerpages-site signal -> zero tables, NOT the env-wide prefix dump'); + assert.equal(result.tableCountScope, 'unavailable'); + assert.deepEqual(result.tableRelationships, []); +}); diff --git a/plugins/power-pages/scripts/tests/integration/discover-integration.test.js b/plugins/power-pages/scripts/tests/integration/discover-integration.test.js index 57e925265..e7a317663 100644 --- a/plugins/power-pages/scripts/tests/integration/discover-integration.test.js +++ b/plugins/power-pages/scripts/tests/integration/discover-integration.test.js @@ -6,11 +6,27 @@ const test = require('node:test'); const assert = require('node:assert/strict'); +const fs = require('fs'); +const os = require('os'); +const path = require('path'); const { startMock } = require('./mock-dataverse'); const { discoverSiteComponents, } = require('../../lib/discover-site-components'); +// Temp site root whose table permissions reference the given entities — the +// site-referenced scoping signal that replaced the publisher-prefix table dump. +function makeSiteRoot(entityLogicalNames) { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'dsc-int-')); + const dir = path.join(root, '.powerpages-site', 'table-permissions'); + fs.mkdirSync(dir, { recursive: true }); + entityLogicalNames.forEach((entity, i) => { + fs.writeFileSync(path.join(dir, `perm-${i}.tablepermission.yml`), + `adx_entitypermission_webrole:\n- ad89f5ee-8665-f111-a826-6045bd00fdda\nentitylogicalname: ${entity}\nentityname: Perm ${i}\nid: f03fefed-8665-f111-a826-000d3a597e6a\nscope: 756150000\n`); + }); + return root; +} + test('integration: discover follows @odata.nextLink pagination against a real HTTP server', async () => { let mockBase = null; @@ -217,23 +233,30 @@ test('integration: discover with publisherPrefix queries env vars + tables endpo LogicalName: 'contoso_account', SchemaName: 'contoso_Account', DisplayName: { UserLocalizedLabel: { Label: 'Account' } }, + IsCustomEntity: true, + IsManaged: false, }, { MetadataId: 'meta-2', LogicalName: 'other_widget', SchemaName: 'other_Widget', DisplayName: { UserLocalizedLabel: { Label: 'Widget' } }, + IsCustomEntity: true, + IsManaged: false, }, ], }, }, ]); + // Site references contoso_account only -> other_widget is dropped (unreferenced). + const projectRoot = makeSiteRoot(['contoso_account']); try { const result = await discoverSiteComponents({ envUrl: mock.baseUrl, token: 'x', siteId: 'site-42', publisherPrefix: 'contoso', + projectRoot, }); assert.equal(result.envVars.length, 1); assert.equal(result.envVars[0].schemaName, 'contoso_FeatureFlag'); @@ -246,5 +269,6 @@ test('integration: discover with publisherPrefix queries env vars + tables endpo assert.equal(edCalls.length, 1); } finally { await mock.close(); + fs.rmSync(projectRoot, { recursive: true, force: true }); } }); diff --git a/plugins/power-pages/scripts/tests/query-metadata.test.js b/plugins/power-pages/scripts/tests/query-metadata.test.js new file mode 100644 index 000000000..9433a9f3c --- /dev/null +++ b/plugins/power-pages/scripts/tests/query-metadata.test.js @@ -0,0 +1,44 @@ +'use strict'; + +const test = require('node:test'); +const assert = require('node:assert/strict'); + +const { queryCustomUnmanagedTables } = require('../lib/query-metadata'); + +function fakeRequestReturning(rows, { paginate = false } = {}) { + let served = false; + return async ({ url }) => { + if (paginate && !served && !/page2/.test(url)) { + served = true; + return { + statusCode: 200, + body: JSON.stringify({ value: rows.slice(0, 1), '@odata.nextLink': 'https://x/page2' }), + }; + } + const body = paginate ? { value: rows.slice(1) } : { value: rows }; + return { statusCode: 200, body: JSON.stringify(body) }; + }; +} + +test('queryCustomUnmanagedTables keeps only custom + unmanaged tables (with schema/display)', async () => { + const req = fakeRequestReturning([ + { LogicalName: 'bp_permit', MetadataId: 'm1', SchemaName: 'bp_Permit', DisplayName: { UserLocalizedLabel: { Label: 'Permit' } }, IsCustomEntity: true, IsManaged: false }, + { LogicalName: 'bp_managed', MetadataId: 'm2', IsCustomEntity: true, IsManaged: true }, // managed -> dropped + { LogicalName: 'account', MetadataId: 'm3', IsCustomEntity: false, IsManaged: false }, // system -> dropped (not custom) + { LogicalName: 'bp_inspection', MetadataId: 'm4', SchemaName: 'bp_Inspection', IsCustomEntity: true, IsManaged: false }, // no DisplayName -> falls back to SchemaName + ]); + const out = await queryCustomUnmanagedTables('https://org.crm.dynamics.com/', 'tok', req); + assert.deepEqual(out, [ + { logicalName: 'bp_permit', metadataId: 'm1', schemaName: 'bp_Permit', displayName: 'Permit' }, + { logicalName: 'bp_inspection', metadataId: 'm4', schemaName: 'bp_Inspection', displayName: 'bp_Inspection' }, + ]); +}); + +test('queryCustomUnmanagedTables paginates via @odata.nextLink', async () => { + const req = fakeRequestReturning([ + { LogicalName: 'a_one', MetadataId: 'm1', IsCustomEntity: true, IsManaged: false }, + { LogicalName: 'a_two', MetadataId: 'm2', IsCustomEntity: true, IsManaged: false }, + ], { paginate: true }); + const out = await queryCustomUnmanagedTables('https://org.crm.dynamics.com', 'tok', req); + assert.deepEqual(out.map((t) => t.logicalName), ['a_one', 'a_two']); +}); diff --git a/plugins/power-pages/scripts/tests/query-table-relationships.test.js b/plugins/power-pages/scripts/tests/query-table-relationships.test.js new file mode 100644 index 000000000..7c5bd446f --- /dev/null +++ b/plugins/power-pages/scripts/tests/query-table-relationships.test.js @@ -0,0 +1,52 @@ +'use strict'; + +const test = require('node:test'); +const assert = require('node:assert/strict'); + +const { fetchTableRelationships } = require('../lib/query-table-relationships'); + +function router(map) { + // map: substring -> { statusCode, value } | throws if error:true + return async ({ url }) => { + for (const [needle, resp] of Object.entries(map)) { + if (url.includes(needle)) { + if (resp.error) return { error: resp.error }; + return { statusCode: resp.statusCode || 200, body: JSON.stringify({ value: resp.value || [] }) }; + } + } + return { statusCode: 200, body: JSON.stringify({ value: [] }) }; + }; +} + +test('fetchTableRelationships maps OneToMany + ManyToMany shapes', async () => { + const req = router({ + 'OneToManyRelationships': { + value: [{ SchemaName: 'bp_permit_step', ReferencedEntity: 'bp_permit', ReferencingEntity: 'bp_step', ReferencingAttribute: 'bp_permitid' }], + }, + 'ManyToManyRelationships': { + value: [{ SchemaName: 'bp_permit_tag', Entity1LogicalName: 'bp_permit', Entity2LogicalName: 'bp_tag' }], + }, + }); + const out = await fetchTableRelationships('https://org.crm.dynamics.com/', 'bp_permit', 'tok', req); + assert.deepEqual(out.oneToMany, [ + { schemaName: 'bp_permit_step', referencedEntity: 'bp_permit', referencingEntity: 'bp_step', referencingAttribute: 'bp_permitid' }, + ]); + assert.deepEqual(out.manyToMany, [ + { schemaName: 'bp_permit_tag', entity1: 'bp_permit', entity2: 'bp_tag' }, + ]); +}); + +test('fetchTableRelationships swallows ManyToMany errors (best-effort)', async () => { + const req = router({ + 'OneToManyRelationships': { value: [] }, + 'ManyToManyRelationships': { statusCode: 404, value: [] }, + }); + const out = await fetchTableRelationships('https://org.crm.dynamics.com', 'x_t', 'tok', req); + assert.deepEqual(out.manyToMany, []); + assert.deepEqual(out.oneToMany, []); +}); + +test('fetchTableRelationships propagates OneToMany errors', async () => { + const req = router({ 'OneToManyRelationships': { statusCode: 404, value: [] } }); + await assert.rejects(() => fetchTableRelationships('https://org.crm.dynamics.com', 'missing', 'tok', req), /HTTP 404/); +}); diff --git a/plugins/power-pages/scripts/tests/resolve-site-tables.test.js b/plugins/power-pages/scripts/tests/resolve-site-tables.test.js new file mode 100644 index 000000000..7dfebc315 --- /dev/null +++ b/plugins/power-pages/scripts/tests/resolve-site-tables.test.js @@ -0,0 +1,98 @@ +'use strict'; + +const test = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('fs'); +const os = require('os'); +const path = require('path'); + +const { collectReferencedEntityNames, scopeCustomTables } = require('../lib/resolve-site-tables'); + +function makeProject(t) { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'resolve-site-tables-')); + t.after(() => fs.rmSync(root, { recursive: true, force: true })); + return root; +} + +function writeTablePermission(root, fileBase, entityLogicalName, displayName) { + const dir = path.join(root, '.powerpages-site', 'table-permissions'); + fs.mkdirSync(dir, { recursive: true }); + fs.writeFileSync( + path.join(dir, `${fileBase}.tablepermission.yml`), + [ + 'adx_entitypermission_webrole:', + '- ad89f5ee-8665-f111-a826-6045bd00fdda', + 'append: true', + 'appendto: true', + 'create: true', + 'delete: true', + `entitylogicalname: ${entityLogicalName}`, + `entityname: ${displayName}`, + 'id: f03fefed-8665-f111-a826-000d3a597e6a', + 'read: true', + 'scope: 756150000', + 'write: true', + '', + ].join('\n'), + ); +} + +test('collectReferencedEntityNames: extracts entitylogicalname from table permissions (EDM bp_* fixture)', (t) => { + const root = makeProject(t); + writeTablePermission(root, 'Admin-to-Permits', 'bp_defaultapplication', 'Admin to Permits'); + writeTablePermission(root, 'Permit-Steps', 'bp_permitstep', 'Permit Steps'); + writeTablePermission(root, 'Notes-Global', 'annotation', 'Notes Global'); // standard table + + const { names, available, sources } = collectReferencedEntityNames({ projectRoot: root }); + assert.equal(available, true); + assert.equal(sources.tablePermissions, 3); + assert.ok(names.has('bp_defaultapplication')); + assert.ok(names.has('bp_permitstep')); + assert.ok(names.has('annotation')); + // entityname (display label) must NOT be added. + assert.ok(!names.has('admin to permits')); +}); + +test('collectReferencedEntityNames: unions datamodel manifest entities', (t) => { + const root = makeProject(t); + writeTablePermission(root, 'Permit', 'bp_permit', 'Permit'); + fs.writeFileSync( + path.join(root, '.datamodel-manifest.json'), + JSON.stringify({ entities: [{ logicalName: 'new_extra' }, { logicalName: 'bp_permit' }] }), + ); + const { names, sources } = collectReferencedEntityNames({ projectRoot: root }); + assert.ok(names.has('bp_permit')); + assert.ok(names.has('new_extra')); + assert.ok(sources.manifest >= 1); +}); + +test('collectReferencedEntityNames: available=false when no .powerpages-site and no manifest', (t) => { + const root = makeProject(t); + const { names, available } = collectReferencedEntityNames({ projectRoot: root }); + assert.equal(available, false); + assert.equal(names.size, 0); +}); + +test('scopeCustomTables: keeps only referenced custom tables; drops unreferenced + standard', (t) => { + const root = makeProject(t); + writeTablePermission(root, 'Permit', 'bp_permit', 'Permit'); + writeTablePermission(root, 'Inspection', 'BP_Inspection', 'Inspection'); // mixed case + writeTablePermission(root, 'Notes', 'annotation', 'Notes'); // standard + + const { names } = collectReferencedEntityNames({ projectRoot: root }); + + // Simulate the env's custom-unmanaged tables (the old prefix dump would return all of these). + const customUnmanaged = [ + { logicalName: 'bp_permit', metadataId: 'm1' }, + { logicalName: 'bp_inspection', metadataId: 'm2' }, + { logicalName: 'new_unrelated1', metadataId: 'm3' }, // not referenced -> dropped + { logicalName: 'new_unrelated2', metadataId: 'm4' }, // not referenced -> dropped + ]; + const scoped = scopeCustomTables(names, customUnmanaged).map((t2) => t2.logicalName).sort(); + assert.deepEqual(scoped, ['bp_inspection', 'bp_permit']); + // 'annotation' is referenced but not in the custom-unmanaged list -> naturally excluded. +}); + +test('scopeCustomTables: empty referenced set -> empty (never a prefix dump)', () => { + assert.deepEqual(scopeCustomTables(new Set(), [{ logicalName: 'new_x' }]), []); +}); diff --git a/plugins/power-pages/scripts/tests/validation-helpers.test.js b/plugins/power-pages/scripts/tests/validation-helpers.test.js index 9b7e2dc87..d48411ca8 100644 --- a/plugins/power-pages/scripts/tests/validation-helpers.test.js +++ b/plugins/power-pages/scripts/tests/validation-helpers.test.js @@ -73,3 +73,28 @@ test('findProjectRoot: returns null when neither marker is present', (t) => { assert.equal(findProjectRoot(root), null); }); +// --- odataGet / odataGetAll (shared pagination) ------------------------------ + +test('odataGetAll follows @odata.nextLink and aggregates all pages', async () => { + const { odataGetAll } = require(helpersPath); + const pages = { + 'https://x/api/data/v9.2/things': { value: [{ id: 1 }, { id: 2 }], '@odata.nextLink': 'https://x/page2' }, + 'https://x/page2': { value: [{ id: 3 }] }, + }; + const fakeRequest = async ({ url }) => ({ statusCode: 200, body: JSON.stringify(pages[url]) }); + const rows = await odataGetAll('https://x/api/data/v9.2/things', 'tok', fakeRequest); + assert.deepEqual(rows.map((r) => r.id), [1, 2, 3]); +}); + +test('odataGet throws on non-2xx', async () => { + const { odataGet } = require(helpersPath); + const fakeRequest = async () => ({ statusCode: 404, body: 'not found' }); + await assert.rejects(() => odataGet('https://x/y', 'tok', fakeRequest), /HTTP 404/); +}); + +test('odataGet throws on transport error', async () => { + const { odataGet } = require(helpersPath); + const fakeRequest = async () => ({ error: 'ECONNRESET' }); + await assert.rejects(() => odataGet('https://x/y', 'tok', fakeRequest), /OData request failed/); +}); + diff --git a/plugins/power-pages/skills/audit-permissions/scripts/query-table-relationships.js b/plugins/power-pages/skills/audit-permissions/scripts/query-table-relationships.js index a5bdf93f3..ca98582dc 100644 --- a/plugins/power-pages/skills/audit-permissions/scripts/query-table-relationships.js +++ b/plugins/power-pages/skills/audit-permissions/scripts/query-table-relationships.js @@ -1,19 +1,21 @@ #!/usr/bin/env node +// Thin CLI wrapper over scripts/lib/query-table-relationships.js. // Queries Dataverse for one-to-many relationships on a given table. // Returns JSON array of { schemaName, referencedEntity, referencingEntity, referencingAttribute }. // // Usage: // node query-table-relationships.js --envUrl --table // -// Output (stdout): JSON array -// [{ "schemaName": "cr4fc_order_orderitem", "referencedEntity": "cr4fc_order", "referencingEntity": "cr4fc_orderitem", "referencingAttribute": "cr4fc_orderid" }] +// Output (stdout): JSON array (OneToMany relationships only — the audit-permissions +// relationship-scope validation consumes schemaName + referencedEntity). // // Exit codes: // 0 = success (JSON on stdout) // 1 = error (message on stderr) -const { getAuthToken, makeRequest } = require('../../../scripts/lib/validation-helpers'); +const { getAuthToken } = require('../../../scripts/lib/validation-helpers'); +const { fetchTableRelationships } = require('../../../scripts/lib/query-table-relationships'); const args = process.argv.slice(2); function getArg(name) { @@ -37,29 +39,10 @@ if (!envUrl || !table) { } try { - const result = await makeRequest({ - url: `${envUrl}/api/data/v9.2/EntityDefinitions(LogicalName='${table}')/OneToManyRelationships?$select=SchemaName,ReferencedEntity,ReferencingEntity,ReferencingAttribute`, - headers: { - Authorization: `Bearer ${token}`, - Accept: 'application/json', - }, - timeout: 15000, - }); - - if (result.error || result.statusCode !== 200) { - process.stderr.write(`API error (${result.statusCode}): ${result.error || result.body}\n`); - process.exit(1); - } - - const parsed = JSON.parse(result.body); - const rels = (parsed.value || []).map(r => ({ - schemaName: r.SchemaName, - referencedEntity: r.ReferencedEntity, - referencingEntity: r.ReferencingEntity, - referencingAttribute: r.ReferencingAttribute, - })); - - process.stdout.write(JSON.stringify(rels, null, 2) + '\n'); + // OneToMany errors propagate here (preserves the original exit-1-on-API-error + // behavior); ManyToMany is best-effort inside the lib and unused by this CLI. + const { oneToMany } = await fetchTableRelationships(envUrl, table, token); + process.stdout.write(JSON.stringify(oneToMany, null, 2) + '\n'); } catch (err) { process.stderr.write(`Request failed: ${err.message}\n`); process.exit(1); diff --git a/plugins/power-pages/skills/setup-solution/SKILL.md b/plugins/power-pages/skills/setup-solution/SKILL.md index 2c44aeb39..e46d11a3e 100644 --- a/plugins/power-pages/skills/setup-solution/SKILL.md +++ b/plugins/power-pages/skills/setup-solution/SKILL.md @@ -304,14 +304,18 @@ GET {envUrl}/api/data/v9.2/powerpagesitelanguages?$filter=_powerpagesiteid_value ``` Store all language IDs. -**D. Dataverse tables** — always discover from the environment, don't rely on a manifest file alone: +**D. Dataverse tables** — discover the tables the **site actually references**, NOT every table sharing the publisher prefix. -1. Read `.datamodel-manifest.json` if present (for the known list of tables created by `setup-datamodel`) -2. **Also** query the environment directly for all custom unmanaged tables, filtering by the publisher prefix: -``` -GET {envUrl}/api/data/v9.2/EntityDefinitions?$select=LogicalName,MetadataId,IsManaged,IsCustomEntity +> **Why not publisher prefix:** prefix-matching over-counts catastrophically with a shared/default publisher (`new_`, env default) — a 6-table site can match 22 unrelated tables — and it also *misses* the site's real tables when they come from a different prefix (e.g. a `bp_*` template under an `edm` publisher). The authoritative signal (SME-confirmed) is the site's **table permissions**: "If a table is used in the site there will be permissions for it." + +Run the shared discovery helper, which scopes custom tables to the site's table permissions (+ datamodel manifest) intersected with the env's custom-unmanaged tables: +```bash +node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/discover-site-components.js" \ + --envUrl "{envUrl}" --token "{token}" --siteId "{websiteRecordId}" \ + --projectRoot "." \ + {if .datamodel-manifest.json elsewhere: --datamodelManifest ""} ``` -Filter client-side: `IsCustomEntity === true && IsManaged === false`. Group by publisher prefix (characters before first `_`). Present only tables whose prefix matches the site publisher — or if no prefix match, present all custom unmanaged tables and let the user decide. +Use the returned `customTables[]` (each `{ id, logicalName, schemaName, displayName }` — `id` is the MetadataId for the `AddSolutionComponent` call). This is already the correct, site-scoped list — do **not** re-filter by prefix. > **Important note on tables**: Dataverse solutions carry **schema only** — entity definitions, columns, relationships, forms, and views. Table **data/records** do NOT travel with the solution. If the target environment needs seed/reference data, that requires a separate data migration step. @@ -628,11 +632,13 @@ If both `missing.powerpagecomponents` (after filtering) and `missing.siteLanguag **This is the key decision point.** Build a full manifest of everything that will be added and present it to the user before writing anything. -If custom tables were discovered, ask via `AskUserQuestion` with `multiSelect: true` **before** showing the final manifest: -- First option: **"Include all N tables (Recommended)"** — pre-selected default +If custom tables were discovered (the site-referenced set from step D), ask via `AskUserQuestion` with `multiSelect: true` **before** showing the final manifest: +- First option: **"Include all N referenced tables (Recommended)"** — pre-selected default. N is the count of tables the site actually references (not an env-wide prefix list). - Then one option per table: `{logicalName} ({DisplayName})` - Last option: **"Exclude all tables"** +> The default list is already scoped to the site's real tables (step D). If the user knows of an additional table the site needs that has no permission yet, they can add it manually — but the default must never be a publisher-prefix dump. + Present as a structured summary: ``` From 56b996f1385fdf3b4a95678d528c37d862a067fb Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 14:41:54 +0530 Subject: [PATCH 12/38] Fix FFD bin-packing under-allocation that overflowed the schema-attr cap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit deriveDomainsByCapacity seeded the packer's bin count from a LOWER bound (max(ceil(tables/maxTableCount), ceil(attrs/maxSchemaAttrs))), so when independent (no-edge) clusters fragment, FFD ran out of bins and dropped the non-fitting cluster into the least-loaded bucket — overflowing it past maxSchemaAttrs with no warning (the oversized-cluster guard only checks per- cluster table COUNT, not attrs). Verified repro: 4 independent 8000-attr tables, maxSchemaAttrs 15000 -> seed n=3 -> one bucket holds 16000 attrs. Seed the packer with the maximum permitted bins instead (one per cluster, capped at maxSchemaSplitSolutions). FFD still consolidates — clusters that fit together share a bin and empty bins are dropped, so the solution count stays minimal — but a cluster that fits nowhere opens a NEW bin rather than overflowing. The existing 16000-attr/2-solution test is unchanged (FFD still consolidates); added a regression test for the 4-independent-table overflow case. 1211 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../scripts/lib/compute-split-plan.js | 16 +++++---- .../scripts/tests/compute-split-plan.test.js | 33 +++++++++++++++++++ 2 files changed, 42 insertions(+), 7 deletions(-) diff --git a/plugins/power-pages/scripts/lib/compute-split-plan.js b/plugins/power-pages/scripts/lib/compute-split-plan.js index f59028400..82305136e 100644 --- a/plugins/power-pages/scripts/lib/compute-split-plan.js +++ b/plugins/power-pages/scripts/lib/compute-split-plan.js @@ -427,14 +427,16 @@ function deriveDomainsByCapacity(estimate, thresholds) { if (tables.length === 0) return [{ name: 'Tables', tableLogicalNames: [] }]; const clusters = buildTableClusters(tables, estimate.tableRelationships || []); - const totalAttrs = tables.reduce((s, t) => s + t.attributeCount, 0); const ceiling = (thresholds && thresholds.maxSchemaSplitSolutions) || 8; - let n = Math.max( - 1, - Math.ceil(tables.length / thresholds.maxTableCount), - Math.ceil(totalAttrs / Math.max(thresholds.maxSchemaAttrs, 1)), - ); - n = Math.min(n, ceiling, clusters.length); + // Seed the packer with the maximum permitted bins (one per cluster, capped at + // maxSchemaSplitSolutions). First-fit-decreasing still consolidates — clusters + // that fit together share a bin and the empty bins are dropped, so the final + // count stays minimal — but a cluster that fits nowhere lands in a NEW bin + // instead of overflowing an existing one. Seeding from a lower bound + // (ceil(tables/maxTable), ceil(attrs/maxAttr)) under-allocated bins and let + // independent attr-heavy clusters bust maxSchemaAttrs in the least-loaded + // bucket, unwarned (the oversized guard only catches per-cluster table count). + const n = Math.min(clusters.length, ceiling); const buckets = packClusters(clusters, n, thresholds); const multi = buckets.length > 1; diff --git a/plugins/power-pages/scripts/tests/compute-split-plan.test.js b/plugins/power-pages/scripts/tests/compute-split-plan.test.js index 48c92c03a..abf985a37 100644 --- a/plugins/power-pages/scripts/tests/compute-split-plan.test.js +++ b/plugins/power-pages/scripts/tests/compute-split-plan.test.js @@ -205,6 +205,39 @@ test('computeSplitPlan Strategy 3 packs tables by capacity (no domains configure assert.equal(tableSolutions.length, 2, '16000 attrs / 15000 cap -> 2 Table solutions, not one-per-table'); }); +test('computeSplitPlan Strategy 3 never overflows the attr cap when independent clusters fragment', () => { + // Regression for the FFD under-allocation bug: 4 INDEPENDENT (no-edge) tables of + // 8000 attrs each = 32000 total. Seeding the packer from the lower bound + // ceil(32000/15000)=3 gave only 3 bins, so the 4th cluster fell into the + // least-loaded bucket -> 16000 attrs (> 15000 cap), unwarned. The packer must + // instead open a 4th bin (clusters.length permits it) so no bucket busts the cap. + const result = computeSplitPlan({ + estimate: baseEstimate({ + tableCount: 4, + schemaAttrCount: 32000, + tables: [ + { logicalName: 'tst_alpha', attributeCount: 8000 }, + { logicalName: 'tst_beta', attributeCount: 8000 }, + { logicalName: 'tst_gamma', attributeCount: 8000 }, + { logicalName: 'tst_delta', attributeCount: 8000 }, + ], + tableRelationships: [], // no edges -> 4 singleton clusters + }), + config: baseConfig(), + meta: { baseName: 'Test', siteName: 'Test Site' }, + }); + assert.equal(result.splitStrategy, 'strategy-3-schema-segmentation'); + const tableSolutions = result.proposedSolutions.filter( + (s) => Array.isArray(s.componentTypes) && s.componentTypes.length === 1 && s.componentTypes[0] === 'Table', + ); + // 4 independent 8000-attr tables -> 4 single-table solutions (each 8000 < 15000), + // NOT 3 with one 16000-attr overflow bucket. + assert.equal(tableSolutions.length, 4, '4 independent 8000-attr tables -> 4 Table solutions (no attr-cap overflow)'); + for (const s of tableSolutions) { + assert.equal(s.tableLogicalNames.length, 1, `${s.uniqueName} must hold exactly one table — no bucket over the attr cap`); + } +}); + test('computeSplitPlan additive Strategy 4 prepends EnvVars solution', () => { const result = computeSplitPlan({ estimate: baseEstimate({ totalSizeMB: 142, webFilesAggregateMB: 110, envVarCount: 800 }), From 5e3d1abb03084f8673af44563c1239b7b5d99c5e Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 18:16:00 +0530 Subject: [PATCH 13/38] Wire --projectRoot into the remaining discover-site-components consumers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The site-referenced table discovery (this PR) made discover-site-components return customTables (and thus missing.customTables) ONLY when --projectRoot (or --datamodelManifest) is passed — without a local signal it returns [] rather than the old publisher-prefix dump. setup-solution Step D was updated to pass --projectRoot, but three other consumers that read missing.customTables were missed, so their "custom tables missing from the solution" completeness check silently reported 0 for every site: - export-solution Phase 2.5 (pre-export completeness) - plan-alm Phase 1 (pre-plan completeness) - deploy-pipeline Phase 3.5 (pre-sync completeness) Added --projectRoot "." to all three so the check is restored with the correct site-scoped count. (setup-solution Phase 5.4b/5.4c calls consume missing.envVars / missing.powerpagecomponents / siteLanguages, not customTables, so they're correct as-is with --publisherPrefix/--solutionId.) 1211 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- plugins/power-pages/skills/deploy-pipeline/SKILL.md | 3 ++- plugins/power-pages/skills/export-solution/SKILL.md | 3 ++- plugins/power-pages/skills/plan-alm/SKILL.md | 3 ++- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/plugins/power-pages/skills/deploy-pipeline/SKILL.md b/plugins/power-pages/skills/deploy-pipeline/SKILL.md index 7465cc0c9..f4635a15d 100644 --- a/plugins/power-pages/skills/deploy-pipeline/SKILL.md +++ b/plugins/power-pages/skills/deploy-pipeline/SKILL.md @@ -300,7 +300,8 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/discover-site-components.js" \ --envUrl "{devEnvUrl}" --token "{DEV_TOKEN}" \ --siteId "{websiteRecordId from .solution-manifest.json}" \ --publisherPrefix "{publisherPrefix from .solution-manifest.json}" \ - --solutionId "{solutionId from .solution-manifest.json}" + --solutionId "{solutionId from .solution-manifest.json}" \ + --projectRoot "." ``` Parse stdout and evaluate `missing.*`. **Before doing anything else**, capture the **pre-sync state** so a post-sync re-confirmation gate can show what changed: diff --git a/plugins/power-pages/skills/export-solution/SKILL.md b/plugins/power-pages/skills/export-solution/SKILL.md index 489796e7d..409409e37 100644 --- a/plugins/power-pages/skills/export-solution/SKILL.md +++ b/plugins/power-pages/skills/export-solution/SKILL.md @@ -158,7 +158,8 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/discover-site-components.js" \ --envUrl "{envUrl}" --token "{token}" \ --siteId "{websiteRecordId}" \ --publisherPrefix "{publisherPrefix from .solution-manifest.json}" \ - --solutionId "{solutionId}" + --solutionId "{solutionId}" \ + --projectRoot "." ``` Parse stdout and evaluate `missing`. **Before doing anything else**, capture the **pre-sync state** so a post-sync re-confirmation gate can show what changed: diff --git a/plugins/power-pages/skills/plan-alm/SKILL.md b/plugins/power-pages/skills/plan-alm/SKILL.md index c36b2dd1d..ed7a34b32 100644 --- a/plugins/power-pages/skills/plan-alm/SKILL.md +++ b/plugins/power-pages/skills/plan-alm/SKILL.md @@ -242,7 +242,8 @@ Steps: --envUrl "{envUrl}" --token "{token}" \ --siteId "{websiteRecordId from powerpages.config.json}" \ --publisherPrefix "{solutionManifest.publisher.prefix}" \ - --solutionId "{solutionManifest.solution.solutionId}" + --solutionId "{solutionManifest.solution.solutionId}" \ + --projectRoot "." ``` Parse stdout and evaluate `missing.*`: From 696d41c1e58d65e51c86dc9d9c0553f0855e29b2 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 12:41:57 +0530 Subject: [PATCH 14/38] Enforce ALM plan refresh via PostToolUse reconcile backstop (auto-heal) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The refresh-alm-plan-data.js calls in each ALM SKILL.md are advisory markdown — silently dropped on session fragmentation, manual execution, or oversight. Three observed gaps where the rendered plan never reflected real run state: ensure-pipelines-host (no refresh call at all), setup-pipeline (refresh lives in Phase 7; phases 5-6 run manually after a resume skip it), activate-site (Phase 5.2b refresh missed). Fix is auto-heal, not fail-validation (per feedback_skill_validation_hooks: never hard-block). After ANY ALM plan skill completes, the centralized PostToolUse hook spawnSyncs `refresh-alm-plan-data.js --reconcile --render`, which ingests any marker (docs/alm/last-*.json) newer than the plan. Because it fires on any ALM skill (not just the marker's writer), a skip in skill A is healed when the next ALM skill B completes. Best-effort and non-blocking: never changes the hook's exit code, honors .alm-deferred, idempotent. - powerpages-hook-utils.js: ALM_PLAN_SKILLS set + isAlmPlanSkill(value) - run-skill-posttool-validation.js: reconcile backstop after the validator - ensure-pipelines-host SKILL.md: explicit Phase 6 self-refresh (direct gap-1 fix; hook reconcile remains the backstop) The reconcile mode + ensure-pipelines-host phase in refresh-alm-plan-data.js ship in the plan-only PR earlier in this stack. 1212 tests pass, alm-lint 0. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../power-pages/.claude-plugin/plugin.json | 2 +- plugins/power-pages/AGENTS.md | 4 ++- .../hooks/run-skill-posttool-validation.js | 36 ++++++++++++++++++- .../scripts/lib/powerpages-hook-utils.js | 31 ++++++++++++++++ .../tests/powerpages-hook-utils.test.js | 26 ++++++++++++++ .../skills/ensure-pipelines-host/SKILL.md | 11 ++++++ 6 files changed, 107 insertions(+), 3 deletions(-) diff --git a/plugins/power-pages/.claude-plugin/plugin.json b/plugins/power-pages/.claude-plugin/plugin.json index 13763c7f2..f3fa3f687 100644 --- a/plugins/power-pages/.claude-plugin/plugin.json +++ b/plugins/power-pages/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "power-pages", - "version": "2.4.0", + "version": "2.5.0", "description": "Create and deploy Power Pages sites using modern development approaches. Supports code sites (SPAs) with React, Angular, Vue, or Astro. Includes ALM orchestration (plan-alm) with a solution-splitting decision tree, per-solution pipelines, Azure Blob asset advisory, manifest schema v2 for multi-solution deployments, and force-link remediation for cross-host pipeline migrations.", "author": { "name": "Microsoft", diff --git a/plugins/power-pages/AGENTS.md b/plugins/power-pages/AGENTS.md index 64555df68..c0ca71cfd 100644 --- a/plugins/power-pages/AGENTS.md +++ b/plugins/power-pages/AGENTS.md @@ -170,6 +170,8 @@ Skills are defined in `SKILL.md` files with YAML frontmatter (name, description, Hook registration is centralized in `hooks/hooks.json` — a single PostToolUse hook (matcher `Skill`) runs `hooks/run-skill-posttool-validation.js` after every Skill tool call. The runner derives tracked skills directly from `skills/*/SKILL.md` via `scripts/lib/powerpages-hook-utils.js`, looks up an optional `skills//scripts/validate*.js` validator for the skill that just completed, and invokes it with the current cwd. +**ALM plan reconcile backstop (auto-heal).** After any **ALM plan skill** completes (`powerpages-hook-utils.js → ALM_PLAN_SKILLS` / `isAlmPlanSkill`) and a `docs/.alm-plan-data.json` exists in the cwd, the runner also `spawnSync`s `refresh-alm-plan-data.js --reconcile --render`. The `refresh-alm-plan-data.js` calls in each SKILL.md are advisory — silently dropped on session fragmentation, manual execution, or oversight — so the reconcile *performs* any refresh whose marker (`docs/alm/last-*.json`) is newer than the plan. This is **best-effort and non-blocking**: it never changes the hook's exit code, honors `.alm-deferred`, and is idempotent. Because it fires on *any* ALM skill (not just the marker's writer), a skip in skill A is healed when the next ALM skill (B) completes. Skills keep their explicit per-phase refresh calls as defense-in-depth + immediate render; the hook is the backstop. + To wire a new skill into validation: 1. Write the validator at `skills//scripts/validate-.js` using the `runValidation((cwd) => { ... })` pattern from `scripts/lib/validation-helpers.js`. @@ -225,7 +227,7 @@ Shared lib modules live at `scripts/lib/` and are imported by other scripts via - `scripts/lib/link-site-setting-to-env-var.js`: Links an `mspp_sitesetting` record to an `environmentvariabledefinition` via OData PATCH on the v9.0 API (not v9.2). HAR-confirmed: navigation property is `EnvironmentValue@odata.bind`; headers `if-match: *` and `clienthost: Browser` are required (omitting causes 400). Args: `--envUrl`, `--token`, `--siteSettingId`, `--definitionId`, `--schemaName`. Output: `{ ok, verified, siteSettingId, definitionId }`. - `scripts/lib/install-pipelines-app.js`: Installs the Power Platform Pipelines application package on an existing Dataverse env (replaces ensure-pipelines-host Phase 4.B's manual PPAC click-through). Resolution: BAP `applicationPackages` LIST + `/install` POST → 200 sync / 202 + Location poll, with PAC CLI fallback (`pac application install --environment-id ... --application-list msdyn_AppDeploymentAnchor`) on 401/403/5xx. 409 on install POST treated as idempotent (already-installed). Args: `--bapToken`, `--envId`, `--instanceApiUrl` (opt — for verification probe), `--hostToken` (opt), `--no-pac-fallback` (opt), `--correlationId`, `--timeoutSec`, `--apiVersion`, `--bapBase`. Output: `{ status, alreadyInstalled, installPath: 'bap'\|'pac'\|'cached', packageUniqueName, pipelinesSolutionVersion, durationSec, correlationId, pollAttempts, locationHeader, pacFallbackReason }`. - `scripts/lib/discover-env-var-definitions.js`: Enumerates env var definitions matching a publisher prefix and joins each with its bound `mspp_sitesetting` (if any). Used by `plan-alm` Phase 1 Step 10b to populate `planData.envVars[]` with row-level metadata so the rendered plan's Env Variables tab shows schema name, type, default value, and bound site setting per definition (instead of just a count). Args: `--envUrl`, `--publisherPrefix`, `--websiteRecordId`, `--token` (opt). Output: `{ envVars: [{ schemaName, type, defaultValue, siteSetting }], count }`. Degrades gracefully (empty array, exit 0) on auth failure or query errors so the renderer's count-summary fallback can take over. -- `scripts/lib/refresh-alm-plan-data.js`: Updates `docs/.alm-plan-data.json` with post-run state from the marker files written by setup-pipeline / deploy-pipeline / ensure-pipelines-host / test-site / import-solution / activate-site / configure-env-variables / setup-solution / export-solution, then optionally re-renders `docs/alm-plan.html`. Used by plan-alm Phases 6 / 7 / 8 so the rendered Pipelines tab, Validation tab, hostResolution card, env var values matrix, checklist, and risks list reflect actual run state instead of frozen pre-run intent. Args: `--projectRoot`, `--phase` (`setup-solution`/`setup-pipeline`/`configure-env-variables`/`deploy-pipeline`/`export-solution`/`import-solution`/`activate-site`/`test-site`/`finalize`), `--render` (also invoke renderer), `--stageName` (required for `test-site`; preferred for `import-solution`/`activate-site` though both can resolve via marker URL match). Output: `{ ok, phase, dataPath, htmlPath, rendered }`. Returns `ok:false` (soft no-op) when `docs/.alm-plan-data.json` is missing — caller should preserve that file across phases for the helper to work. Plan-alm Phase 3 must NOT delete the file after the initial render — it's read by `check-alm-plan.js` for downstream Phase 0 ALM-plan gates and by this helper for post-run refreshes. **Cross-cutting behaviors**: (a) `setStepStatus` flips the matching entry in `planData.steps[]` to `completed` (or `failed` when the phase's marker indicates failure) — case-insensitive keyword match + stage filter, respects `skip: true`, never regresses completed→pending; (b) `deploy-pipeline` AND `configure-env-variables` both backfill `planData.envVars[i].values{}` from the project root's `deployment-settings.json` so the rendered plan's "Values by Environment" matrix auto-populates (accepts both top-level-stage and nested-`stages` shapes; `SchemaName`/`Value` and camelCase variants; never overwrites a populated cell — manual override wins); (c) `configure-env-variables` and `setup-solution` both re-ingest `docs/alm/last-env-vars.json` (when present) so freshly-created definitions appear in `planData.envVars[]` and `plannedEnvVarCount` zeros out; (d) `export-solution` ingests `docs/alm/last-export.json` into `planData.manualMeta.lastExport` (all 10 marker fields: solutionUniqueName/solutionId/previousVersion/version/managed/sourceEnvironmentUrl/zipPath/fileSizeBytes/asyncOperationId/exportedAt) so the Manual-path tab can show the most recent export. Marker absence is a silent step-sync-only no-op (no `manualMeta.lastExport: null` row in the rendered plan); (e) `deploy-pipeline` ingests the `batchValidation` block from `last-deploy.json` into `planData.pipelineMeta.lastDeploy.batchValidation` (totalSolutions/succeeded/failed/pendingApproval/timedOut/elapsedSeconds/perSolutionStageRunIds) so the rendered plan can show the Phase 3.6 parallel-validation outcome distinct from the serial deploy outcome. Explicitly set to `null` for single-solution / legacy v2 deploys so renderers can branch on it; legacy `elapsedSecondsApprox` field name is accepted and normalized to `elapsedSeconds` on ingest. +- `scripts/lib/refresh-alm-plan-data.js`: Updates `docs/.alm-plan-data.json` with post-run state from the marker files written by setup-pipeline / deploy-pipeline / ensure-pipelines-host / test-site / import-solution / activate-site / configure-env-variables / setup-solution / export-solution, then optionally re-renders `docs/alm-plan.html`. Used by plan-alm Phases 6 / 7 / 8 so the rendered Pipelines tab, Validation tab, hostResolution card, env var values matrix, checklist, and risks list reflect actual run state instead of frozen pre-run intent. Args: `--projectRoot`, `--phase` (`setup-solution`/`setup-pipeline`/`configure-env-variables`/`deploy-pipeline`/`export-solution`/`import-solution`/`activate-site`/`test-site`/`ensure-pipelines-host`/`finalize`) **OR `--reconcile`** (mutually exclusive with `--phase`), `--render` (also invoke renderer), `--stageName` (required for `test-site`; preferred for `import-solution`/`activate-site` though both can resolve via marker URL match). Output: `{ ok, phase, dataPath, htmlPath, rendered }`. Returns `ok:false` (soft no-op) when `docs/.alm-plan-data.json` is missing — caller should preserve that file across phases for the helper to work. Plan-alm Phase 3 must NOT delete the file after the initial render — it's read by `check-alm-plan.js` for downstream Phase 0 ALM-plan gates and by this helper for post-run refreshes. **Cross-cutting behaviors**: (a) `setStepStatus` flips the matching entry in `planData.steps[]` to `completed` (or `failed` when the phase's marker indicates failure) — case-insensitive keyword match + stage filter, respects `skip: true`, never regresses completed→pending; (b) `deploy-pipeline` AND `configure-env-variables` both backfill `planData.envVars[i].values{}` from the project root's `deployment-settings.json` so the rendered plan's "Values by Environment" matrix auto-populates (accepts both top-level-stage and nested-`stages` shapes; `SchemaName`/`Value` and camelCase variants; never overwrites a populated cell — manual override wins); (c) `configure-env-variables` and `setup-solution` both re-ingest `docs/alm/last-env-vars.json` (when present) so freshly-created definitions appear in `planData.envVars[]` and `plannedEnvVarCount` zeros out; (d) `export-solution` ingests `docs/alm/last-export.json` into `planData.manualMeta.lastExport` (all 10 marker fields: solutionUniqueName/solutionId/previousVersion/version/managed/sourceEnvironmentUrl/zipPath/fileSizeBytes/asyncOperationId/exportedAt) so the Manual-path tab can show the most recent export. Marker absence is a silent step-sync-only no-op (no `manualMeta.lastExport: null` row in the rendered plan); (e) `deploy-pipeline` ingests the `batchValidation` block from `last-deploy.json` into `planData.pipelineMeta.lastDeploy.batchValidation` (totalSolutions/succeeded/failed/pendingApproval/timedOut/elapsedSeconds/perSolutionStageRunIds) so the rendered plan can show the Phase 3.6 parallel-validation outcome distinct from the serial deploy outcome. Explicitly set to `null` for single-solution / legacy v2 deploys so renderers can branch on it; legacy `elapsedSecondsApprox` field name is accepted and normalized to `elapsedSeconds` on ingest. **`ensure-pipelines-host` phase**: host-only update of `planData.hostResolution` from `last-host-check.json` (drops NoHost risks) WITHOUT touching `pipelineMeta` or the `Setup pipeline` step — for when the host was resolved but the pipeline doesn't exist yet. **`--reconcile` mode**: the enforcement backstop — scans the `last-*.json` markers and, for each one newer than `docs/.alm-plan-data.json` (a skipped refresh), applies the mapped phase (`MARKER_TO_PHASE`; `lastPipeline`→setup-pipeline supersedes the host-only phase; `lastEnvVars`→configure-env-variables if `deployment-settings.json` exists else setup-solution) against a single loaded planData, writes once, renders once. Honors `.alm-deferred`, soft no-op when no plan, idempotent. Output `{ ok, reconciled:[phases], rendered }`. Invoked by the PostToolUse hook after every ALM skill (see Hooks). #### PP Pipelines diff --git a/plugins/power-pages/hooks/run-skill-posttool-validation.js b/plugins/power-pages/hooks/run-skill-posttool-validation.js index b53595790..529deeda3 100644 --- a/plugins/power-pages/hooks/run-skill-posttool-validation.js +++ b/plugins/power-pages/hooks/run-skill-posttool-validation.js @@ -1,10 +1,12 @@ #!/usr/bin/env node +const fs = require('fs'); const path = require('path'); const { spawnSync } = require('child_process'); const { getTrackedSkillFromToolInput, getValidatorScript, + isAlmPlanSkill, } = require('../scripts/lib/powerpages-hook-utils'); const DEBUG = process.env.DEBUG === '1' || process.env.DEBUG === 'true'; @@ -36,19 +38,51 @@ process.stdin.on('end', () => { process.exit(0); } + const cwd = input.cwd || process.cwd(); + const validatorScript = getValidatorScript(skillName); if (validatorScript) { const validatorPath = path.join(__dirname, '..', validatorScript); const result = spawnSync(process.execPath, [validatorPath], { input: inputData, encoding: 'utf8', - cwd: input.cwd || process.cwd(), + cwd, }); if (result.stdout) process.stdout.write(result.stdout); if (result.stderr) process.stderr.write(result.stderr); validatorStatus = result.status ?? 0; debug(`[power-pages hook] Validator exited with code ${validatorStatus}\n`); } + + // ALM plan reconcile backstop (auto-heal). The refresh-alm-plan-data.js calls + // in each SKILL.md are advisory — silently dropped on session fragmentation, + // manual execution, or oversight. After ANY ALM plan skill completes, reconcile + // the plan against the marker files: any marker newer than the plan (a skipped + // refresh) is ingested automatically. Best-effort and NON-blocking — it never + // changes the hook's exit code (the validator's status stands). Triggering on + // any ALM skill (not just the marker's writer) catches a skip that surfaces only + // when the NEXT ALM skill runs. Honors .alm-deferred + no-plan inside reconcile. + if (isAlmPlanSkill(skillName) && fs.existsSync(path.join(cwd, 'docs', '.alm-plan-data.json'))) { + try { + const refreshPath = path.join(__dirname, '..', 'scripts', 'lib', 'refresh-alm-plan-data.js'); + const rec = spawnSync(process.execPath, [refreshPath, '--projectRoot', cwd, '--reconcile', '--render'], { + encoding: 'utf8', + cwd, + timeout: 20000, + }); + let reconciled = []; + try { reconciled = (JSON.parse((rec.stdout || '').trim()).reconciled) || []; } catch {} + if (reconciled.length > 0) { + process.stdout.write( + `[power-pages] ALM plan was out of sync with ${reconciled.length} run marker(s) — refreshed automatically (${reconciled.join(', ')}).\n`, + ); + } + debug(`[power-pages hook] reconcile reconciled=${JSON.stringify(reconciled)}\n`); + } catch (e) { + // Best-effort — a reconcile failure must never break the skill or the hook. + debug(`[power-pages hook] reconcile error (ignored): ${e.message}\n`); + } + } } catch (err) { process.stderr.write(`[power-pages hook] Unexpected error: ${err.message}\n`); validatorStatus = 0; diff --git a/plugins/power-pages/scripts/lib/powerpages-hook-utils.js b/plugins/power-pages/scripts/lib/powerpages-hook-utils.js index 9775fe0d5..5d5f98211 100644 --- a/plugins/power-pages/scripts/lib/powerpages-hook-utils.js +++ b/plugins/power-pages/scripts/lib/powerpages-hook-utils.js @@ -105,9 +105,40 @@ function getValidatorScript(skillName) { return TRACKED_SKILLS[skillName]?.validatorScript ?? null; } +// Skills that write a `docs/alm/last-*.json` marker or otherwise consume the ALM +// plan. After any of these completes, the PostToolUse hook runs a plan reconcile +// (auto-heal) — so a refresh step skipped by ONE skill is caught when the NEXT +// ALM skill completes (covers manual/cross-session execution). +const ALM_PLAN_SKILLS = new Set([ + 'setup-solution', + 'setup-pipeline', + 'deploy-pipeline', + 'export-solution', + 'import-solution', + 'configure-env-variables', + 'activate-site', + 'test-site', + 'ensure-pipelines-host', + 'force-link-environment', +]); + +/** + * True when `value` (a raw skill name, `/skill`, or `power-pages:skill`) resolves + * to an ALM plan skill. Normalizes via `detectTrackedSkill`, so it also confirms + * the skill actually exists in this plugin. + * @param {string} value + * @returns {boolean} + */ +function isAlmPlanSkill(value) { + const name = detectTrackedSkill(value); + return name != null && ALM_PLAN_SKILLS.has(name); +} + module.exports = { TRACKED_SKILLS, + ALM_PLAN_SKILLS, detectTrackedSkill, getTrackedSkillFromToolInput, getValidatorScript, + isAlmPlanSkill, }; diff --git a/plugins/power-pages/scripts/tests/powerpages-hook-utils.test.js b/plugins/power-pages/scripts/tests/powerpages-hook-utils.test.js index 65718dde3..e1becdaa3 100644 --- a/plugins/power-pages/scripts/tests/powerpages-hook-utils.test.js +++ b/plugins/power-pages/scripts/tests/powerpages-hook-utils.test.js @@ -5,9 +5,11 @@ const path = require('path'); const { TRACKED_SKILLS, + ALM_PLAN_SKILLS, detectTrackedSkill, getTrackedSkillFromToolInput, getValidatorScript, + isAlmPlanSkill, } = require('../lib/powerpages-hook-utils'); const SKILLS_DIR = path.join(__dirname, '..', '..', 'skills'); @@ -148,3 +150,27 @@ test('Object.prototype keys are not mistaken for tracked skills', () => { assert.equal(detectTrackedSkill('__proto__'), null); assert.equal(getTrackedSkillFromToolInput({ skill: 'toString' }), null); }); + +// --- ALM_PLAN_SKILLS / isAlmPlanSkill (reconcile-trigger gating) ------------- + +test('isAlmPlanSkill: true for ALM plan skills, normalizing prefixes', () => { + assert.equal(isAlmPlanSkill('activate-site'), true); + assert.equal(isAlmPlanSkill('/power-pages:activate-site'), true); + assert.equal(isAlmPlanSkill('/activate-site'), true); + assert.equal(isAlmPlanSkill('ensure-pipelines-host'), true); + assert.equal(isAlmPlanSkill('setup-pipeline'), true); +}); + +test('isAlmPlanSkill: false for non-ALM skills and junk', () => { + assert.equal(isAlmPlanSkill('create-site'), false); + assert.equal(isAlmPlanSkill('add-seo'), false); + assert.equal(isAlmPlanSkill('not-a-real-skill'), false); + assert.equal(isAlmPlanSkill(null), false); + assert.equal(isAlmPlanSkill(undefined), false); +}); + +test('ALM_PLAN_SKILLS members are all real tracked skills', () => { + for (const name of ALM_PLAN_SKILLS) { + assert.ok(TRACKED_SKILLS[name], `ALM_PLAN_SKILLS member "${name}" must be a tracked skill (have a SKILL.md)`); + } +}); diff --git a/plugins/power-pages/skills/ensure-pipelines-host/SKILL.md b/plugins/power-pages/skills/ensure-pipelines-host/SKILL.md index 03866e39c..383325951 100644 --- a/plugins/power-pages/skills/ensure-pipelines-host/SKILL.md +++ b/plugins/power-pages/skills/ensure-pipelines-host/SKILL.md @@ -900,6 +900,17 @@ Record skill usage: Follow the skill tracking instructions in the reference to record this skill's usage. Use `--skillName "EnsurePipelinesHost"`. +**Refresh the ALM plan (if one exists):** + +```bash +node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ + --projectRoot "." \ + --phase ensure-pipelines-host \ + --render +``` + +This updates `planData.hostResolution` from the `docs/alm/last-host-check.json` you just wrote (host-only — no pipeline yet) and drops the pre-run NoHost risks, then re-renders `docs/alm-plan.html`. **Do this here, not just in setup-pipeline Phase 7** — a host install can take 18+ minutes and cross a session boundary, so deferring the refresh risks the plan never reflecting the host. When `docs/.alm-plan-data.json` is absent (standalone, not part of an ALM plan), the helper returns `ok:false` as a soft no-op. The centralized PostToolUse hook also reconciles the plan as a backstop, but refreshing at the source keeps the rendered plan current immediately. + Present summary table: | Field | Value | From e74e4b710179c26535443cf2bbc9f38a53030ff0 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 14:27:50 +0530 Subject: [PATCH 15/38] Surface reconcile failures in the hook + document failed[] contract Follows the Copilot review fix in #191 (reconcile no longer swallows per-phase errors): the PostToolUse hook now reads result.failed alongside reconciled and prints a one-line, non-blocking notice naming the phases that could not heal, so a marker-schema break is diagnosable at the hook level (details still go to the lib's stderr). AGENTS.md reconcile bullet updated to document the failed[] field. 1213 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- plugins/power-pages/AGENTS.md | 2 +- .../hooks/run-skill-posttool-validation.js | 16 ++++++++++++++-- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/plugins/power-pages/AGENTS.md b/plugins/power-pages/AGENTS.md index c0ca71cfd..2cafc9a4d 100644 --- a/plugins/power-pages/AGENTS.md +++ b/plugins/power-pages/AGENTS.md @@ -227,7 +227,7 @@ Shared lib modules live at `scripts/lib/` and are imported by other scripts via - `scripts/lib/link-site-setting-to-env-var.js`: Links an `mspp_sitesetting` record to an `environmentvariabledefinition` via OData PATCH on the v9.0 API (not v9.2). HAR-confirmed: navigation property is `EnvironmentValue@odata.bind`; headers `if-match: *` and `clienthost: Browser` are required (omitting causes 400). Args: `--envUrl`, `--token`, `--siteSettingId`, `--definitionId`, `--schemaName`. Output: `{ ok, verified, siteSettingId, definitionId }`. - `scripts/lib/install-pipelines-app.js`: Installs the Power Platform Pipelines application package on an existing Dataverse env (replaces ensure-pipelines-host Phase 4.B's manual PPAC click-through). Resolution: BAP `applicationPackages` LIST + `/install` POST → 200 sync / 202 + Location poll, with PAC CLI fallback (`pac application install --environment-id ... --application-list msdyn_AppDeploymentAnchor`) on 401/403/5xx. 409 on install POST treated as idempotent (already-installed). Args: `--bapToken`, `--envId`, `--instanceApiUrl` (opt — for verification probe), `--hostToken` (opt), `--no-pac-fallback` (opt), `--correlationId`, `--timeoutSec`, `--apiVersion`, `--bapBase`. Output: `{ status, alreadyInstalled, installPath: 'bap'\|'pac'\|'cached', packageUniqueName, pipelinesSolutionVersion, durationSec, correlationId, pollAttempts, locationHeader, pacFallbackReason }`. - `scripts/lib/discover-env-var-definitions.js`: Enumerates env var definitions matching a publisher prefix and joins each with its bound `mspp_sitesetting` (if any). Used by `plan-alm` Phase 1 Step 10b to populate `planData.envVars[]` with row-level metadata so the rendered plan's Env Variables tab shows schema name, type, default value, and bound site setting per definition (instead of just a count). Args: `--envUrl`, `--publisherPrefix`, `--websiteRecordId`, `--token` (opt). Output: `{ envVars: [{ schemaName, type, defaultValue, siteSetting }], count }`. Degrades gracefully (empty array, exit 0) on auth failure or query errors so the renderer's count-summary fallback can take over. -- `scripts/lib/refresh-alm-plan-data.js`: Updates `docs/.alm-plan-data.json` with post-run state from the marker files written by setup-pipeline / deploy-pipeline / ensure-pipelines-host / test-site / import-solution / activate-site / configure-env-variables / setup-solution / export-solution, then optionally re-renders `docs/alm-plan.html`. Used by plan-alm Phases 6 / 7 / 8 so the rendered Pipelines tab, Validation tab, hostResolution card, env var values matrix, checklist, and risks list reflect actual run state instead of frozen pre-run intent. Args: `--projectRoot`, `--phase` (`setup-solution`/`setup-pipeline`/`configure-env-variables`/`deploy-pipeline`/`export-solution`/`import-solution`/`activate-site`/`test-site`/`ensure-pipelines-host`/`finalize`) **OR `--reconcile`** (mutually exclusive with `--phase`), `--render` (also invoke renderer), `--stageName` (required for `test-site`; preferred for `import-solution`/`activate-site` though both can resolve via marker URL match). Output: `{ ok, phase, dataPath, htmlPath, rendered }`. Returns `ok:false` (soft no-op) when `docs/.alm-plan-data.json` is missing — caller should preserve that file across phases for the helper to work. Plan-alm Phase 3 must NOT delete the file after the initial render — it's read by `check-alm-plan.js` for downstream Phase 0 ALM-plan gates and by this helper for post-run refreshes. **Cross-cutting behaviors**: (a) `setStepStatus` flips the matching entry in `planData.steps[]` to `completed` (or `failed` when the phase's marker indicates failure) — case-insensitive keyword match + stage filter, respects `skip: true`, never regresses completed→pending; (b) `deploy-pipeline` AND `configure-env-variables` both backfill `planData.envVars[i].values{}` from the project root's `deployment-settings.json` so the rendered plan's "Values by Environment" matrix auto-populates (accepts both top-level-stage and nested-`stages` shapes; `SchemaName`/`Value` and camelCase variants; never overwrites a populated cell — manual override wins); (c) `configure-env-variables` and `setup-solution` both re-ingest `docs/alm/last-env-vars.json` (when present) so freshly-created definitions appear in `planData.envVars[]` and `plannedEnvVarCount` zeros out; (d) `export-solution` ingests `docs/alm/last-export.json` into `planData.manualMeta.lastExport` (all 10 marker fields: solutionUniqueName/solutionId/previousVersion/version/managed/sourceEnvironmentUrl/zipPath/fileSizeBytes/asyncOperationId/exportedAt) so the Manual-path tab can show the most recent export. Marker absence is a silent step-sync-only no-op (no `manualMeta.lastExport: null` row in the rendered plan); (e) `deploy-pipeline` ingests the `batchValidation` block from `last-deploy.json` into `planData.pipelineMeta.lastDeploy.batchValidation` (totalSolutions/succeeded/failed/pendingApproval/timedOut/elapsedSeconds/perSolutionStageRunIds) so the rendered plan can show the Phase 3.6 parallel-validation outcome distinct from the serial deploy outcome. Explicitly set to `null` for single-solution / legacy v2 deploys so renderers can branch on it; legacy `elapsedSecondsApprox` field name is accepted and normalized to `elapsedSeconds` on ingest. **`ensure-pipelines-host` phase**: host-only update of `planData.hostResolution` from `last-host-check.json` (drops NoHost risks) WITHOUT touching `pipelineMeta` or the `Setup pipeline` step — for when the host was resolved but the pipeline doesn't exist yet. **`--reconcile` mode**: the enforcement backstop — scans the `last-*.json` markers and, for each one newer than `docs/.alm-plan-data.json` (a skipped refresh), applies the mapped phase (`MARKER_TO_PHASE`; `lastPipeline`→setup-pipeline supersedes the host-only phase; `lastEnvVars`→configure-env-variables if `deployment-settings.json` exists else setup-solution) against a single loaded planData, writes once, renders once. Honors `.alm-deferred`, soft no-op when no plan, idempotent. Output `{ ok, reconciled:[phases], rendered }`. Invoked by the PostToolUse hook after every ALM skill (see Hooks). +- `scripts/lib/refresh-alm-plan-data.js`: Updates `docs/.alm-plan-data.json` with post-run state from the marker files written by setup-pipeline / deploy-pipeline / ensure-pipelines-host / test-site / import-solution / activate-site / configure-env-variables / setup-solution / export-solution, then optionally re-renders `docs/alm-plan.html`. Used by plan-alm Phases 6 / 7 / 8 so the rendered Pipelines tab, Validation tab, hostResolution card, env var values matrix, checklist, and risks list reflect actual run state instead of frozen pre-run intent. Args: `--projectRoot`, `--phase` (`setup-solution`/`setup-pipeline`/`configure-env-variables`/`deploy-pipeline`/`export-solution`/`import-solution`/`activate-site`/`test-site`/`ensure-pipelines-host`/`finalize`) **OR `--reconcile`** (mutually exclusive with `--phase`), `--render` (also invoke renderer), `--stageName` (required for `test-site`; preferred for `import-solution`/`activate-site` though both can resolve via marker URL match). Output: `{ ok, phase, dataPath, htmlPath, rendered }`. Returns `ok:false` (soft no-op) when `docs/.alm-plan-data.json` is missing — caller should preserve that file across phases for the helper to work. Plan-alm Phase 3 must NOT delete the file after the initial render — it's read by `check-alm-plan.js` for downstream Phase 0 ALM-plan gates and by this helper for post-run refreshes. **Cross-cutting behaviors**: (a) `setStepStatus` flips the matching entry in `planData.steps[]` to `completed` (or `failed` when the phase's marker indicates failure) — case-insensitive keyword match + stage filter, respects `skip: true`, never regresses completed→pending; (b) `deploy-pipeline` AND `configure-env-variables` both backfill `planData.envVars[i].values{}` from the project root's `deployment-settings.json` so the rendered plan's "Values by Environment" matrix auto-populates (accepts both top-level-stage and nested-`stages` shapes; `SchemaName`/`Value` and camelCase variants; never overwrites a populated cell — manual override wins); (c) `configure-env-variables` and `setup-solution` both re-ingest `docs/alm/last-env-vars.json` (when present) so freshly-created definitions appear in `planData.envVars[]` and `plannedEnvVarCount` zeros out; (d) `export-solution` ingests `docs/alm/last-export.json` into `planData.manualMeta.lastExport` (all 10 marker fields: solutionUniqueName/solutionId/previousVersion/version/managed/sourceEnvironmentUrl/zipPath/fileSizeBytes/asyncOperationId/exportedAt) so the Manual-path tab can show the most recent export. Marker absence is a silent step-sync-only no-op (no `manualMeta.lastExport: null` row in the rendered plan); (e) `deploy-pipeline` ingests the `batchValidation` block from `last-deploy.json` into `planData.pipelineMeta.lastDeploy.batchValidation` (totalSolutions/succeeded/failed/pendingApproval/timedOut/elapsedSeconds/perSolutionStageRunIds) so the rendered plan can show the Phase 3.6 parallel-validation outcome distinct from the serial deploy outcome. Explicitly set to `null` for single-solution / legacy v2 deploys so renderers can branch on it; legacy `elapsedSecondsApprox` field name is accepted and normalized to `elapsedSeconds` on ingest. **`ensure-pipelines-host` phase**: host-only update of `planData.hostResolution` from `last-host-check.json` (drops NoHost risks) WITHOUT touching `pipelineMeta` or the `Setup pipeline` step — for when the host was resolved but the pipeline doesn't exist yet. **`--reconcile` mode**: the enforcement backstop — scans the `last-*.json` markers and, for each one newer than `docs/.alm-plan-data.json` (a skipped refresh), applies the mapped phase (`MARKER_TO_PHASE`; `lastPipeline`→setup-pipeline supersedes the host-only phase; `lastEnvVars`→configure-env-variables if `deployment-settings.json` exists else setup-solution) against a single loaded planData, writes once, renders once. Honors `.alm-deferred`, soft no-op when no plan, idempotent. Output `{ ok, reconciled:[phases healed], failed:[{phase,error}], rendered }` — a phase whose refresh throws (e.g. a marker schema it can't parse) is captured in `failed` (and written to stderr) instead of being silently swallowed, while the remaining phases still heal. Invoked by the PostToolUse hook after every ALM skill (see Hooks). #### PP Pipelines diff --git a/plugins/power-pages/hooks/run-skill-posttool-validation.js b/plugins/power-pages/hooks/run-skill-posttool-validation.js index 529deeda3..ee1bace99 100644 --- a/plugins/power-pages/hooks/run-skill-posttool-validation.js +++ b/plugins/power-pages/hooks/run-skill-posttool-validation.js @@ -71,13 +71,25 @@ process.stdin.on('end', () => { timeout: 20000, }); let reconciled = []; - try { reconciled = (JSON.parse((rec.stdout || '').trim()).reconciled) || []; } catch {} + let failed = []; + try { + const out = JSON.parse((rec.stdout || '').trim()); + reconciled = out.reconciled || []; + failed = out.failed || []; + } catch {} if (reconciled.length > 0) { process.stdout.write( `[power-pages] ALM plan was out of sync with ${reconciled.length} run marker(s) — refreshed automatically (${reconciled.join(', ')}).\n`, ); } - debug(`[power-pages hook] reconcile reconciled=${JSON.stringify(reconciled)}\n`); + if (failed.length > 0) { + // Non-blocking, but surfaced — a swallowed reconcile failure is exactly + // what makes a stale plan impossible to diagnose. + process.stdout.write( + `[power-pages] ALM plan reconcile could not heal ${failed.length} phase(s): ${failed.map((f) => f.phase).join(', ')}. See stderr for details.\n`, + ); + } + debug(`[power-pages hook] reconcile reconciled=${JSON.stringify(reconciled)} failed=${JSON.stringify(failed)}\n`); } catch (e) { // Best-effort — a reconcile failure must never break the skill or the hook. debug(`[power-pages hook] reconcile error (ignored): ${e.message}\n`); From a5e20e70bd5453a28cbb33579a81fc3b97729b99 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 18:17:20 +0530 Subject: [PATCH 16/38] Fix stale "plan-alm Phases 6/7/8" reference in AGENTS.md Regression pass finding: the refresh-alm-plan-data.js doc entry still said it was "Used by plan-alm Phases 6 / 7 / 8", but those phases were deleted when plan-alm was converted to a plan-only 4-phase planner. The helper is now driven by the execution skills' final-phase refresh + the PostToolUse --reconcile backstop, not by plan-alm. Corrected the clause (CLAUDE.md is a symlink to AGENTS.md, so both are fixed in one edit). 1218 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- plugins/power-pages/AGENTS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/plugins/power-pages/AGENTS.md b/plugins/power-pages/AGENTS.md index 2cafc9a4d..528c87aca 100644 --- a/plugins/power-pages/AGENTS.md +++ b/plugins/power-pages/AGENTS.md @@ -227,7 +227,7 @@ Shared lib modules live at `scripts/lib/` and are imported by other scripts via - `scripts/lib/link-site-setting-to-env-var.js`: Links an `mspp_sitesetting` record to an `environmentvariabledefinition` via OData PATCH on the v9.0 API (not v9.2). HAR-confirmed: navigation property is `EnvironmentValue@odata.bind`; headers `if-match: *` and `clienthost: Browser` are required (omitting causes 400). Args: `--envUrl`, `--token`, `--siteSettingId`, `--definitionId`, `--schemaName`. Output: `{ ok, verified, siteSettingId, definitionId }`. - `scripts/lib/install-pipelines-app.js`: Installs the Power Platform Pipelines application package on an existing Dataverse env (replaces ensure-pipelines-host Phase 4.B's manual PPAC click-through). Resolution: BAP `applicationPackages` LIST + `/install` POST → 200 sync / 202 + Location poll, with PAC CLI fallback (`pac application install --environment-id ... --application-list msdyn_AppDeploymentAnchor`) on 401/403/5xx. 409 on install POST treated as idempotent (already-installed). Args: `--bapToken`, `--envId`, `--instanceApiUrl` (opt — for verification probe), `--hostToken` (opt), `--no-pac-fallback` (opt), `--correlationId`, `--timeoutSec`, `--apiVersion`, `--bapBase`. Output: `{ status, alreadyInstalled, installPath: 'bap'\|'pac'\|'cached', packageUniqueName, pipelinesSolutionVersion, durationSec, correlationId, pollAttempts, locationHeader, pacFallbackReason }`. - `scripts/lib/discover-env-var-definitions.js`: Enumerates env var definitions matching a publisher prefix and joins each with its bound `mspp_sitesetting` (if any). Used by `plan-alm` Phase 1 Step 10b to populate `planData.envVars[]` with row-level metadata so the rendered plan's Env Variables tab shows schema name, type, default value, and bound site setting per definition (instead of just a count). Args: `--envUrl`, `--publisherPrefix`, `--websiteRecordId`, `--token` (opt). Output: `{ envVars: [{ schemaName, type, defaultValue, siteSetting }], count }`. Degrades gracefully (empty array, exit 0) on auth failure or query errors so the renderer's count-summary fallback can take over. -- `scripts/lib/refresh-alm-plan-data.js`: Updates `docs/.alm-plan-data.json` with post-run state from the marker files written by setup-pipeline / deploy-pipeline / ensure-pipelines-host / test-site / import-solution / activate-site / configure-env-variables / setup-solution / export-solution, then optionally re-renders `docs/alm-plan.html`. Used by plan-alm Phases 6 / 7 / 8 so the rendered Pipelines tab, Validation tab, hostResolution card, env var values matrix, checklist, and risks list reflect actual run state instead of frozen pre-run intent. Args: `--projectRoot`, `--phase` (`setup-solution`/`setup-pipeline`/`configure-env-variables`/`deploy-pipeline`/`export-solution`/`import-solution`/`activate-site`/`test-site`/`ensure-pipelines-host`/`finalize`) **OR `--reconcile`** (mutually exclusive with `--phase`), `--render` (also invoke renderer), `--stageName` (required for `test-site`; preferred for `import-solution`/`activate-site` though both can resolve via marker URL match). Output: `{ ok, phase, dataPath, htmlPath, rendered }`. Returns `ok:false` (soft no-op) when `docs/.alm-plan-data.json` is missing — caller should preserve that file across phases for the helper to work. Plan-alm Phase 3 must NOT delete the file after the initial render — it's read by `check-alm-plan.js` for downstream Phase 0 ALM-plan gates and by this helper for post-run refreshes. **Cross-cutting behaviors**: (a) `setStepStatus` flips the matching entry in `planData.steps[]` to `completed` (or `failed` when the phase's marker indicates failure) — case-insensitive keyword match + stage filter, respects `skip: true`, never regresses completed→pending; (b) `deploy-pipeline` AND `configure-env-variables` both backfill `planData.envVars[i].values{}` from the project root's `deployment-settings.json` so the rendered plan's "Values by Environment" matrix auto-populates (accepts both top-level-stage and nested-`stages` shapes; `SchemaName`/`Value` and camelCase variants; never overwrites a populated cell — manual override wins); (c) `configure-env-variables` and `setup-solution` both re-ingest `docs/alm/last-env-vars.json` (when present) so freshly-created definitions appear in `planData.envVars[]` and `plannedEnvVarCount` zeros out; (d) `export-solution` ingests `docs/alm/last-export.json` into `planData.manualMeta.lastExport` (all 10 marker fields: solutionUniqueName/solutionId/previousVersion/version/managed/sourceEnvironmentUrl/zipPath/fileSizeBytes/asyncOperationId/exportedAt) so the Manual-path tab can show the most recent export. Marker absence is a silent step-sync-only no-op (no `manualMeta.lastExport: null` row in the rendered plan); (e) `deploy-pipeline` ingests the `batchValidation` block from `last-deploy.json` into `planData.pipelineMeta.lastDeploy.batchValidation` (totalSolutions/succeeded/failed/pendingApproval/timedOut/elapsedSeconds/perSolutionStageRunIds) so the rendered plan can show the Phase 3.6 parallel-validation outcome distinct from the serial deploy outcome. Explicitly set to `null` for single-solution / legacy v2 deploys so renderers can branch on it; legacy `elapsedSecondsApprox` field name is accepted and normalized to `elapsedSeconds` on ingest. **`ensure-pipelines-host` phase**: host-only update of `planData.hostResolution` from `last-host-check.json` (drops NoHost risks) WITHOUT touching `pipelineMeta` or the `Setup pipeline` step — for when the host was resolved but the pipeline doesn't exist yet. **`--reconcile` mode**: the enforcement backstop — scans the `last-*.json` markers and, for each one newer than `docs/.alm-plan-data.json` (a skipped refresh), applies the mapped phase (`MARKER_TO_PHASE`; `lastPipeline`→setup-pipeline supersedes the host-only phase; `lastEnvVars`→configure-env-variables if `deployment-settings.json` exists else setup-solution) against a single loaded planData, writes once, renders once. Honors `.alm-deferred`, soft no-op when no plan, idempotent. Output `{ ok, reconciled:[phases healed], failed:[{phase,error}], rendered }` — a phase whose refresh throws (e.g. a marker schema it can't parse) is captured in `failed` (and written to stderr) instead of being silently swallowed, while the remaining phases still heal. Invoked by the PostToolUse hook after every ALM skill (see Hooks). +- `scripts/lib/refresh-alm-plan-data.js`: Updates `docs/.alm-plan-data.json` with post-run state from the marker files written by setup-pipeline / deploy-pipeline / ensure-pipelines-host / test-site / import-solution / activate-site / configure-env-variables / setup-solution / export-solution, then optionally re-renders `docs/alm-plan.html`. Driven by the execution skills' final-phase refresh (and the PostToolUse `--reconcile` backstop) — NOT by plan-alm, which is now a plan-only 4-phase planner that only renders the initial plan in Phase 3 — so the rendered Pipelines tab, Validation tab, hostResolution card, env var values matrix, checklist, and risks list reflect actual run state instead of frozen pre-run intent. Args: `--projectRoot`, `--phase` (`setup-solution`/`setup-pipeline`/`configure-env-variables`/`deploy-pipeline`/`export-solution`/`import-solution`/`activate-site`/`test-site`/`ensure-pipelines-host`/`finalize`) **OR `--reconcile`** (mutually exclusive with `--phase`), `--render` (also invoke renderer), `--stageName` (required for `test-site`; preferred for `import-solution`/`activate-site` though both can resolve via marker URL match). Output: `{ ok, phase, dataPath, htmlPath, rendered }`. Returns `ok:false` (soft no-op) when `docs/.alm-plan-data.json` is missing — caller should preserve that file across phases for the helper to work. Plan-alm Phase 3 must NOT delete the file after the initial render — it's read by `check-alm-plan.js` for downstream Phase 0 ALM-plan gates and by this helper for post-run refreshes. **Cross-cutting behaviors**: (a) `setStepStatus` flips the matching entry in `planData.steps[]` to `completed` (or `failed` when the phase's marker indicates failure) — case-insensitive keyword match + stage filter, respects `skip: true`, never regresses completed→pending; (b) `deploy-pipeline` AND `configure-env-variables` both backfill `planData.envVars[i].values{}` from the project root's `deployment-settings.json` so the rendered plan's "Values by Environment" matrix auto-populates (accepts both top-level-stage and nested-`stages` shapes; `SchemaName`/`Value` and camelCase variants; never overwrites a populated cell — manual override wins); (c) `configure-env-variables` and `setup-solution` both re-ingest `docs/alm/last-env-vars.json` (when present) so freshly-created definitions appear in `planData.envVars[]` and `plannedEnvVarCount` zeros out; (d) `export-solution` ingests `docs/alm/last-export.json` into `planData.manualMeta.lastExport` (all 10 marker fields: solutionUniqueName/solutionId/previousVersion/version/managed/sourceEnvironmentUrl/zipPath/fileSizeBytes/asyncOperationId/exportedAt) so the Manual-path tab can show the most recent export. Marker absence is a silent step-sync-only no-op (no `manualMeta.lastExport: null` row in the rendered plan); (e) `deploy-pipeline` ingests the `batchValidation` block from `last-deploy.json` into `planData.pipelineMeta.lastDeploy.batchValidation` (totalSolutions/succeeded/failed/pendingApproval/timedOut/elapsedSeconds/perSolutionStageRunIds) so the rendered plan can show the Phase 3.6 parallel-validation outcome distinct from the serial deploy outcome. Explicitly set to `null` for single-solution / legacy v2 deploys so renderers can branch on it; legacy `elapsedSecondsApprox` field name is accepted and normalized to `elapsedSeconds` on ingest. **`ensure-pipelines-host` phase**: host-only update of `planData.hostResolution` from `last-host-check.json` (drops NoHost risks) WITHOUT touching `pipelineMeta` or the `Setup pipeline` step — for when the host was resolved but the pipeline doesn't exist yet. **`--reconcile` mode**: the enforcement backstop — scans the `last-*.json` markers and, for each one newer than `docs/.alm-plan-data.json` (a skipped refresh), applies the mapped phase (`MARKER_TO_PHASE`; `lastPipeline`→setup-pipeline supersedes the host-only phase; `lastEnvVars`→configure-env-variables if `deployment-settings.json` exists else setup-solution) against a single loaded planData, writes once, renders once. Honors `.alm-deferred`, soft no-op when no plan, idempotent. Output `{ ok, reconciled:[phases healed], failed:[{phase,error}], rendered }` — a phase whose refresh throws (e.g. a marker schema it can't parse) is captured in `failed` (and written to stderr) instead of being silently swallowed, while the remaining phases still heal. Invoked by the PostToolUse hook after every ALM skill (see Hooks). #### PP Pipelines From dfb672ac20c0c417eabee9a7ccb5a7fb3ea4a1d7 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 21:38:10 +0530 Subject: [PATCH 17/38] Activate the PLAN_STATUS lifecycle: Approved -> In Execution -> Completed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The plan had an elaborate heartbeat/active-chain lifecycle (check-alm-plan.js: Draft|Approved|In Execution|Completed + 60-min LAST_INVOCATION_AT heartbeat, stale-heartbeat reclassification) that was entirely DEAD: nothing ever set "In Execution", and refreshFinalize ("Completed") was never called. So every plan was stuck at "Approved" and the heartbeat machinery never engaged. Gap 1 — Approved -> In Execution (check-alm-plan.js): plan-alm is plan-only and leaves the plan "Approved"; the FIRST execution skill's Phase 0 call now promotes it to "In Execution" and writes the first heartbeat (one atomic write). Gated on writeHeartbeat, so read-only callers don't mutate the plan — and plan-alm's own deferral check now passes --no-heartbeat (a planner re-run isn't execution and must not promote). Gap 5 — In Execution -> Completed (refresh-alm-plan-data.js): a completion evaluator runs after every phase's step-sync (refresh + reconcile) and flips PLAN_STATUS to "Completed" + stamps COMPLETED_AT once every non-skip step is completed and none failed. The last execution skill terminates the plan automatically — no skill needs to call --phase finalize. A failed step blocks completion so a failed deploy can't look "done". Gaps 2/3/4 (ensure-pipelines-host / setup-pipeline / activate-site refresh calls) were already in place + covered by the reconcile backstop — no change needed. +8 tests (3 promotion in check-alm-plan, 5 completion in refresh); 1227 pass, alm-lint 0 findings. AGENTS.md documents both transitions. Co-Authored-By: Claude Opus 4.8 (1M context) --- plugins/power-pages/AGENTS.md | 4 +- .../power-pages/scripts/lib/check-alm-plan.js | 17 +++- .../scripts/lib/refresh-alm-plan-data.js | 36 +++++++ .../scripts/tests/check-alm-plan.test.js | 44 ++++++++- .../tests/refresh-alm-plan-data.test.js | 94 +++++++++++++++++++ plugins/power-pages/skills/plan-alm/SKILL.md | 4 +- 6 files changed, 194 insertions(+), 5 deletions(-) diff --git a/plugins/power-pages/AGENTS.md b/plugins/power-pages/AGENTS.md index 528c87aca..f2860ede7 100644 --- a/plugins/power-pages/AGENTS.md +++ b/plugins/power-pages/AGENTS.md @@ -200,7 +200,7 @@ Shared lib modules live at `scripts/lib/` and are imported by other scripts via - `scripts/lib/verify-alm-prerequisites.js`: Verifies all prerequisites for ALM skills — PAC CLI installed + authenticated (`pac env who`), Azure CLI installed + logged in, Dataverse API reachable (`WhoAmI`). Args: `--envUrl` (opt, overrides env from PAC CLI), `--require-manifest` (fails if `.solution-manifest.json` not found). Output: `{ envUrl, token, userId, organizationId, tenantId }`. Exit 0 on success, exit 1 on any failure. Used by `setup-solution`, `export-solution`, `import-solution`, `setup-pipeline`, `deploy-pipeline`, `plan-alm`. - `scripts/lib/detect-project-context.js`: Reads Power Pages project context from the project root. Resolves site identity in order: (1) `powerpages.config.json` → `siteType: "code"` (SPA sites); (2) `.powerpages-site/website.yml` → `siteType: "data-model"` (standard/enhanced data-model "EDM" sites, which have **no** `powerpages.config.json` — `id`→`websiteRecordId`, `name`→`siteName`, `environmentUrl: null` since the local files carry no env URL). Also reads `.solution-manifest.json` and `.datamodel-manifest.json`. Args: `--projectRoot` (opt, auto-discovered from cwd if omitted). Output: `{ projectRoot, siteType, siteName, websiteRecordId, environmentUrl, solutionManifest, datamodelManifest }`. Exit 0 on success, exit 1 only if **neither** `powerpages.config.json` nor `.powerpages-site/website.yml` is found. Note: `findProjectRoot` (in `validation-helpers.js`) likewise treats a `.powerpages-site/` directory as a project-root marker, not just `powerpages.config.json`, so data-model sites are discoverable. - `scripts/lib/alm-paths.js`: Single source of truth for ALM artifact paths. Exports `ALM_DIR` (always `docs/alm`), `FILE_NAMES` (frozen object mapping logical key → filename for all 14 ALM artifacts), `almDir(projectRoot) → path`, `almPath(projectRoot, key) → path`, `ensureAlmDir(projectRoot) → path` (mkdir -p idempotent). Every ALM-only state file (5 plan/decision JSONs + 9 `last-*.json` skill-run markers including `last-export.json`) writes under `/docs/alm/`. **Always resolve through this helper** — never inline a raw `docs/alm/...` path in a script. Files intentionally NOT moved here (and not in `FILE_NAMES`): `.solution-manifest.json`, `.datamodel-manifest.json`, `.alm-config.json`, `.alm-deferred`, `deployment-settings.json`. Adding a new ALM marker means adding its key + filename to `FILE_NAMES` first; `almPath` throws on unknown keys to catch typos at call-site. -- `scripts/lib/check-alm-plan.js`: Phase 0 gate helper used by every ALM skill to detect (a) whether an ALM plan exists for this project, (b) whether the user has explicitly deferred ALM via the `.alm-deferred` marker, and (c) whether an existing plan is stale (the source solution was modified after the plan was generated). Args: `--projectRoot`, `--envUrl` (opt — required for staleness check), `--token` (opt), `--solutionId` (opt — required for staleness check). Output: `{ exists, deferred, deferral, planPath, htmlPath, stale, staleness: { reason, detail }, generatedAt, planStatus, solution: {...} }`. Without env/solution context the helper does an existence-only check; with them it queries Dataverse for `solutions(solutionId)?$select=modifiedon` and compares against `planData.generatedAt`. Used by `setup-solution`, `setup-pipeline`, `deploy-pipeline`, `export-solution`, `import-solution`, `configure-env-variables`, `ensure-pipelines-host`, `force-link-environment` Phase 0 gates — the "fail closed when no plan" pattern. +- `scripts/lib/check-alm-plan.js`: Phase 0 gate helper used by every ALM skill to detect (a) whether an ALM plan exists for this project, (b) whether the user has explicitly deferred ALM via the `.alm-deferred` marker, and (c) whether an existing plan is stale (the source solution was modified after the plan was generated). Args: `--projectRoot`, `--envUrl` (opt — required for staleness check), `--token` (opt), `--solutionId` (opt — required for staleness check). Output: `{ exists, deferred, deferral, planPath, htmlPath, stale, staleness: { reason, detail }, generatedAt, planStatus, solution: {...} }`. Without env/solution context the helper does an existence-only check; with them it queries Dataverse for `solutions(solutionId)?$select=modifiedon` and compares against `planData.generatedAt`. Used by `setup-solution`, `setup-pipeline`, `deploy-pipeline`, `export-solution`, `import-solution`, `configure-env-variables`, `ensure-pipelines-host`, `force-link-environment` Phase 0 gates — the "fail closed when no plan" pattern. **PLAN_STATUS lifecycle — promotes `Approved` → `In Execution`:** plan-alm is plan-only and leaves the plan `Approved`; this helper performs the `Approved` → `In Execution` transition (and writes the first heartbeat) the **first time an execution skill's Phase 0 runs** — it is the only thing that sets `In Execution`, so without it the heartbeat/active-chain machinery (multi-hour-deploy `stale-heartbeat` reclassification) never engages. Gated on heartbeat-write: read-only callers pass `--no-heartbeat` (plan-alm's own deferral check, audits, tests) and are never promoted. The terminal `In Execution` → `Completed` transition is owned by `refresh-alm-plan-data.js` (completion evaluator). - `scripts/lib/resolve-target-solution.js`: Resolves "which solution should this new Dataverse record land in?" Implements the strict 3-step order from the ALM-aware-by-default principle: (1) explicit `--solutionUniqueName` (or equivalent caller arg) wins; (2) `.solution-manifest.json` in the project root; (3) neither → throw `NoSolutionConfiguredError`. **The module NEVER auto-picks from Dataverse** — interactive prompt UX is the caller's responsibility (catch the error, present an `AskUserQuestion` list, re-invoke with `explicit` populated). Callers that need to confirm the solution still exists in Dataverse can pass `verifyExists: true`; the module then enriches the result with `{ solutionId, version, ismanaged }`. Component-creation scripts must require this helper and pass through `--solutionUniqueName` so records land in the user's solution instead of `Default`. #### Solution Splitting Decision Tree (v1.3.0+) @@ -227,7 +227,7 @@ Shared lib modules live at `scripts/lib/` and are imported by other scripts via - `scripts/lib/link-site-setting-to-env-var.js`: Links an `mspp_sitesetting` record to an `environmentvariabledefinition` via OData PATCH on the v9.0 API (not v9.2). HAR-confirmed: navigation property is `EnvironmentValue@odata.bind`; headers `if-match: *` and `clienthost: Browser` are required (omitting causes 400). Args: `--envUrl`, `--token`, `--siteSettingId`, `--definitionId`, `--schemaName`. Output: `{ ok, verified, siteSettingId, definitionId }`. - `scripts/lib/install-pipelines-app.js`: Installs the Power Platform Pipelines application package on an existing Dataverse env (replaces ensure-pipelines-host Phase 4.B's manual PPAC click-through). Resolution: BAP `applicationPackages` LIST + `/install` POST → 200 sync / 202 + Location poll, with PAC CLI fallback (`pac application install --environment-id ... --application-list msdyn_AppDeploymentAnchor`) on 401/403/5xx. 409 on install POST treated as idempotent (already-installed). Args: `--bapToken`, `--envId`, `--instanceApiUrl` (opt — for verification probe), `--hostToken` (opt), `--no-pac-fallback` (opt), `--correlationId`, `--timeoutSec`, `--apiVersion`, `--bapBase`. Output: `{ status, alreadyInstalled, installPath: 'bap'\|'pac'\|'cached', packageUniqueName, pipelinesSolutionVersion, durationSec, correlationId, pollAttempts, locationHeader, pacFallbackReason }`. - `scripts/lib/discover-env-var-definitions.js`: Enumerates env var definitions matching a publisher prefix and joins each with its bound `mspp_sitesetting` (if any). Used by `plan-alm` Phase 1 Step 10b to populate `planData.envVars[]` with row-level metadata so the rendered plan's Env Variables tab shows schema name, type, default value, and bound site setting per definition (instead of just a count). Args: `--envUrl`, `--publisherPrefix`, `--websiteRecordId`, `--token` (opt). Output: `{ envVars: [{ schemaName, type, defaultValue, siteSetting }], count }`. Degrades gracefully (empty array, exit 0) on auth failure or query errors so the renderer's count-summary fallback can take over. -- `scripts/lib/refresh-alm-plan-data.js`: Updates `docs/.alm-plan-data.json` with post-run state from the marker files written by setup-pipeline / deploy-pipeline / ensure-pipelines-host / test-site / import-solution / activate-site / configure-env-variables / setup-solution / export-solution, then optionally re-renders `docs/alm-plan.html`. Driven by the execution skills' final-phase refresh (and the PostToolUse `--reconcile` backstop) — NOT by plan-alm, which is now a plan-only 4-phase planner that only renders the initial plan in Phase 3 — so the rendered Pipelines tab, Validation tab, hostResolution card, env var values matrix, checklist, and risks list reflect actual run state instead of frozen pre-run intent. Args: `--projectRoot`, `--phase` (`setup-solution`/`setup-pipeline`/`configure-env-variables`/`deploy-pipeline`/`export-solution`/`import-solution`/`activate-site`/`test-site`/`ensure-pipelines-host`/`finalize`) **OR `--reconcile`** (mutually exclusive with `--phase`), `--render` (also invoke renderer), `--stageName` (required for `test-site`; preferred for `import-solution`/`activate-site` though both can resolve via marker URL match). Output: `{ ok, phase, dataPath, htmlPath, rendered }`. Returns `ok:false` (soft no-op) when `docs/.alm-plan-data.json` is missing — caller should preserve that file across phases for the helper to work. Plan-alm Phase 3 must NOT delete the file after the initial render — it's read by `check-alm-plan.js` for downstream Phase 0 ALM-plan gates and by this helper for post-run refreshes. **Cross-cutting behaviors**: (a) `setStepStatus` flips the matching entry in `planData.steps[]` to `completed` (or `failed` when the phase's marker indicates failure) — case-insensitive keyword match + stage filter, respects `skip: true`, never regresses completed→pending; (b) `deploy-pipeline` AND `configure-env-variables` both backfill `planData.envVars[i].values{}` from the project root's `deployment-settings.json` so the rendered plan's "Values by Environment" matrix auto-populates (accepts both top-level-stage and nested-`stages` shapes; `SchemaName`/`Value` and camelCase variants; never overwrites a populated cell — manual override wins); (c) `configure-env-variables` and `setup-solution` both re-ingest `docs/alm/last-env-vars.json` (when present) so freshly-created definitions appear in `planData.envVars[]` and `plannedEnvVarCount` zeros out; (d) `export-solution` ingests `docs/alm/last-export.json` into `planData.manualMeta.lastExport` (all 10 marker fields: solutionUniqueName/solutionId/previousVersion/version/managed/sourceEnvironmentUrl/zipPath/fileSizeBytes/asyncOperationId/exportedAt) so the Manual-path tab can show the most recent export. Marker absence is a silent step-sync-only no-op (no `manualMeta.lastExport: null` row in the rendered plan); (e) `deploy-pipeline` ingests the `batchValidation` block from `last-deploy.json` into `planData.pipelineMeta.lastDeploy.batchValidation` (totalSolutions/succeeded/failed/pendingApproval/timedOut/elapsedSeconds/perSolutionStageRunIds) so the rendered plan can show the Phase 3.6 parallel-validation outcome distinct from the serial deploy outcome. Explicitly set to `null` for single-solution / legacy v2 deploys so renderers can branch on it; legacy `elapsedSecondsApprox` field name is accepted and normalized to `elapsedSeconds` on ingest. **`ensure-pipelines-host` phase**: host-only update of `planData.hostResolution` from `last-host-check.json` (drops NoHost risks) WITHOUT touching `pipelineMeta` or the `Setup pipeline` step — for when the host was resolved but the pipeline doesn't exist yet. **`--reconcile` mode**: the enforcement backstop — scans the `last-*.json` markers and, for each one newer than `docs/.alm-plan-data.json` (a skipped refresh), applies the mapped phase (`MARKER_TO_PHASE`; `lastPipeline`→setup-pipeline supersedes the host-only phase; `lastEnvVars`→configure-env-variables if `deployment-settings.json` exists else setup-solution) against a single loaded planData, writes once, renders once. Honors `.alm-deferred`, soft no-op when no plan, idempotent. Output `{ ok, reconciled:[phases healed], failed:[{phase,error}], rendered }` — a phase whose refresh throws (e.g. a marker schema it can't parse) is captured in `failed` (and written to stderr) instead of being silently swallowed, while the remaining phases still heal. Invoked by the PostToolUse hook after every ALM skill (see Hooks). +- `scripts/lib/refresh-alm-plan-data.js`: Updates `docs/.alm-plan-data.json` with post-run state from the marker files written by setup-pipeline / deploy-pipeline / ensure-pipelines-host / test-site / import-solution / activate-site / configure-env-variables / setup-solution / export-solution, then optionally re-renders `docs/alm-plan.html`. Driven by the execution skills' final-phase refresh (and the PostToolUse `--reconcile` backstop) — NOT by plan-alm, which is now a plan-only 4-phase planner that only renders the initial plan in Phase 3 — so the rendered Pipelines tab, Validation tab, hostResolution card, env var values matrix, checklist, and risks list reflect actual run state instead of frozen pre-run intent. Args: `--projectRoot`, `--phase` (`setup-solution`/`setup-pipeline`/`configure-env-variables`/`deploy-pipeline`/`export-solution`/`import-solution`/`activate-site`/`test-site`/`ensure-pipelines-host`/`finalize`) **OR `--reconcile`** (mutually exclusive with `--phase`), `--render` (also invoke renderer), `--stageName` (required for `test-site`; preferred for `import-solution`/`activate-site` though both can resolve via marker URL match). Output: `{ ok, phase, dataPath, htmlPath, rendered }`. Returns `ok:false` (soft no-op) when `docs/.alm-plan-data.json` is missing — caller should preserve that file across phases for the helper to work. Plan-alm Phase 3 must NOT delete the file after the initial render — it's read by `check-alm-plan.js` for downstream Phase 0 ALM-plan gates and by this helper for post-run refreshes. **Cross-cutting behaviors**: (a) `setStepStatus` flips the matching entry in `planData.steps[]` to `completed` (or `failed` when the phase's marker indicates failure) — case-insensitive keyword match + stage filter, respects `skip: true`, never regresses completed→pending; (b) `deploy-pipeline` AND `configure-env-variables` both backfill `planData.envVars[i].values{}` from the project root's `deployment-settings.json` so the rendered plan's "Values by Environment" matrix auto-populates (accepts both top-level-stage and nested-`stages` shapes; `SchemaName`/`Value` and camelCase variants; never overwrites a populated cell — manual override wins); (c) `configure-env-variables` and `setup-solution` both re-ingest `docs/alm/last-env-vars.json` (when present) so freshly-created definitions appear in `planData.envVars[]` and `plannedEnvVarCount` zeros out; (d) `export-solution` ingests `docs/alm/last-export.json` into `planData.manualMeta.lastExport` (all 10 marker fields: solutionUniqueName/solutionId/previousVersion/version/managed/sourceEnvironmentUrl/zipPath/fileSizeBytes/asyncOperationId/exportedAt) so the Manual-path tab can show the most recent export. Marker absence is a silent step-sync-only no-op (no `manualMeta.lastExport: null` row in the rendered plan); (e) `deploy-pipeline` ingests the `batchValidation` block from `last-deploy.json` into `planData.pipelineMeta.lastDeploy.batchValidation` (totalSolutions/succeeded/failed/pendingApproval/timedOut/elapsedSeconds/perSolutionStageRunIds) so the rendered plan can show the Phase 3.6 parallel-validation outcome distinct from the serial deploy outcome. Explicitly set to `null` for single-solution / legacy v2 deploys so renderers can branch on it; legacy `elapsedSecondsApprox` field name is accepted and normalized to `elapsedSeconds` on ingest. **`ensure-pipelines-host` phase**: host-only update of `planData.hostResolution` from `last-host-check.json` (drops NoHost risks) WITHOUT touching `pipelineMeta` or the `Setup pipeline` step — for when the host was resolved but the pipeline doesn't exist yet. **`--reconcile` mode**: the enforcement backstop — scans the `last-*.json` markers and, for each one newer than `docs/.alm-plan-data.json` (a skipped refresh), applies the mapped phase (`MARKER_TO_PHASE`; `lastPipeline`→setup-pipeline supersedes the host-only phase; `lastEnvVars`→configure-env-variables if `deployment-settings.json` exists else setup-solution) against a single loaded planData, writes once, renders once. Honors `.alm-deferred`, soft no-op when no plan, idempotent. Output `{ ok, reconciled:[phases healed], failed:[{phase,error}], rendered }` — a phase whose refresh throws (e.g. a marker schema it can't parse) is captured in `failed` (and written to stderr) instead of being silently swallowed, while the remaining phases still heal. Invoked by the PostToolUse hook after every ALM skill (see Hooks). **Completion evaluator (`In Execution` → `Completed`):** after every phase's step-sync (both `refresh()` and `reconcile()`), `evaluatePlanCompletion` flips `PLAN_STATUS` to `Completed` + stamps `COMPLETED_AT` once every non-`skip` step is `completed` and none is `failed`. This is what makes the LAST execution skill terminate the plan automatically — no skill calls `--phase finalize` (the explicit `finalize` phase / `refreshFinalize` exists but nothing invoked it, so the lifecycle previously never reached `Completed`). Only advances from `In Execution` (the normal post-promotion state — see `check-alm-plan.js`) or `Approved` (defensive fallback); never regresses a `Draft` or already-`Completed` plan, and a `failed` step blocks completion so a failed deploy can't look "done". #### PP Pipelines diff --git a/plugins/power-pages/scripts/lib/check-alm-plan.js b/plugins/power-pages/scripts/lib/check-alm-plan.js index f599b2e94..57f11524e 100644 --- a/plugins/power-pages/scripts/lib/check-alm-plan.js +++ b/plugins/power-pages/scripts/lib/check-alm-plan.js @@ -233,7 +233,22 @@ async function checkAlmPlan({ projectRoot, envUrl, token, solutionId, makeReques // way each in-chain skill's Phase 0 call both observes the prior heartbeat // (for its own decision) AND keeps the chain alive for the next skill. const priorLastInvocationAt = planData.LAST_INVOCATION_AT || null; - const planStatus = planData.PLAN_STATUS || null; + let planStatus = planData.PLAN_STATUS || null; + + // Promote Approved -> In Execution on the FIRST execution-skill Phase 0 entry. + // plan-alm is plan-only: it leaves the plan "Approved" and the user runs the + // execution skills themselves. The first execution skill to reach its Phase 0 + // gate is what actually starts execution, so flip the plan to "In Execution" + // here — this is the transition that activates the heartbeat/active-chain + // machinery (nothing else sets it). The heartbeat write below then persists + // both the new status AND the first heartbeat in one atomic write. Gated on + // `writeHeartbeat` so read-only callers (--no-heartbeat: audits, tests, and + // plan-alm's own deferral check) never mutate the plan's status. + if (writeHeartbeat && planStatus === 'Approved') { + planStatus = 'In Execution'; + planData.PLAN_STATUS = 'In Execution'; + } + const inExecution = computeInExecution(planStatus, priorLastInvocationAt, nowMs); if (writeHeartbeat && planStatus === 'In Execution') { diff --git a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js index 564aa925f..364625520 100644 --- a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js +++ b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js @@ -36,6 +36,13 @@ // finalize: // - PLAN_STATUS = "Completed" // +// After every phase's step-sync, a completion evaluator flips PLAN_STATUS to +// "Completed" (+ COMPLETED_AT) once all non-skip steps are completed and none +// failed — so the last execution skill terminates the plan automatically, +// without any skill needing to call the explicit "finalize" phase. The +// Approved -> In Execution promotion that starts the lifecycle lives in +// check-alm-plan.js (first execution-skill Phase 0). +// // stdout JSON includes `nextStep: { name, skill: string | null } | null` (when ok:true) — the first // still-pending checklist step and the slash command that runs it. Execution // skills echo this so the user knows the next step to invoke (user-driven @@ -527,6 +534,33 @@ function refreshFinalize(planData) { return planData; } +// Completion evaluator. Once every non-skipped checklist step is `completed` +// (and none `failed`), the plan has been fully executed — transition it to +// "Completed" + stamp COMPLETED_AT. Runs after each phase's step-sync (in both +// refresh() and reconcile()), so the LAST execution skill to finish flips the +// plan terminal automatically — no skill has to call `--phase finalize` +// explicitly (nothing did, so the lifecycle previously never completed). Only +// advances from a pre-terminal, non-draft state: "In Execution" (the normal +// case after check-alm-plan.js promoted it on the first execution skill) or +// "Approved" (defensive fallback if that promotion didn't run). Never regresses +// a Draft or an already-Completed plan, and a `failed` step blocks completion +// (e.g. a failed deploy must not look "done" just because later steps are +// pending-skipped). +function evaluatePlanCompletion(planData) { + if (!planData) return; + const status = planData.PLAN_STATUS; + if (status !== 'In Execution' && status !== 'Approved') return; + if (!Array.isArray(planData.steps)) return; + const live = planData.steps.filter((s) => s && s.skip !== true && typeof s.name === 'string'); + if (live.length === 0) return; + const anyFailed = live.some((s) => s.status === 'failed'); + const allCompleted = live.every((s) => s.status === 'completed'); + if (!anyFailed && allCompleted) { + planData.PLAN_STATUS = 'Completed'; + planData.COMPLETED_AT = new Date().toISOString(); + } +} + // Refresh-phase helper: stamp `LAST_SYNC_AT` on planData so check-alm-plan.js's // freshness check correctly accounts for source-solution modifications caused // by the just-completed phase. Called by every phase that touches the source @@ -1085,6 +1119,7 @@ function reconcile({ projectRoot, render, rendererPath }) { process.stderr.write(`[refresh-alm-plan-data] reconcile phase "${phase}" failed: ${e.message}\n`); } } + evaluatePlanCompletion(planData); fs.writeFileSync(dataPath, JSON.stringify(planData, null, 2), 'utf8'); let rendered = false; @@ -1135,6 +1170,7 @@ function refresh({ projectRoot, phase, render, rendererPath, stageName }) { } applyRefresh(planData, phase, projectRoot, stageName); + evaluatePlanCompletion(planData); fs.writeFileSync(dataPath, JSON.stringify(planData, null, 2), 'utf8'); let rendered = false; diff --git a/plugins/power-pages/scripts/tests/check-alm-plan.test.js b/plugins/power-pages/scripts/tests/check-alm-plan.test.js index 52eee4df0..d04302754 100644 --- a/plugins/power-pages/scripts/tests/check-alm-plan.test.js +++ b/plugins/power-pages/scripts/tests/check-alm-plan.test.js @@ -54,7 +54,8 @@ test('returns exists:true / stale:false when plan exists and no env credentials' PLAN_STATUS: 'Approved', }); try { - const r = await checkAlmPlan({ projectRoot: dir }); + // writeHeartbeat:false → read-only check; an Approved plan is NOT promoted. + const r = await checkAlmPlan({ projectRoot: dir, writeHeartbeat: false }); assert.equal(r.exists, true); assert.equal(r.stale, false); assert.equal(r.generatedAt, '2026-04-01T00:00:00.000Z'); @@ -515,3 +516,44 @@ test('Unparseable LAST_SYNC_AT falls back to GENERATED_AT (defensive)', async () fs.rmSync(dir, { recursive: true, force: true }); } }); + +// --- Approved -> In Execution promotion (lifecycle activation) --------------- + +test('promotes Approved -> In Execution on the first execution-skill Phase 0 (writeHeartbeat default)', async () => { + const dir = tempProject({ SITE_NAME: 'T', PLAN_STATUS: 'Approved' }); + try { + const r = await checkAlmPlan({ projectRoot: dir }); + assert.equal(r.planStatus, 'In Execution', 'returned status reflects the promotion'); + assert.equal(r.inExecution.status, 'active', 'newly In Execution with first heartbeat is active'); + // Promotion + heartbeat are persisted to disk for the next in-chain skill. + const onDisk = JSON.parse(fs.readFileSync(path.join(dir, 'docs', '.alm-plan-data.json'), 'utf8')); + assert.equal(onDisk.PLAN_STATUS, 'In Execution'); + assert.ok(onDisk.LAST_INVOCATION_AT, 'first heartbeat written'); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } +}); + +test('does NOT promote Approved when writeHeartbeat is false (read-only callers: plan-alm, audits, tests)', async () => { + const dir = tempProject({ SITE_NAME: 'T', PLAN_STATUS: 'Approved' }); + try { + const r = await checkAlmPlan({ projectRoot: dir, writeHeartbeat: false }); + assert.equal(r.planStatus, 'Approved', 'read-only check leaves the plan Approved'); + const onDisk = JSON.parse(fs.readFileSync(path.join(dir, 'docs', '.alm-plan-data.json'), 'utf8')); + assert.equal(onDisk.PLAN_STATUS, 'Approved', 'disk untouched'); + assert.equal(onDisk.LAST_INVOCATION_AT, undefined, 'no heartbeat written'); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } +}); + +test('does NOT promote a Draft plan (only Approved is promotable)', async () => { + const dir = tempProject({ SITE_NAME: 'T', PLAN_STATUS: 'Draft' }); + try { + const r = await checkAlmPlan({ projectRoot: dir }); + assert.equal(r.planStatus, 'Draft'); + assert.equal(r.inExecution.status, 'not-running'); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } +}); diff --git a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js index ab23aecea..91c01898d 100644 --- a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js +++ b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js @@ -1952,3 +1952,97 @@ test('reconcile: soft no-op when there is no plan', (t) => { assert.equal(result.ok, false); assert.equal(result.reason, 'no-plan'); }); + +// --- Gap 5: completion evaluator (In Execution -> Completed) ----------------- + +test('completion: a refresh that leaves every non-skip step completed flips PLAN_STATUS to Completed', (t) => { + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'T', STRATEGY: 'pp-pipelines', PLAN_STATUS: 'In Execution', + validationRuns: { Staging: null }, + steps: [ + { name: 'Setup solution', status: 'completed' }, + { name: 'Deploy via pipeline to Staging', status: 'completed' }, + { name: 'Test site in Staging', status: 'pending' }, + ], + }); + // test-site flips the last pending step -> all done -> plan completes. + writeJson(path.join(root, 'docs', 'alm', 'last-test-site.json'), { runOutcome: 'passed', runAt: '2026-06-16T00:00:00.000Z' }); + + refresh({ projectRoot: root, phase: 'test-site', render: false, stageName: 'Staging' }); + + const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); + assert.equal(planData.PLAN_STATUS, 'Completed', 'all steps done -> Completed'); + assert.ok(planData.COMPLETED_AT, 'COMPLETED_AT stamped'); +}); + +test('completion: a still-pending step keeps PLAN_STATUS at In Execution', (t) => { + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'T', STRATEGY: 'pp-pipelines', PLAN_STATUS: 'In Execution', + steps: [ + { name: 'Deploy via pipeline to Staging', status: 'pending' }, + { name: 'Deploy via pipeline to Production', status: 'pending' }, + ], + }); + writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { stageName: 'Staging', status: 'Succeeded', deployedAt: '2026-06-16T00:00:00.000Z' }); + + refresh({ projectRoot: root, phase: 'deploy-pipeline', render: false }); + + const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); + assert.equal(planData.steps[0].status, 'completed', 'Staging deploy flipped'); + assert.equal(planData.PLAN_STATUS, 'In Execution', 'Production still pending -> not Completed'); + assert.equal(planData.COMPLETED_AT, undefined); +}); + +test('completion: a failed step blocks completion even if all others are done', (t) => { + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'T', STRATEGY: 'pp-pipelines', PLAN_STATUS: 'In Execution', + steps: [ + { name: 'Setup solution', status: 'completed' }, + { name: 'Deploy via pipeline to Staging', status: 'pending' }, + ], + }); + // A FAILED deploy marker flips the deploy step to 'failed', not 'completed'. + writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { stageName: 'Staging', status: 'Failed', deployedAt: '2026-06-16T00:00:00.000Z' }); + + refresh({ projectRoot: root, phase: 'deploy-pipeline', render: false }); + + const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); + assert.equal(planData.steps[1].status, 'failed'); + assert.equal(planData.PLAN_STATUS, 'In Execution', 'a failed step must NOT complete the plan'); +}); + +test('completion: skip:true steps are ignored when deciding completion', (t) => { + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'T', STRATEGY: 'pp-pipelines', PLAN_STATUS: 'In Execution', + validationRuns: { Staging: null }, + steps: [ + { name: 'Deploy via pipeline to Staging', status: 'completed' }, + { name: 'Test site in Staging', status: 'pending', skip: true }, + ], + }); + // Re-run a no-op-ish refresh; the skipped Test step must not block completion. + writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { stageName: 'Staging', status: 'Succeeded', deployedAt: '2026-06-16T00:00:00.000Z' }); + + refresh({ projectRoot: root, phase: 'deploy-pipeline', render: false }); + + const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); + assert.equal(planData.PLAN_STATUS, 'Completed', 'only non-skip steps count -> Completed'); +}); + +test('completion: a Draft plan is never auto-completed', (t) => { + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'T', STRATEGY: 'pp-pipelines', PLAN_STATUS: 'Draft', + steps: [{ name: 'Deploy via pipeline to Staging', status: 'completed' }], + }); + writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { stageName: 'Staging', status: 'Succeeded', deployedAt: '2026-06-16T00:00:00.000Z' }); + + refresh({ projectRoot: root, phase: 'deploy-pipeline', render: false }); + + const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); + assert.equal(planData.PLAN_STATUS, 'Draft', 'Draft is not a pre-terminal execution state'); +}); diff --git a/plugins/power-pages/skills/plan-alm/SKILL.md b/plugins/power-pages/skills/plan-alm/SKILL.md index ed7a34b32..e0e4e1f59 100644 --- a/plugins/power-pages/skills/plan-alm/SKILL.md +++ b/plugins/power-pages/skills/plan-alm/SKILL.md @@ -43,9 +43,11 @@ Steps: 0. **Detect prior ALM deferral for this project.** Before any discovery work, check whether the project root contains a `.alm-deferred` marker file. The marker is written by users who explicitly opted ALM-skill validators out of "missing artifacts" warnings (e.g. *"this site is handled separately"* or *"ni-dev — no ALM"*). If a user is now invoking `plan-alm`, we should surface that the marker is present and ask what to do, rather than silently proceeding (which would build a plan the user previously decided not to maintain) or silently removing the marker (which would re-enable nags on every other ALM skill). ```bash - node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/check-alm-plan.js" --projectRoot "." + node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/check-alm-plan.js" --projectRoot "." --no-heartbeat ``` + > Use `--no-heartbeat` here: this is a **read-only** deferral check by the *planner*, not an execution-skill Phase 0 gate. Without it, `check-alm-plan.js` would promote an already-`Approved` plan to `In Execution` (and refresh the heartbeat) just because you re-opened `plan-alm` — but re-planning isn't execution. Execution skills call it *without* `--no-heartbeat` so the first one to run does the `Approved → In Execution` promotion. + > 🚦 **Gate (progress · plan-alm:1.deferral):** `.alm-deferred` marker present — continue and remove, continue and keep marker, or cancel. Determines whether downstream ALM skills resume gate enforcement. From 3db879da6a696bd5cc93095873268a772a0d0353 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 22:56:37 +0530 Subject: [PATCH 18/38] Surface COMPLETED_AT in the ALM plan footer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When the lifecycle reaches "Completed", the plan footer now shows a "Completed: " line beneath the approver/approval-date, rendered from COMPLETED_AT. Conditional — the `__COMPLETED_LINE__` placeholder maps to an empty string until the plan completes, so there's no orphan token and the line only appears once execution is done. (The template already styled the .completed and .in-execution status badges; this adds the timestamp detail.) +2 renderer tests. 1229 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../scripts/tests/render-alm-plan.test.js | 36 +++++++++++++++++++ .../plan-alm/assets/alm-plan-template.html | 1 + .../plan-alm/scripts/render-alm-plan.js | 8 ++++- 3 files changed, 44 insertions(+), 1 deletion(-) diff --git a/plugins/power-pages/scripts/tests/render-alm-plan.test.js b/plugins/power-pages/scripts/tests/render-alm-plan.test.js index 9380a86a7..f91b17f63 100644 --- a/plugins/power-pages/scripts/tests/render-alm-plan.test.js +++ b/plugins/power-pages/scripts/tests/render-alm-plan.test.js @@ -1944,3 +1944,39 @@ test('render-alm-plan: checklist link onclick reuses the existing data-tab click fs.rmSync(tmpDir, { recursive: true, force: true }); } }); + +// ── COMPLETED_AT footer line (lifecycle terminal) ──────────────────────────── + +test('render-alm-plan: surfaces COMPLETED_AT in the footer when the plan is Completed', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'render-alm-completed-')); + const outputPath = path.join(tmpDir, 'alm-plan.html'); + try { + const { status } = runScript( + makeValidData({ PLAN_STATUS: 'Completed', COMPLETED_AT: '2026-06-16T18:30:00.000Z' }), + outputPath, + ); + assert.equal(status, 0); + const html = fs.readFileSync(outputPath, 'utf8'); + assert.match(html, /id="completed-at"/, 'completed-at span rendered'); + assert.match(html, /Completed:<\/strong>\s*2026-06-16T18:30:00\.000Z<\/span>/); + assert.ok(!html.includes('__COMPLETED_LINE__'), 'no orphan placeholder'); + assert.match(html, /class="plan-status completed"/, 'status badge styled completed'); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } +}); + +test('render-alm-plan: omits the Completed footer line when COMPLETED_AT is absent', () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'render-alm-nocompleted-')); + const outputPath = path.join(tmpDir, 'alm-plan.html'); + try { + const { status } = runScript(makeValidData({ PLAN_STATUS: 'In Execution' }), outputPath); + assert.equal(status, 0); + const html = fs.readFileSync(outputPath, 'utf8'); + assert.ok(!html.includes('id="completed-at"'), 'no completed-at span when not completed'); + assert.ok(!html.includes('__COMPLETED_LINE__'), 'placeholder still replaced (empty)'); + assert.match(html, /class="plan-status in-execution"/, 'status badge styled in-execution'); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } +}); diff --git a/plugins/power-pages/skills/plan-alm/assets/alm-plan-template.html b/plugins/power-pages/skills/plan-alm/assets/alm-plan-template.html index 495fa4e13..be5b53a80 100644 --- a/plugins/power-pages/skills/plan-alm/assets/alm-plan-template.html +++ b/plugins/power-pages/skills/plan-alm/assets/alm-plan-template.html @@ -419,6 +419,7 @@

Execution Checklist

Approved by: __APPROVED_BY__  ·  Approval date: __APPROVAL_DATE__ + __COMPLETED_LINE__
diff --git a/plugins/power-pages/skills/plan-alm/scripts/render-alm-plan.js b/plugins/power-pages/skills/plan-alm/scripts/render-alm-plan.js index 33453328b..5f66bc00f 100644 --- a/plugins/power-pages/skills/plan-alm/scripts/render-alm-plan.js +++ b/plugins/power-pages/skills/plan-alm/scripts/render-alm-plan.js @@ -6,7 +6,7 @@ * node render-alm-plan.js --output --data * * Required top-level keys in the JSON data file: - * SITE_NAME, GENERATED_AT, STRATEGY, PLAN_STATUS, APPROVED_BY, APPROVAL_DATE, + * SITE_NAME, GENERATED_AT, STRATEGY, PLAN_STATUS, APPROVED_BY, APPROVAL_DATE, COMPLETED_AT, * stages, steps, risks * * Optional v2 keys (added for split-solutions support): @@ -1448,6 +1448,12 @@ const replacements = { PLAN_STATUS: escapeHtml(data.PLAN_STATUS || 'Draft'), APPROVED_BY: escapeHtml(data.APPROVED_BY || ''), APPROVAL_DATE: escapeHtml(data.APPROVAL_DATE || ''), + // Completion footer line — only rendered once the plan reaches "Completed" + // (refresh-alm-plan-data.js stamps COMPLETED_AT when every step is done). + // Empty string otherwise, so the placeholder is always replaced (no orphan token). + COMPLETED_LINE: data.COMPLETED_AT + ? `
Completed: ${escapeHtml(data.COMPLETED_AT)}` + : '', OVERVIEW_SUMMARY: buildOverviewSummary(), STAT_COMPONENTS: (componentCount || 0).toLocaleString(), STAT_ENVVARS: envVarStatDisplay(), From f5ca2bb9d377b570f3c519ba938da6af0c6a3e18 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 12:37:31 +0530 Subject: [PATCH 19/38] Site-referenced table discovery + dependency-aware solution splitting MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replaces publisher-prefix table discovery (which over-counts catastrophically with a shared/default publisher — a 6-table site matched 22 unrelated tables — and misses real tables from a different prefix) with site-referenced scoping: the custom tables the site's `.powerpages-site/table-permissions/` (+ datamodel manifest) actually reference, intersected with the env's custom-unmanaged tables. SME-confirmed: table permissions are the complete signal. Replaces the old one-solution-per-table-name-stem split heuristic (which produced ~one solution per table — e.g. a 21-solution split) with a dependency-aware, capacity-bounded packer: union-find connected-component clusters over table relationships (lookups + N:N), then first-fit-decreasing bin-packing of whole clusters into the fewest solutions under maxTableCount/maxSchemaAttrs, capped at maxSchemaSplitSolutions (8). The split trigger + thresholds are unchanged — only the packing. New shared libs: - resolve-site-tables.js — site-referenced table scoping (single source of truth) - query-metadata.js — consolidated custom-unmanaged-table query - query-table-relationships.js — relationship edges (lib; audit-permissions CLI is now a thin wrapper) - validation-helpers.js — odataGet/odataGetAll shared paginator estimate-solution-size.js now emits tableCountScope + tableRelationships[]; compute-split-plan.js consumes the edges. setup-solution Phase 5.2.D uses the shared discovery helper. 1209 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../power-pages/.claude-plugin/plugin.json | 2 +- plugins/power-pages/AGENTS.md | 8 +- .../power-pages/scripts/lib/alm-thresholds.js | 6 + .../scripts/lib/compute-split-plan.js | 127 +++++++++++++++--- .../scripts/lib/discover-site-components.js | 62 +++++---- .../scripts/lib/estimate-solution-size.js | 118 +++++++++++----- .../power-pages/scripts/lib/query-metadata.js | 44 ++++++ .../scripts/lib/query-table-relationships.js | 65 +++++++++ .../scripts/lib/resolve-site-tables.js | 103 ++++++++++++++ .../scripts/lib/validation-helpers.js | 49 +++++++ .../scripts/tests/compute-split-plan.test.js | 101 +++++++++++++- .../tests/discover-site-components.test.js | 40 +++++- .../tests/estimate-solution-size.test.js | 84 ++++++++++++ .../integration/discover-integration.test.js | 24 ++++ .../scripts/tests/query-metadata.test.js | 44 ++++++ .../tests/query-table-relationships.test.js | 52 +++++++ .../scripts/tests/resolve-site-tables.test.js | 98 ++++++++++++++ .../scripts/tests/validation-helpers.test.js | 25 ++++ .../scripts/query-table-relationships.js | 35 ++--- .../skills/setup-solution/SKILL.md | 22 +-- 20 files changed, 984 insertions(+), 125 deletions(-) create mode 100644 plugins/power-pages/scripts/lib/query-metadata.js create mode 100644 plugins/power-pages/scripts/lib/query-table-relationships.js create mode 100644 plugins/power-pages/scripts/lib/resolve-site-tables.js create mode 100644 plugins/power-pages/scripts/tests/query-metadata.test.js create mode 100644 plugins/power-pages/scripts/tests/query-table-relationships.test.js create mode 100644 plugins/power-pages/scripts/tests/resolve-site-tables.test.js diff --git a/plugins/power-pages/.claude-plugin/plugin.json b/plugins/power-pages/.claude-plugin/plugin.json index 25996f2b7..13763c7f2 100644 --- a/plugins/power-pages/.claude-plugin/plugin.json +++ b/plugins/power-pages/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "power-pages", - "version": "2.3.0", + "version": "2.4.0", "description": "Create and deploy Power Pages sites using modern development approaches. Supports code sites (SPAs) with React, Angular, Vue, or Astro. Includes ALM orchestration (plan-alm) with a solution-splitting decision tree, per-solution pipelines, Azure Blob asset advisory, manifest schema v2 for multi-solution deployments, and force-link remediation for cross-host pipeline migrations.", "author": { "name": "Microsoft", diff --git a/plugins/power-pages/AGENTS.md b/plugins/power-pages/AGENTS.md index 899cfad71..dc28b330d 100644 --- a/plugins/power-pages/AGENTS.md +++ b/plugins/power-pages/AGENTS.md @@ -204,8 +204,12 @@ Shared lib modules live at `scripts/lib/` and are imported by other scripts via #### Solution Splitting Decision Tree (v1.3.0+) - `scripts/lib/alm-thresholds.js`: Central default threshold constants for the split decision tree. Loads optional `.alm-config.json` from project root and merges over defaults. Exports `DEFAULTS`, `DEFAULT_CONFIG`, `loadConfig(projectRoot)`, `classifyTier(value, greenUpperExclusive, yellowUpperExclusive)`, `deepMerge(target, source)`. Used by `estimate-solution-size.js` and `compute-split-plan.js`. -- `scripts/lib/estimate-solution-size.js`: Estimates solution size + component counts by querying Dataverse. Args: `--envUrl`, `--websiteRecordId`, `--token` (opt), `--publisherPrefix` (opt), `--siteName` (opt), `--solutionId` (opt — scopes env var count to the target solution; without it falls back to a publisher-prefix tenant-wide query that overcounts when prefix is shared), `--datamodelManifest` (opt), `--projectRoot` (opt — enables disk cross-check: walks the local build-output directory (`dist/`, `public-output/`, `build/`, `.output/`) and surfaces the byte total). Output: `{ totalSizeMB, componentCountSiteTotal, componentCountSiteActionable, componentCountInSolution, orphansOnSite, tableCount, schemaAttrCount, webFilesAggregateMB, webFilesIndividual[], webFileCount, webFileSampleSize, webFilesDiskMeasuredMB, webFilesDiskMeasuredPath, webFilesDiskFileCount, cloudFlowCount, botCount, envVarCount, envVarCountScope, envVarCountTenantWide, mediaRatio, siteType, tables[], breakdown, estimationMethod, estimationAccuracyPct, truncationSuspected, truncationWarnings, ppcGroundTruthCount }`. Metadata-based estimation with ±15% caveat. Web-file size is measured via stratified sample (first 50 + middle 50 + last 50, cap 150) scaled to full count. Disk fields are null unless `--projectRoot` was passed AND a build-output directory was found. Truncation canaries fire when Dataverse pagination disagrees with `@odata.count`, when ppcs land on a page-size boundary, when sampled average bytes/file < 1 KB at scale, or when the disk total exceeds the Dataverse total by >2× — any signal flips `truncationSuspected: true` with a per-cause `truncationWarnings[]` entry. Used by `plan-alm` Phase 1 Step 10. -- `scripts/lib/compute-split-plan.js`: Runs the split decision tree against a size-estimate blob. Args: `--estimate `, `--projectRoot` (opt — for `.alm-config.json` overrides), `--siteName` (opt), `--publisherPrefix` (opt). Output: `{ sizeAnalysis, assetAdvisory, splitStrategy, appliedStrategies, compositeSubPartitioned, proposedSolutions[], recommendations[], truncationSuspected, truncationWarnings }`. Evaluates strategies in priority order: Strategy 3 (Schema Segmentation) → Strategy 1 (Layer Split) → Strategy 2 (Change-Frequency) → Strategy 4 (Config Isolation). Strategy 4 stacks additively. Strategy 1 also runs a composite sub-partition pass: when Core still exceeds the size OR component-count cap after Web Assets are peeled off, Core is replaced with change-frequency-shaped sub-children (`_Foundation`/`_Config`/`_Content`, plus `_Integration` whenever the parent had any flows or bots — coverage takes priority over the `changeFreqMinFlows` heuristic, which governs only the TOP-LEVEL strategy choice). When additive Strategy 4 is firing concurrently (a top-level `_EnvVars` solution), `_Config` drops `Environment Variable` from its componentTypes to avoid double-claim; when it isn't, `_Config` absorbs env vars so they have an owner. Sub-partitioning sets `compositeSubPartitioned: true` and appends `composite-sub-partition` to `appliedStrategies`. `validateSplits` checks BOTH the size AND component-count cap per split (skipping `isFutureBuffer` solutions). Supports `.alm-config.json` overrides including `strategyOverride` to bypass the tree. See `solution-splitting-logic.md` spec in design docs for full logic. +- `scripts/lib/estimate-solution-size.js`: Estimates solution size + component counts by querying Dataverse. Args: `--envUrl`, `--websiteRecordId`, `--token` (opt), `--publisherPrefix` (opt), `--siteName` (opt), `--solutionId` (opt — scopes env var count to the target solution; without it falls back to a publisher-prefix tenant-wide query that overcounts when prefix is shared), `--datamodelManifest` (opt), `--projectRoot` (opt — enables disk cross-check: walks the local build-output directory (`dist/`, `public-output/`, `build/`, `.output/`) and surfaces the byte total). Output: `{ totalSizeMB, componentCountSiteTotal, componentCountSiteActionable, componentCountInSolution, orphansOnSite, tableCount, tableCountScope, schemaAttrCount, webFilesAggregateMB, webFilesIndividual[], webFileCount, webFileSampleSize, webFilesDiskMeasuredMB, webFilesDiskMeasuredPath, webFilesDiskFileCount, cloudFlowCount, botCount, envVarCount, envVarCountScope, envVarCountTenantWide, mediaRatio, siteType, tables[], tableRelationships[], breakdown, estimationMethod, estimationAccuracyPct, truncationSuspected, truncationWarnings, ppcGroundTruthCount }`. **Table discovery is site-referenced, NOT publisher-prefix:** `tableCount`/`tables[]` are scoped to the custom tables the site actually references — its `.powerpages-site/table-permissions/` (+ datamodel manifest) intersected with the env's custom-unmanaged tables (via `resolve-site-tables.js` + `query-metadata.js`). `tableCountScope` ∈ `"site-referenced" | "manifest-only" | "unavailable"` (the last → 0 tables, never an env-wide prefix dump). `--publisherPrefix` now scopes ONLY the env var count, not tables. `tableRelationships[]` are `[a,b]` dependency edges (lookups + N:N, via `query-table-relationships.js`) among the scoped tables, consumed by `compute-split-plan.js` to cluster related tables into the same solution. Metadata-based estimation with ±15% caveat. Web-file size is measured via stratified sample (first 50 + middle 50 + last 50, cap 150) scaled to full count. Disk fields are null unless `--projectRoot` was passed AND a build-output directory was found. Truncation canaries fire when Dataverse pagination disagrees with `@odata.count`, when ppcs land on a page-size boundary, when sampled average bytes/file < 1 KB at scale, or when the disk total exceeds the Dataverse total by >2× — any signal flips `truncationSuspected: true` with a per-cause `truncationWarnings[]` entry. Used by `plan-alm` Phase 1 Step 10. +- `scripts/lib/compute-split-plan.js`: Runs the split decision tree against a size-estimate blob. Args: `--estimate `, `--projectRoot` (opt — for `.alm-config.json` overrides), `--siteName` (opt), `--publisherPrefix` (opt). Output: `{ sizeAnalysis, assetAdvisory, splitStrategy, appliedStrategies, compositeSubPartitioned, proposedSolutions[], recommendations[], truncationSuspected, truncationWarnings }`. Evaluates strategies in priority order: Strategy 3 (Schema Segmentation) → Strategy 1 (Layer Split) → Strategy 2 (Change-Frequency) → Strategy 4 (Config Isolation). **Schema Segmentation is dependency-aware + capacity-bounded:** it builds connected-component clusters from `estimate.tableRelationships` (union-find), then bin-packs whole clusters (never splitting a relationship) into the fewest solutions that keep each under `maxTableCount`/`maxSchemaAttrs`, capped at `maxSchemaSplitSolutions` (default 8). This replaced the old one-solution-per-table-name-stem heuristic that produced ~one solution per table. An indivisible cluster over the cap stays whole and raises an oversized-cluster `recommendations[]` warning. The split trigger + thresholds are unchanged — only the packing. Strategy 4 stacks additively. Strategy 1 also runs a composite sub-partition pass: when Core still exceeds the size OR component-count cap after Web Assets are peeled off, Core is replaced with change-frequency-shaped sub-children (`_Foundation`/`_Config`/`_Content`, plus `_Integration` whenever the parent had any flows or bots — coverage takes priority over the `changeFreqMinFlows` heuristic, which governs only the TOP-LEVEL strategy choice). When additive Strategy 4 is firing concurrently (a top-level `_EnvVars` solution), `_Config` drops `Environment Variable` from its componentTypes to avoid double-claim; when it isn't, `_Config` absorbs env vars so they have an owner. Sub-partitioning sets `compositeSubPartitioned: true` and appends `composite-sub-partition` to `appliedStrategies`. `validateSplits` checks BOTH the size AND component-count cap per split (skipping `isFutureBuffer` solutions). Supports `.alm-config.json` overrides including `strategyOverride` to bypass the tree. See `solution-splitting-logic.md` spec in design docs for full logic. +- `scripts/lib/resolve-site-tables.js`: Single source of truth for "which custom tables does this site actually use." `collectReferencedEntityNames({ projectRoot, datamodelManifestPath })` reads `.powerpages-site/table-permissions/*.tablepermission.yml` (`entitylogicalname`, via `powerpages-config.js → loadTablePermissions`) + the datamodel manifest → `{ names:Set, available, sources }`. `scopeCustomTables(referencedNames, customUnmanagedTables)` intersects that set with the env's custom-unmanaged tables. SME-confirmed: table permissions are the complete signal ("if a table is used in the site there will be permissions for it"), so forms/lists are NOT scanned. Used by `estimate-solution-size.js` and `discover-site-components.js` to replace the publisher-prefix table dump. +- `scripts/lib/query-metadata.js`: `queryCustomUnmanagedTables(envUrl, token, makeRequest?)` → `[{ logicalName, metadataId, schemaName, displayName }]` (the single `EntityDefinitions?$filter=IsCustomEntity` query, `IsManaged===false` filtered). Consolidates the formerly-triplicated custom-table query (estimator, discover-site-components, setup-solution). Reuses `odataGetAll` from `validation-helpers.js`. +- `scripts/lib/query-table-relationships.js`: `fetchTableRelationships(envUrl, table, token, makeRequest?)` → `{ oneToMany[], manyToMany[] }`. Extracted from `skills/audit-permissions/scripts/query-table-relationships.js` (now a thin CLI wrapper over this lib) and extended with ManyToMany. OneToMany errors propagate; ManyToMany is best-effort. Used by the estimator to build `tableRelationships[]` and by audit-permissions for relationship-scope validation. +- `scripts/lib/validation-helpers.js` also exports `odataGet(url, token, makeRequest?)` + `odataGetAll(url, token, makeRequest?, maxPages?)` — the shared, injectable OData GET + `@odata.nextLink` pagination used by the new metadata/relationship helpers (avoids each lib rolling its own paginator). #### Solution Management diff --git a/plugins/power-pages/scripts/lib/alm-thresholds.js b/plugins/power-pages/scripts/lib/alm-thresholds.js index 3b57f030c..4df94e50b 100644 --- a/plugins/power-pages/scripts/lib/alm-thresholds.js +++ b/plugins/power-pages/scripts/lib/alm-thresholds.js @@ -21,6 +21,12 @@ const DEFAULTS = Object.freeze({ hardFlagComponentCount: 10000, maxSchemaAttrs: 15000, maxTableCount: 20, + // Safety ceiling on the number of auto-derived schema-split solutions. The + // schema-segmentation packing keeps each solution under maxTableCount / + // maxSchemaAttrs, but caps the COUNT here so a pathological schema can't + // explode into dozens of solutions — beyond this, the hardFlagComponentCount + // recommendation tells the user to archive/consolidate instead. + maxSchemaSplitSolutions: 8, maxAggregateWebFilesMB: 40, maxSingleFileMB: 2, maxEnvVarCount: 500, diff --git a/plugins/power-pages/scripts/lib/compute-split-plan.js b/plugins/power-pages/scripts/lib/compute-split-plan.js index e4291b371..f59028400 100644 --- a/plugins/power-pages/scripts/lib/compute-split-plan.js +++ b/plugins/power-pages/scripts/lib/compute-split-plan.js @@ -299,7 +299,7 @@ function partitionByChangeFrequency(estimate, meta) { function partitionBySchema(estimate, meta, config) { const explicitDomains = Array.isArray(config.domains) && config.domains.length > 0 ? config.domains - : deriveDomainsFromPrefix(estimate); + : deriveDomainsByCapacity(estimate, config.thresholds); // Derive domain vs site size shares from the estimator's breakdown when available, // falling back to a 50/50 heuristic only if breakdown is absent. @@ -319,9 +319,12 @@ function partitionBySchema(estimate, meta, config) { componentTypes: ['Table'], description: `Schema domain: ${dom.name}. Tables: ${(dom.tableLogicalNames || []).join(', ') || '(derived)'}${domainDescSuffix}`, sizeMB: round(sizePerDomain), - componentCount: Math.ceil( - (estimate.schemaAttrCount || 0) / domainCount, - ), + // A Table domain's component count IS its table count when known (each table + // is one Entity solution component). Falls back to an even attr-share split + // only for explicit domains that didn't list their tables. + componentCount: (dom.tableLogicalNames && dom.tableLogicalNames.length > 0) + ? dom.tableLogicalNames.length + : Math.ceil((estimate.schemaAttrCount || 0) / domainCount), components: [], tableLogicalNames: dom.tableLogicalNames || [], })); @@ -345,23 +348,99 @@ function partitionBySchema(estimate, meta, config) { return [...domainSolutions, siteSolution]; } -function deriveDomainsFromPrefix(estimate) { - const tables = estimate.tables || []; - if (tables.length === 0) return [{ name: 'All', tableLogicalNames: [] }]; +// --- Dependency-aware schema packing --------------------------------------- +// +// Replaces the old "one solution per table-name stem" heuristic (which produced +// ~one solution per table for any distinctly-named schema). Tables connected by +// a relationship MUST ship together, so we: +// 1. Group tables into connected components (union-find over the estimator's +// `tableRelationships` edges). Because components have no edges between +// them, packing whole components into separate solutions never cuts a +// relationship — so there are no cross-/circular-solution table deps and +// import order among the table solutions is irrelevant. +// 2. Bin-pack the components into the FEWEST solutions that keep each under the +// per-solution caps (maxTableCount tables AND maxSchemaAttrs columns), +// capped at maxSchemaSplitSolutions. + +function normalizeTables(estimate) { + return (estimate.tables || []) + .map((t) => ({ + logicalName: (t && (t.logicalName || t)).toString(), + attributeCount: (t && t.attributeCount) || 0, + })) + .filter((t) => t.logicalName); +} +// Union-find over tables + relationship edges -> array of clusters (each a list +// of table objects). A table with no edges is its own singleton cluster. +function buildTableClusters(tables, edges) { + const idx = new Map(); + tables.forEach((t, i) => idx.set(t.logicalName.toLowerCase(), i)); + const parent = tables.map((_, i) => i); + const find = (x) => { while (parent[x] !== x) { parent[x] = parent[parent[x]]; x = parent[x]; } return x; }; + const union = (a, b) => { const ra = find(a), rb = find(b); if (ra !== rb) parent[ra] = rb; }; + for (const e of edges || []) { + if (!Array.isArray(e) || e.length < 2) continue; + const ia = idx.get(String(e[0]).toLowerCase()); + const ib = idx.get(String(e[1]).toLowerCase()); + if (ia != null && ib != null) union(ia, ib); + } const groups = new Map(); - for (const t of tables) { - const name = (t.logicalName || t).toString(); - const afterPrefix = name.includes('_') ? name.split('_').slice(1).join('_') : name; - const stem = afterPrefix.split(/[_]/)[0] || 'misc'; - const key = stem.charAt(0).toUpperCase() + stem.slice(1); - if (!groups.has(key)) groups.set(key, []); - groups.get(key).push(name); + tables.forEach((t, i) => { + const r = find(i); + if (!groups.has(r)) groups.set(r, []); + groups.get(r).push(t); + }); + return [...groups.values()]; +} + +function clusterAttrs(cluster) { + return cluster.reduce((s, t) => s + (t.attributeCount || 0), 0); +} + +// First-fit-decreasing pack of whole clusters into `n` buckets, respecting the +// per-solution table + attribute caps. A cluster that fits nowhere under the +// caps (oversized, or n too small) goes to the least-loaded bucket — that bucket +// then exceeds a cap and is surfaced by the oversized-cluster recommendation. +function packClusters(clusters, n, thresholds) { + const sorted = [...clusters].sort((a, b) => (clusterAttrs(b) - clusterAttrs(a)) || (b.length - a.length)); + const buckets = Array.from({ length: Math.max(n, 1) }, () => ({ tables: [], attrs: 0 })); + for (const cluster of sorted) { + const cAttrs = clusterAttrs(cluster); + let target = buckets.findIndex( + (b) => b.tables.length + cluster.length <= thresholds.maxTableCount && + b.attrs + cAttrs <= thresholds.maxSchemaAttrs, + ); + if (target === -1) { + target = buckets.reduce((best, b, i) => (b.attrs < buckets[best].attrs ? i : best), 0); + } + buckets[target].tables.push(...cluster); + buckets[target].attrs += cAttrs; } + return buckets.filter((b) => b.tables.length > 0); +} - return Array.from(groups.entries()).map(([name, tableLogicalNames]) => ({ - name, - tableLogicalNames, +// Returns capacity-bounded "domains" (one per packed bucket) in the same shape +// the schema partitioner consumes: { name, tableLogicalNames }. +function deriveDomainsByCapacity(estimate, thresholds) { + const tables = normalizeTables(estimate); + if (tables.length === 0) return [{ name: 'Tables', tableLogicalNames: [] }]; + + const clusters = buildTableClusters(tables, estimate.tableRelationships || []); + const totalAttrs = tables.reduce((s, t) => s + t.attributeCount, 0); + const ceiling = (thresholds && thresholds.maxSchemaSplitSolutions) || 8; + let n = Math.max( + 1, + Math.ceil(tables.length / thresholds.maxTableCount), + Math.ceil(totalAttrs / Math.max(thresholds.maxSchemaAttrs, 1)), + ); + n = Math.min(n, ceiling, clusters.length); + + const buckets = packClusters(clusters, n, thresholds); + const multi = buckets.length > 1; + return buckets.map((b, i) => ({ + name: multi ? `Tables ${i + 1}` : 'Tables', + tableLogicalNames: b.tables.map((t) => t.logicalName), })); } @@ -704,6 +783,17 @@ function computeSplitPlan({ estimate, config, meta }) { proposedSolutions = appendFutureBuffer(proposedSolutions, meta); const splitWarnings = validateSplits(proposedSolutions, config.thresholds); + // Oversized-cluster guard: a Table solution holding more tables than the + // per-solution cap means a single connected dependency cluster couldn't be + // split without cutting a relationship. Name it so the user can decide whether + // to denormalize the schema or raise the cap — we never silently split a cluster. + const oversizedClusterWarnings = proposedSolutions + .filter((s) => Array.isArray(s.tableLogicalNames) && + s.tableLogicalNames.length > config.thresholds.maxTableCount) + .map((s) => ({ + type: 'warning', + message: `Solution ${s.uniqueName} holds ${s.tableLogicalNames.length} related tables — above the ${config.thresholds.maxTableCount}-per-solution cap — because they form one dependency cluster that cannot be split without breaking a relationship. Consider denormalizing the schema or raising maxTableCount in .alm-config.json.`, + })); // Surface estimator-side truncation warnings as `recommendations[]` entries // so the rendered plan shows them inline. These get the `error` type because // a truncated input is more dangerous than a normal split-decision warning @@ -715,7 +805,8 @@ function computeSplitPlan({ estimate, config, meta }) { })); const recommendations = truncationRecs .concat(buildRecommendations(estimate, strategy, config)) - .concat(splitWarnings); + .concat(splitWarnings) + .concat(oversizedClusterWarnings); const appliedStrategies = [strategy.primary]; if (strategy.additive) appliedStrategies.push('strategy-4-config-isolation'); diff --git a/plugins/power-pages/scripts/lib/discover-site-components.js b/plugins/power-pages/scripts/lib/discover-site-components.js index fdd1cf421..c774e14b5 100644 --- a/plugins/power-pages/scripts/lib/discover-site-components.js +++ b/plugins/power-pages/scripts/lib/discover-site-components.js @@ -53,6 +53,8 @@ 'use strict'; const helpers = require('./validation-helpers'); +const { queryCustomUnmanagedTables } = require('./query-metadata'); +const { collectReferencedEntityNames, scopeCustomTables } = require('./resolve-site-tables'); /** Authoritative picklist labels for powerpagecomponenttype. */ const PPC_TYPE_LABELS = Object.freeze({ @@ -107,6 +109,8 @@ function parseArgs(argv) { siteId: null, publisherPrefix: null, solutionId: null, + projectRoot: null, + datamodelManifestPath: null, }; for (let i = 0; i < args.length; i++) { if (args[i] === '--envUrl' && args[i + 1]) out.envUrl = args[++i]; @@ -114,6 +118,8 @@ function parseArgs(argv) { else if (args[i] === '--siteId' && args[i + 1]) out.siteId = args[++i]; else if (args[i] === '--publisherPrefix' && args[i + 1]) out.publisherPrefix = args[++i]; else if (args[i] === '--solutionId' && args[i + 1]) out.solutionId = args[++i]; + else if (args[i] === '--projectRoot' && args[i + 1]) out.projectRoot = args[++i]; + else if (args[i] === '--datamodelManifest' && args[i + 1]) out.datamodelManifestPath = args[++i]; } return out; } @@ -166,6 +172,8 @@ async function discoverSiteComponents({ siteId, publisherPrefix = null, solutionId = null, + projectRoot = null, + datamodelManifestPath = null, makeRequest = helpers.makeRequest, } = {}) { if (!envUrl) throw new Error('--envUrl is required'); @@ -232,10 +240,13 @@ async function discoverSiteComponents({ ? await discoverEnvVars({ baseUrl, token, publisherPrefix, makeRequest }) : []; - // 5) Custom tables filtered by publisher prefix (optional) - const customTables = publisherPrefix - ? await discoverCustomTables({ baseUrl, token, publisherPrefix, makeRequest }) - : []; + // 5) Custom tables the SITE references (table permissions + datamodel manifest), + // intersected with the env's custom-unmanaged tables. NOT a publisher-prefix + // dump — that over-counted unrelated tables sharing the prefix (the + // new_/default-publisher bug). Empty when no local signal is available. + const customTables = await discoverCustomTables({ + baseUrl, token, projectRoot, datamodelManifestPath, makeRequest, + }); const result = { siteId, @@ -382,28 +393,27 @@ async function discoverEnvVars({ baseUrl, token, publisherPrefix, makeRequest }) })); } -async function discoverCustomTables({ baseUrl, token, publisherPrefix, makeRequest }) { - // The $metadata/EntityDefinitions endpoint doesn't support `startswith` (0x8006088a), - // so we fetch all custom tables and filter client-side. Custom-entity sets are small - // enough that a single request is fine. MetadataId is included so callers can diff - // against solutioncomponents.objectid (componenttype 1 = Entity). - // publisherPrefix validated at the entry point of discoverSiteComponents. - const prefixLower = String(publisherPrefix).trim().toLowerCase(); - const url = - `${baseUrl}/api/data/v9.2/EntityDefinitions` + - `?$filter=IsCustomEntity eq true` + - `&$select=LogicalName,SchemaName,DisplayName,MetadataId`; - const rows = await odataGetAll(url, token, makeRequest); - return rows - .filter((r) => (r.LogicalName || '').toLowerCase().startsWith(`${prefixLower}_`)) - .map((r) => ({ - id: r.MetadataId, - logicalName: r.LogicalName, - schemaName: r.SchemaName, - displayName: - (r.DisplayName && r.DisplayName.UserLocalizedLabel && r.DisplayName.UserLocalizedLabel.Label) || - r.SchemaName, - })); +async function discoverCustomTables({ baseUrl, token, projectRoot, datamodelManifestPath, makeRequest }) { + // Scope to the tables the SITE references (its table permissions + datamodel + // manifest — SME-confirmed complete), intersected with the env's custom-unmanaged + // tables. Replaces the old publisher-prefix dump that returned every table + // sharing the prefix (catastrophic with `new_`/default publishers). MetadataId + // is returned as `id` so callers can diff against solutioncomponents.objectid + // (componenttype 1 = Entity). + const { names, available } = collectReferencedEntityNames({ projectRoot, datamodelManifestPath }); + if (!available) return []; // no local signal — empty, NEVER a prefix dump + let customUnmanaged = []; + try { + customUnmanaged = await queryCustomUnmanagedTables(baseUrl, token, makeRequest); + } catch { + customUnmanaged = []; + } + return scopeCustomTables(names, customUnmanaged).map((t) => ({ + id: t.metadataId, + logicalName: t.logicalName, + schemaName: t.schemaName, + displayName: t.displayName, + })); } if (require.main === module) { diff --git a/plugins/power-pages/scripts/lib/estimate-solution-size.js b/plugins/power-pages/scripts/lib/estimate-solution-size.js index f38a6e844..911146e07 100644 --- a/plugins/power-pages/scripts/lib/estimate-solution-size.js +++ b/plugins/power-pages/scripts/lib/estimate-solution-size.js @@ -27,6 +27,9 @@ const helpers = require('./validation-helpers'); const { getAuthToken } = helpers; +const { queryCustomUnmanagedTables } = require('./query-metadata'); +const { fetchTableRelationships } = require('./query-table-relationships'); +const { collectReferencedEntityNames, scopeCustomTables } = require('./resolve-site-tables'); // `makeRequest` is accessed via `helpers.makeRequest` (not destructured) so // tests can inject a mock by mutating `helpers.makeRequest` before calling // the top-level `estimateSolutionSize`. See estimate-solution-size.test.js for @@ -285,44 +288,78 @@ async function discoverPowerPageSiteLanguages(envUrl, websiteRecordId, token) { } } -async function discoverTables(envUrl, publisherPrefix, token, manifestPath) { - // Try manifest first - const fs = require('fs'); - let manifestTables = []; - if (manifestPath && fs.existsSync(manifestPath)) { - try { - const man = JSON.parse(fs.readFileSync(manifestPath, 'utf8')); - const entries = man.entities || man.tables || []; - manifestTables = entries.map((e) => ({ - logicalName: e.logicalName || e.LogicalName || e.name, - metadataId: e.metadataId || e.MetadataId, - })); - } catch {} +// Discovers the custom tables the SITE actually references — NOT every table +// sharing the publisher prefix. The old prefix-wide enumeration over-counted +// catastrophically with a shared/default publisher (`new_`, env default): a +// 6-table site reported 22 tables, which cascaded into absurd schema splits. +// +// Source of truth = the site's table permissions (+ datamodel manifest), per +// SME: "If a table is used in the site there will be permissions for it." +// We intersect those referenced names with the env's custom-unmanaged tables so +// standard tables (contact/annotation) and managed template tables drop out. +// +// Returns `{ tables, tableCountScope }` where scope ∈ +// "site-referenced" | "manifest-only" | "unavailable". +// On no local signal we return zero tables (NEVER a prefix dump) so a missing +// `.powerpages-site/` degrades safe instead of inflating the plan. +async function discoverTables(envUrl, token, { projectRoot, datamodelManifestPath } = {}) { + let customUnmanaged = []; + try { + customUnmanaged = await queryCustomUnmanagedTables(envUrl, token); + } catch { + customUnmanaged = []; } - // Query EntityDefinitions for custom unmanaged tables. - // Verified 2026-04-22 against org1e98cc97 (v9.2): EntityDefinitions does NOT - // support `$top` (returns 400 "The query parameter $top is not supported"). - // We filter server-side to IsCustomEntity=true to keep the payload bounded — - // there's still no client-side pagination needed for typical tenants. - const path = - `EntityDefinitions` + - `?$filter=IsCustomEntity eq true` + - `&$select=LogicalName,MetadataId,IsManaged,IsCustomEntity`; - const all = await collectPaginated(envUrl, path, token, 10); - const custom = all.filter((e) => e.IsCustomEntity === true && e.IsManaged === false); - const matchingPrefix = publisherPrefix - ? custom.filter((e) => (e.LogicalName || '').toLowerCase().startsWith(`${publisherPrefix.toLowerCase()}_`)) - : custom; + const { names, available, sources } = collectReferencedEntityNames({ projectRoot, datamodelManifestPath }); + + let scope; + let scoped = []; + if (!available) { + scope = 'unavailable'; + } else { + scoped = scopeCustomTables(names, customUnmanaged); + scope = sources.tablePermissions > 0 ? 'site-referenced' : 'manifest-only'; + } const byName = new Map(); - for (const t of [...manifestTables, ...matchingPrefix.map((e) => ({ - logicalName: e.LogicalName, - metadataId: e.MetadataId, - }))]) { - if (t.logicalName && !byName.has(t.logicalName)) byName.set(t.logicalName, t); + for (const t of scoped) { + if (t.logicalName && !byName.has(t.logicalName)) { + byName.set(t.logicalName, { logicalName: t.logicalName, metadataId: t.metadataId }); + } } - return Array.from(byName.values()); + return { tables: Array.from(byName.values()), tableCountScope: scope }; +} + +// Build the deduped, scoped dependency-edge list among the site's tables. +// Each edge `[a, b]` (lowercased logical names, a (t.logicalName || '').toLowerCase()).filter(Boolean)); + const seen = new Set(); + const edges = []; + const addEdge = (x, y) => { + const a = String(x || '').toLowerCase(); + const b = String(y || '').toLowerCase(); + if (!a || !b || a === b || !inSet.has(a) || !inSet.has(b)) return; + const key = a < b ? `${a}|${b}` : `${b}|${a}`; + if (seen.has(key)) return; + seen.add(key); + edges.push(a < b ? [a, b] : [b, a]); + }; + for (const t of tables) { + let rel; + try { + rel = await fetchTableRelationships(envUrl, t.logicalName, token); + } catch { + continue; // inaccessible table — skip its edges + } + for (const e of rel.oneToMany) addEdge(e.referencedEntity, e.referencingEntity); + for (const e of rel.manyToMany) addEdge(e.entity1, e.entity2); + } + return edges; } async function countAttributesForTables(envUrl, tables, token) { @@ -712,8 +749,14 @@ async function estimateSolutionSize({ envUrl, websiteRecordId, token, publisherP // (which includes them under componenttype 10428). const siteLanguages = await discoverPowerPageSiteLanguages(envUrl, websiteRecordId, resolved); - const tables = await discoverTables(envUrl, publisherPrefix, resolved, datamodelManifest); + const { tables, tableCountScope } = await discoverTables(envUrl, resolved, { + projectRoot, + datamodelManifestPath: datamodelManifest, + }); const schemaAttrCount = await countAttributesForTables(envUrl, tables, resolved); + // Dependency edges among the scoped tables — drives the schema-split clustering + // so related tables ship in the same solution (never split a relationship). + const tableRelationships = await discoverTableRelationships(envUrl, tables, resolved); // Tenant-wide env var defs matching the publisher prefix. This is the // fallback used when no solution is set up yet (fresh project); for sites @@ -1024,6 +1067,10 @@ async function estimateSolutionSize({ envUrl, websiteRecordId, token, publisherP } : null, tableCount: tables.length, + // How the table set was scoped: "site-referenced" (table permissions), + // "manifest-only" (datamodel manifest, no permissions), or "unavailable" + // (no local signal — tableCount reflects 0, NOT a publisher-prefix dump). + tableCountScope, schemaAttrCount, webFilesAggregateMB: round1(webFilesAggregateBytes / (1024 * 1024)), webFilesIndividual: webMeasure.individual, @@ -1059,6 +1106,9 @@ async function estimateSolutionSize({ envUrl, websiteRecordId, token, publisherP mediaRatio: Math.round(webMeasure.mediaRatio * 100) / 100, siteType: 'code-site', tables: tables.map((t) => ({ logicalName: t.logicalName, attributeCount: t.attributeCount || 0 })), + // Dependency edges among the scoped tables ([a,b], lowercased, a} + */ +async function queryCustomUnmanagedTables(envUrl, token, request = helpers.makeRequest) { + const base = String(envUrl).replace(/\/+$/, ''); + const url = + `${base}/api/data/v9.2/EntityDefinitions` + + `?$filter=IsCustomEntity eq true` + + `&$select=LogicalName,MetadataId,SchemaName,DisplayName,IsManaged,IsCustomEntity`; + const rows = await helpers.odataGetAll(url, token, request); + return rows + .filter((e) => e && e.IsCustomEntity === true && e.IsManaged === false) + .map((e) => ({ + logicalName: e.LogicalName, + metadataId: e.MetadataId, + schemaName: e.SchemaName, + displayName: + (e.DisplayName && e.DisplayName.UserLocalizedLabel && e.DisplayName.UserLocalizedLabel.Label) || + e.SchemaName, + })); +} + +module.exports = { queryCustomUnmanagedTables }; diff --git a/plugins/power-pages/scripts/lib/query-table-relationships.js b/plugins/power-pages/scripts/lib/query-table-relationships.js new file mode 100644 index 000000000..11850f171 --- /dev/null +++ b/plugins/power-pages/scripts/lib/query-table-relationships.js @@ -0,0 +1,65 @@ +#!/usr/bin/env node + +// Shared Dataverse relationship queries for a table: lookups (OneToMany) + N:N +// (ManyToMany). Extracted from skills/audit-permissions/scripts/query-table-relationships.js +// so it can be require()d (that file is a self-executing CLI). The CLI is now a +// thin wrapper over this module. Used by: +// - estimate-solution-size.js — to build the dependency graph for the +// schema-split clustering (so related tables ship in the same solution). +// - audit-permissions — to validate contact/account/parent relationship scopes. + +'use strict'; + +const helpers = require('./validation-helpers'); + +/** + * Fetches OneToMany (lookup-backed) and ManyToMany relationships for a table. + * + * OneToMany errors propagate (a genuinely missing/inaccessible table is a real + * error the CLI surfaces via exit 1; the estimator wraps the call per-table for + * resilience). ManyToMany is best-effort — many tables/envs have no N:N and the + * navigation property can be finicky — so its errors are swallowed to `[]`. + * + * @param {string} envUrl - environment base URL + * @param {string} table - table logical name + * @param {string} token - bearer token + * @param {Function} [request=helpers.makeRequest] - injectable for tests + * @returns {Promise<{ + * oneToMany: { schemaName, referencedEntity, referencingEntity, referencingAttribute }[], + * manyToMany: { schemaName, entity1, entity2 }[] + * }>} + */ +async function fetchTableRelationships(envUrl, table, token, request = helpers.makeRequest) { + const base = String(envUrl).replace(/\/+$/, ''); + const safe = String(table).replace(/'/g, "''"); + + const o2mUrl = + `${base}/api/data/v9.2/EntityDefinitions(LogicalName='${safe}')/OneToManyRelationships` + + `?$select=SchemaName,ReferencedEntity,ReferencingEntity,ReferencingAttribute`; + const o2mRows = await helpers.odataGetAll(o2mUrl, token, request); + const oneToMany = o2mRows.map((r) => ({ + schemaName: r.SchemaName, + referencedEntity: r.ReferencedEntity, + referencingEntity: r.ReferencingEntity, + referencingAttribute: r.ReferencingAttribute, + })); + + let manyToMany = []; + try { + const m2mUrl = + `${base}/api/data/v9.2/EntityDefinitions(LogicalName='${safe}')/ManyToManyRelationships` + + `?$select=SchemaName,Entity1LogicalName,Entity2LogicalName`; + const m2mRows = await helpers.odataGetAll(m2mUrl, token, request); + manyToMany = m2mRows.map((r) => ({ + schemaName: r.SchemaName, + entity1: r.Entity1LogicalName, + entity2: r.Entity2LogicalName, + })); + } catch { + // N:N unavailable for this table/env — best-effort, leave empty. + } + + return { oneToMany, manyToMany }; +} + +module.exports = { fetchTableRelationships }; diff --git a/plugins/power-pages/scripts/lib/resolve-site-tables.js b/plugins/power-pages/scripts/lib/resolve-site-tables.js new file mode 100644 index 000000000..043f35322 --- /dev/null +++ b/plugins/power-pages/scripts/lib/resolve-site-tables.js @@ -0,0 +1,103 @@ +#!/usr/bin/env node + +// Resolves the custom Dataverse tables a Power Pages site ACTUALLY references, +// so ALM table discovery stops scooping up every table that merely shares the +// publisher prefix (the `new_` / default-publisher over-count bug). +// +// SME-confirmed source of truth: "We rely on Table permissions from the site. +// If a table is used in the site that means there will be permissions for it." +// So the site's table permissions (+ the datamodel manifest) are the complete +// list of tables the site uses — no need to also scan forms/lists. +// +// Two steps, kept separate so the Dataverse query (custom-unmanaged tables) can +// be supplied by the caller (estimate-solution-size.js / discover-site-components.js): +// 1. collectReferencedEntityNames({ projectRoot, datamodelManifestPath }) +// -> the set of entity logical names the site references (local read). +// 2. scopeCustomTables(referencedNames, customUnmanagedTables) +// -> the caller's custom-unmanaged table list, intersected with that set. +// +// Intersecting with custom-UNMANAGED tables drops standard tables (contact, +// annotation) and managed template tables — leaving exactly the tables the +// user's solution would own. + +'use strict'; + +const fs = require('fs'); +const path = require('path'); +const { loadTablePermissions } = require('./powerpages-config'); + +/** + * Collects the entity logical names referenced by the site's table permissions + * and its datamodel manifest (both local reads; no Dataverse). + * + * @param {object} opts + * @param {string} [opts.projectRoot] - site project root (contains .powerpages-site/) + * @param {string} [opts.datamodelManifestPath] - explicit manifest path (defaults to + * `/.datamodel-manifest.json`) + * @returns {{ names: Set, available: boolean, sources: { tablePermissions: number, manifest: number } }} + * `names` are lowercased. `available` is false only when neither a + * `.powerpages-site/table-permissions/` directory nor a manifest was found. + */ +function collectReferencedEntityNames({ projectRoot, datamodelManifestPath } = {}) { + const names = new Set(); + const sources = { tablePermissions: 0, manifest: 0 }; + let sawTablePermissionsDir = false; + let sawManifest = false; + + // 1. Table permissions — `entitylogicalname` per `*.tablepermission.yml`. + if (projectRoot) { + const dir = path.join(projectRoot, '.powerpages-site', 'table-permissions'); + if (fs.existsSync(dir)) { + sawTablePermissionsDir = true; + let records = []; + try { records = loadTablePermissions(dir); } catch { records = []; } + for (const r of records) { + const name = r && r.entitylogicalname; // NOT entityname (that's the display label) + if (typeof name === 'string' && name.trim()) { + names.add(name.trim().toLowerCase()); + sources.tablePermissions += 1; + } + } + } + } + + // 2. Datamodel manifest — tables created for this site by setup-datamodel. + const manifestPath = datamodelManifestPath || + (projectRoot ? path.join(projectRoot, '.datamodel-manifest.json') : null); + if (manifestPath && fs.existsSync(manifestPath)) { + sawManifest = true; + try { + const man = JSON.parse(fs.readFileSync(manifestPath, 'utf8')); + const entries = man.entities || man.tables || []; + for (const e of entries) { + const name = e && (e.logicalName || e.LogicalName || e.name); + if (typeof name === 'string' && name.trim()) { + names.add(name.trim().toLowerCase()); + sources.manifest += 1; + } + } + } catch { + // Malformed manifest — ignore its contents but still count it as a signal. + } + } + + return { names, available: sawTablePermissionsDir || sawManifest, sources }; +} + +/** + * Intersects the caller's custom-unmanaged table list with the referenced-name + * set. Returns the tables the site actually uses (and that the user's solution + * would own). + * + * @param {Set} referencedNames - lowercased logical names (from collectReferencedEntityNames) + * @param {{ logicalName: string }[]} customUnmanagedTables + * @returns {{ logicalName: string }[]} + */ +function scopeCustomTables(referencedNames, customUnmanagedTables) { + if (!referencedNames || referencedNames.size === 0) return []; + return (customUnmanagedTables || []).filter( + (t) => t && typeof t.logicalName === 'string' && referencedNames.has(t.logicalName.toLowerCase()), + ); +} + +module.exports = { collectReferencedEntityNames, scopeCustomTables }; diff --git a/plugins/power-pages/scripts/lib/validation-helpers.js b/plugins/power-pages/scripts/lib/validation-helpers.js index 28037792e..8a3b594cc 100644 --- a/plugins/power-pages/scripts/lib/validation-helpers.js +++ b/plugins/power-pages/scripts/lib/validation-helpers.js @@ -236,6 +236,53 @@ function makeRequest({ url, method = 'GET', headers = {}, body = null, includeHe }); } +/** + * Single Dataverse OData GET (v9.2 headers, `Prefer: odata.maxpagesize=5000`), + * throws on non-2xx. `url` is absolute — pass an `@odata.nextLink` straight back in. + * @param {string} url - absolute URL + * @param {string} token - bearer token + * @param {Function} [request=makeRequest] - injectable for tests + * @returns {Promise} parsed JSON body + */ +async function odataGet(url, token, request = makeRequest) { + const res = await request({ + url, + headers: { + Authorization: `Bearer ${token}`, + Accept: 'application/json', + 'OData-MaxVersion': '4.0', + 'OData-Version': '4.0', + Prefer: 'odata.maxpagesize=5000', + }, + timeout: 30000, + }); + if (res.error) throw new Error(`OData request failed: ${res.error}`); + if (res.statusCode < 200 || res.statusCode >= 300) { + throw new Error(`HTTP ${res.statusCode} from ${url}: ${(res.body || '').slice(0, 400)}`); + } + return JSON.parse(res.body); +} + +/** + * Follows `@odata.nextLink`, aggregating every page's `value[]` into one array. + * `maxPages` is a runaway-loop safety cap (100 × 5000 ≈ 500K rows). + * @param {string} url - absolute starting URL + * @param {string} token - bearer token + * @param {Function} [request=makeRequest] - injectable for tests + * @param {number} [maxPages=100] + * @returns {Promise} + */ +async function odataGetAll(url, token, request = makeRequest, maxPages = 100) { + const out = []; + let next = url; + for (let p = 0; p < maxPages && next; p++) { + const page = await odataGet(next, token, request); + if (Array.isArray(page.value)) out.push(...page.value); + next = page['@odata.nextLink'] || null; + } + return out; +} + /** Cloud → Power Platform API base URL mapping */ const CLOUD_TO_API = { 'Public': 'https://api.powerplatform.com', @@ -265,6 +312,8 @@ module.exports = { UUID_REGEX, getAuthToken, makeRequest, + odataGet, + odataGetAll, getEnvironmentUrl, getPacAuthInfo, CLOUD_TO_API, diff --git a/plugins/power-pages/scripts/tests/compute-split-plan.test.js b/plugins/power-pages/scripts/tests/compute-split-plan.test.js index 8fda1569b..48c92c03a 100644 --- a/plugins/power-pages/scripts/tests/compute-split-plan.test.js +++ b/plugins/power-pages/scripts/tests/compute-split-plan.test.js @@ -183,23 +183,26 @@ test('computeSplitPlan Strategy 3 uses explicit config.domains when present', () assert.equal(result.proposedSolutions[3].isFutureBuffer, true); }); -test('computeSplitPlan Strategy 3 falls back to prefix heuristic when no domains configured', () => { +test('computeSplitPlan Strategy 3 packs tables by capacity (no domains configured) — bounded, not per-table', () => { const result = computeSplitPlan({ estimate: baseEstimate({ tableCount: 22, - schemaAttrCount: 16000, + schemaAttrCount: 16000, // > maxSchemaAttrs(15000) -> 2 buckets by attrs tables: [ - { logicalName: 'tst_product' }, - { logicalName: 'tst_productVariant' }, - { logicalName: 'tst_order' }, - { logicalName: 'tst_orderLine' }, + { logicalName: 'tst_product', attributeCount: 4000 }, + { logicalName: 'tst_productVariant', attributeCount: 4000 }, + { logicalName: 'tst_order', attributeCount: 4000 }, + { logicalName: 'tst_orderLine', attributeCount: 4000 }, ], }), config: baseConfig(), meta: { baseName: 'Test', siteName: 'Test Site' }, }); assert.equal(result.splitStrategy, 'strategy-3-schema-segmentation'); - assert.ok(result.proposedSolutions.length >= 2); + const tableSolutions = result.proposedSolutions.filter( + (s) => Array.isArray(s.componentTypes) && s.componentTypes.length === 1 && s.componentTypes[0] === 'Table', + ); + assert.equal(tableSolutions.length, 2, '16000 attrs / 15000 cap -> 2 Table solutions, not one-per-table'); }); test('computeSplitPlan additive Strategy 4 prepends EnvVars solution', () => { @@ -601,3 +604,87 @@ test('partitionBySchema uses breakdown.tables to size domain solutions', () => { // Site solution absorbs the remainder assert.equal(solutions[2].sizeMB, 60); }); + +// --- dependency-aware capacity packing (the "21 solutions" fix) -------------- + +function makeTables(n, attrsEach, prefix = 'tbl') { + return Array.from({ length: n }, (_, i) => ({ logicalName: `${prefix}_${i}`, attributeCount: attrsEach })); +} + +test('Strategy 3: 34 distinct tables / 32.3k cols -> a HANDFUL of solutions, never ~34', () => { + const tables = makeTables(34, 950); // 34 * 950 = 32300 + const result = computeSplitPlan({ + estimate: baseEstimate({ totalSizeMB: 68, tableCount: 34, schemaAttrCount: 32300, componentCountSiteTotal: 3000, tables, tableRelationships: [] }), + config: baseConfig(), + meta: { baseName: 'Big', siteName: 'Big Site' }, + }); + assert.equal(result.splitStrategy, 'strategy-3-schema-segmentation'); + const tableSolutions = result.proposedSolutions.filter((s) => s.componentTypes && s.componentTypes[0] === 'Table' && s.componentTypes.length === 1); + assert.ok(tableSolutions.length >= 2 && tableSolutions.length <= 8, `expected a handful of Table solutions, got ${tableSolutions.length}`); + // Every Table solution stays under the per-solution table cap. + for (const s of tableSolutions) { + assert.ok(s.tableLogicalNames.length <= 20, `Table solution ${s.uniqueName} has ${s.tableLogicalNames.length} tables (> cap)`); + } + // All 34 tables are placed exactly once across the Table solutions. + const placed = tableSolutions.flatMap((s) => s.tableLogicalNames); + assert.equal(placed.length, 34); + assert.equal(new Set(placed).size, 34); +}); + +test('Strategy 3: 22 tables with low cols -> 2 Table solutions (count-driven), not 22', () => { + const tables = makeTables(22, 50); + const result = computeSplitPlan({ + estimate: baseEstimate({ tableCount: 22, schemaAttrCount: 1100, tables, tableRelationships: [] }), + config: baseConfig(), + meta: { baseName: 'M', siteName: 'M' }, + }); + assert.equal(result.splitStrategy, 'strategy-3-schema-segmentation'); + const tableSolutions = result.proposedSolutions.filter((s) => s.componentTypes && s.componentTypes[0] === 'Table' && s.componentTypes.length === 1); + assert.equal(tableSolutions.length, 2, '22 tables / 20-per-solution -> 2 Table solutions'); +}); + +test('Strategy 3 does NOT trigger for <=20 tables with low cols -> single', () => { + const tables = makeTables(18, 50); + const { primary } = selectStrategy(baseEstimate({ tableCount: 18, schemaAttrCount: 900, tables }), baseConfig()); + assert.equal(primary, 'single'); +}); + +test('Strategy 3: dependency clusters are never split across solutions', () => { + // Cluster A (4 tables) + Cluster B (2 tables) + 20 standalone = 26 tables, low cols. + const clusterA = ['rel_a0', 'rel_a1', 'rel_a2', 'rel_a3']; + const clusterB = ['rel_b0', 'rel_b1']; + const standalone = makeTables(20, 50, 'solo').map((t) => t.logicalName); + const tables = [...clusterA, ...clusterB, ...standalone].map((n) => ({ logicalName: n, attributeCount: 50 })); + const edges = [ + ['rel_a0', 'rel_a1'], ['rel_a1', 'rel_a2'], ['rel_a2', 'rel_a3'], // A connected + ['rel_b0', 'rel_b1'], // B connected + ]; + const result = computeSplitPlan({ + estimate: baseEstimate({ tableCount: 26, schemaAttrCount: 1300, tables, tableRelationships: edges }), + config: baseConfig(), + meta: { baseName: 'Dep', siteName: 'Dep' }, + }); + assert.equal(result.splitStrategy, 'strategy-3-schema-segmentation'); + const tableSolutions = result.proposedSolutions.filter((s) => s.componentTypes && s.componentTypes[0] === 'Table' && s.componentTypes.length === 1); + const home = (name) => tableSolutions.findIndex((s) => s.tableLogicalNames.includes(name)); + // Every table in cluster A shares one solution; same for B. + assert.ok(home('rel_a0') !== -1); + assert.ok(clusterA.every((n) => home(n) === home('rel_a0')), 'cluster A must not be split across solutions'); + assert.ok(clusterB.every((n) => home(n) === home('rel_b0')), 'cluster B must not be split across solutions'); +}); + +test('Strategy 3: an oversized single cluster (>cap) stays whole + raises a warning', () => { + // 25 tables all chained into ONE connected cluster -> cannot be split. + const names = makeTables(25, 50, 'big').map((t) => t.logicalName); + const tables = names.map((n) => ({ logicalName: n, attributeCount: 50 })); + const edges = names.slice(1).map((n, i) => [names[i], n]); // chain a0-a1-a2-...-a24 + const result = computeSplitPlan({ + estimate: baseEstimate({ tableCount: 25, schemaAttrCount: 1250, tables, tableRelationships: edges }), + config: baseConfig(), + meta: { baseName: 'Mega', siteName: 'Mega' }, + }); + const tableSolutions = result.proposedSolutions.filter((s) => s.componentTypes && s.componentTypes[0] === 'Table' && s.componentTypes.length === 1); + assert.equal(tableSolutions.length, 1, 'one indivisible cluster -> one Table solution'); + assert.equal(tableSolutions[0].tableLogicalNames.length, 25); + assert.ok(result.recommendations.some((r) => /dependency cluster that cannot be split/.test(r.message)), 'oversized-cluster warning must fire'); +}); diff --git a/plugins/power-pages/scripts/tests/discover-site-components.test.js b/plugins/power-pages/scripts/tests/discover-site-components.test.js index 320c89289..47ea7328e 100644 --- a/plugins/power-pages/scripts/tests/discover-site-components.test.js +++ b/plugins/power-pages/scripts/tests/discover-site-components.test.js @@ -2,6 +2,9 @@ const test = require('node:test'); const assert = require('node:assert/strict'); +const fs = require('fs'); +const os = require('os'); +const path = require('path'); const { discoverSiteComponents, @@ -9,6 +12,21 @@ const { PPC_DEFAULT_INCLUDE, } = require('../lib/discover-site-components'); +// Creates a temp site root whose `.powerpages-site/table-permissions/` references +// the given entity logical names — the SME-confirmed signal for "tables the site +// uses" that now scopes custom-table discovery (replacing the publisher-prefix dump). +function tempSiteWithTablePermissions(t, entityLogicalNames) { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'dsc-site-')); + t.after(() => fs.rmSync(root, { recursive: true, force: true })); + const dir = path.join(root, '.powerpages-site', 'table-permissions'); + fs.mkdirSync(dir, { recursive: true }); + entityLogicalNames.forEach((entity, i) => { + fs.writeFileSync(path.join(dir, `perm-${i}.tablepermission.yml`), + `adx_entitypermission_webrole:\n- ad89f5ee-8665-f111-a826-6045bd00fdda\nentitylogicalname: ${entity}\nentityname: Perm ${i}\nid: f03fefed-8665-f111-a826-000d3a597e6a\nscope: 756150000\n`); + }); + return root; +} + /** * Creates a fake `makeRequest` that matches URL fragments to response bodies. * Each entry is `[urlFragment, responseObject]`; the first matching entry wins. @@ -169,7 +187,7 @@ test('computes missing[] diff against an existing solution', async () => { ); }); -test('diffs custom tables by MetadataId against solutioncomponents.objectid', async () => { +test('diffs custom tables by MetadataId against solutioncomponents.objectid', async (t) => { const makeRequest = fakeRequest([ ['/powerpagecomponents?', { value: [] }], ['/workflows?', { value: [] }], @@ -183,12 +201,16 @@ test('diffs custom tables by MetadataId against solutioncomponents.objectid', as LogicalName: 'crd50_already', SchemaName: 'crd50_Already', DisplayName: { UserLocalizedLabel: { Label: 'Already' } }, + IsCustomEntity: true, + IsManaged: false, }, { MetadataId: 'meta-missing', LogicalName: 'crd50_missing', SchemaName: 'crd50_Missing', DisplayName: { UserLocalizedLabel: { Label: 'Missing' } }, + IsCustomEntity: true, + IsManaged: false, }, ], }, @@ -199,12 +221,14 @@ test('diffs custom tables by MetadataId against solutioncomponents.objectid', as ], ]); + const projectRoot = tempSiteWithTablePermissions(t, ['crd50_already', 'crd50_missing']); const result = await discoverSiteComponents({ envUrl: 'https://example.crm.dynamics.com', token: 'tok', siteId: 'site-guid', publisherPrefix: 'crd50', solutionId: 'sol-guid', + projectRoot, makeRequest, }); @@ -258,7 +282,7 @@ test('matching is case-insensitive on solution object IDs', async () => { assert.equal(result.missing.powerpagecomponents.length, 0); }); -test('discovers env vars and custom tables when publisherPrefix is passed', async () => { +test('discovers env vars and scopes custom tables to site references (not publisher prefix)', async (t) => { const makeRequest = fakeRequest([ ['/powerpagecomponents?', { value: [] }], ['/workflows?', { value: [] }], @@ -288,24 +312,34 @@ test('discovers env vars and custom tables when publisherPrefix is passed', asyn LogicalName: 'crd50_invoice', SchemaName: 'crd50_Invoice', DisplayName: { UserLocalizedLabel: { Label: 'Invoice' } }, + IsCustomEntity: true, + IsManaged: false, }, { MetadataId: 'meta-other-widget', - // A custom table from a different publisher — must be filtered out client-side. + // A custom table the site does NOT reference — dropped by the site-reference + // scoping (not by prefix). Even shares no prefix concern: it's simply unused. LogicalName: 'other_widget', SchemaName: 'other_Widget', DisplayName: { UserLocalizedLabel: { Label: 'Widget' } }, + IsCustomEntity: true, + IsManaged: false, }, ], }, ], ]); + // Site references ONLY crd50_invoice (not other_widget). The scoping must keep + // crd50_invoice and drop other_widget — because it isn't referenced, NOT because + // of its prefix (the whole point of the fix). + const projectRoot = tempSiteWithTablePermissions(t, ['crd50_invoice']); const result = await discoverSiteComponents({ envUrl: 'https://example.crm.dynamics.com', token: 'tok', siteId: 'site-guid', publisherPrefix: 'crd50', + projectRoot, makeRequest, }); diff --git a/plugins/power-pages/scripts/tests/estimate-solution-size.test.js b/plugins/power-pages/scripts/tests/estimate-solution-size.test.js index 8432c5240..78055b750 100644 --- a/plugins/power-pages/scripts/tests/estimate-solution-size.test.js +++ b/plugins/power-pages/scripts/tests/estimate-solution-size.test.js @@ -705,3 +705,87 @@ test('disk-measurement gracefully no-ops when projectRoot has no build-output di 'disk-vs-dataverse canary must not fire when no build dir found', ); }); + +// --- site-referenced table scoping + dependency edges (the prefix-overcount fix) --- + +test('estimateSolutionSize scopes tables to site references (not publisher prefix) + emits relationships', async (t) => { + const fs = require('fs'); + const os = require('os'); + const path = require('path'); + + // Temp site root with table permissions referencing only bp_permit + bp_permitstep. + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'est-scope-')); + t.after(() => fs.rmSync(root, { recursive: true, force: true })); + const tpDir = path.join(root, '.powerpages-site', 'table-permissions'); + fs.mkdirSync(tpDir, { recursive: true }); + for (const [base, entity] of [['Permit', 'bp_permit'], ['Step', 'bp_permitstep'], ['Notes', 'annotation']]) { + fs.writeFileSync(path.join(tpDir, `${base}.tablepermission.yml`), + `adx_entitypermission_webrole:\n- ad89f5ee-8665-f111-a826-6045bd00fdda\nentitylogicalname: ${entity}\nentityname: ${base}\nid: f03fefed-8665-f111-a826-000d3a597e6a\nscope: 756150000\n`); + } + + withMockedMakeRequest(t, async ({ url }) => { + if (url.includes('OneToManyRelationships')) { + // bp_permit has a lookup to bp_permitstep (both in scope) + one to contact (out of scope). + if (url.includes("LogicalName='bp_permit'")) { + return { statusCode: 200, body: JSON.stringify({ value: [ + { SchemaName: 'bp_permit_step', ReferencedEntity: 'bp_permit', ReferencingEntity: 'bp_permitstep', ReferencingAttribute: 'bp_permitid' }, + { SchemaName: 'contact_permit', ReferencedEntity: 'contact', ReferencingEntity: 'bp_permit', ReferencingAttribute: 'bp_contactid' }, + ] }) }; + } + return { statusCode: 200, body: JSON.stringify({ value: [] }) }; + } + if (url.includes('ManyToManyRelationships')) return { statusCode: 200, body: JSON.stringify({ value: [] }) }; + if (url.includes('/Attributes')) return { statusCode: 200, body: JSON.stringify({ value: [{ LogicalName: 'c1' }, { LogicalName: 'c2' }] }) }; + if (url.includes('EntityDefinitions') && url.includes('IsCustomEntity')) { + // Env has 4 custom tables; only bp_* are referenced. new_* are the prefix-noise. + return { statusCode: 200, body: JSON.stringify({ value: [ + { LogicalName: 'bp_permit', MetadataId: 'm1', IsCustomEntity: true, IsManaged: false }, + { LogicalName: 'bp_permitstep', MetadataId: 'm2', IsCustomEntity: true, IsManaged: false }, + { LogicalName: 'new_unrelated1', MetadataId: 'm3', IsCustomEntity: true, IsManaged: false }, + { LogicalName: 'new_unrelated2', MetadataId: 'm4', IsCustomEntity: true, IsManaged: false }, + ] }) }; + } + if (url.includes('powerpagecomponents') && url.includes('$count')) { + return { statusCode: 200, body: JSON.stringify({ '@odata.count': 0, value: [] }) }; + } + return { statusCode: 200, body: JSON.stringify({ value: [] }) }; + }); + + const result = await estimateSolutionSize({ + envUrl: 'https://test.crm.dynamics.com', + websiteRecordId: '00000000-0000-0000-0000-000000000001', + publisherPrefix: 'new', // the (now-irrelevant for tables) shared prefix + projectRoot: root, + token: 'fake-token', + }); + + assert.equal(result.tableCount, 2, 'only the 2 site-referenced bp_* tables — NOT the 2 new_* prefix matches'); + assert.equal(result.tableCountScope, 'site-referenced'); + assert.deepEqual(result.tables.map((x) => x.logicalName).sort(), ['bp_permit', 'bp_permitstep']); + // Edge between the two scoped tables; the contact edge is dropped (out of scope). + assert.deepEqual(result.tableRelationships, [['bp_permit', 'bp_permitstep']]); +}); + +test('estimateSolutionSize tableCountScope is "unavailable" with no local signal (never a prefix dump)', async (t) => { + withMockedMakeRequest(t, async ({ url }) => { + if (url.includes('EntityDefinitions') && url.includes('IsCustomEntity')) { + return { statusCode: 200, body: JSON.stringify({ value: [ + { LogicalName: 'new_a', MetadataId: 'm1', IsCustomEntity: true, IsManaged: false }, + { LogicalName: 'new_b', MetadataId: 'm2', IsCustomEntity: true, IsManaged: false }, + ] }) }; + } + if (url.includes('powerpagecomponents') && url.includes('$count')) { + return { statusCode: 200, body: JSON.stringify({ '@odata.count': 0, value: [] }) }; + } + return { statusCode: 200, body: JSON.stringify({ value: [] }) }; + }); + const result = await estimateSolutionSize({ + envUrl: 'https://test.crm.dynamics.com', + websiteRecordId: '00000000-0000-0000-0000-000000000001', + publisherPrefix: 'new', + token: 'fake-token', // no projectRoot + }); + assert.equal(result.tableCount, 0, 'no .powerpages-site signal -> zero tables, NOT the env-wide prefix dump'); + assert.equal(result.tableCountScope, 'unavailable'); + assert.deepEqual(result.tableRelationships, []); +}); diff --git a/plugins/power-pages/scripts/tests/integration/discover-integration.test.js b/plugins/power-pages/scripts/tests/integration/discover-integration.test.js index 57e925265..e7a317663 100644 --- a/plugins/power-pages/scripts/tests/integration/discover-integration.test.js +++ b/plugins/power-pages/scripts/tests/integration/discover-integration.test.js @@ -6,11 +6,27 @@ const test = require('node:test'); const assert = require('node:assert/strict'); +const fs = require('fs'); +const os = require('os'); +const path = require('path'); const { startMock } = require('./mock-dataverse'); const { discoverSiteComponents, } = require('../../lib/discover-site-components'); +// Temp site root whose table permissions reference the given entities — the +// site-referenced scoping signal that replaced the publisher-prefix table dump. +function makeSiteRoot(entityLogicalNames) { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'dsc-int-')); + const dir = path.join(root, '.powerpages-site', 'table-permissions'); + fs.mkdirSync(dir, { recursive: true }); + entityLogicalNames.forEach((entity, i) => { + fs.writeFileSync(path.join(dir, `perm-${i}.tablepermission.yml`), + `adx_entitypermission_webrole:\n- ad89f5ee-8665-f111-a826-6045bd00fdda\nentitylogicalname: ${entity}\nentityname: Perm ${i}\nid: f03fefed-8665-f111-a826-000d3a597e6a\nscope: 756150000\n`); + }); + return root; +} + test('integration: discover follows @odata.nextLink pagination against a real HTTP server', async () => { let mockBase = null; @@ -217,23 +233,30 @@ test('integration: discover with publisherPrefix queries env vars + tables endpo LogicalName: 'contoso_account', SchemaName: 'contoso_Account', DisplayName: { UserLocalizedLabel: { Label: 'Account' } }, + IsCustomEntity: true, + IsManaged: false, }, { MetadataId: 'meta-2', LogicalName: 'other_widget', SchemaName: 'other_Widget', DisplayName: { UserLocalizedLabel: { Label: 'Widget' } }, + IsCustomEntity: true, + IsManaged: false, }, ], }, }, ]); + // Site references contoso_account only -> other_widget is dropped (unreferenced). + const projectRoot = makeSiteRoot(['contoso_account']); try { const result = await discoverSiteComponents({ envUrl: mock.baseUrl, token: 'x', siteId: 'site-42', publisherPrefix: 'contoso', + projectRoot, }); assert.equal(result.envVars.length, 1); assert.equal(result.envVars[0].schemaName, 'contoso_FeatureFlag'); @@ -246,5 +269,6 @@ test('integration: discover with publisherPrefix queries env vars + tables endpo assert.equal(edCalls.length, 1); } finally { await mock.close(); + fs.rmSync(projectRoot, { recursive: true, force: true }); } }); diff --git a/plugins/power-pages/scripts/tests/query-metadata.test.js b/plugins/power-pages/scripts/tests/query-metadata.test.js new file mode 100644 index 000000000..9433a9f3c --- /dev/null +++ b/plugins/power-pages/scripts/tests/query-metadata.test.js @@ -0,0 +1,44 @@ +'use strict'; + +const test = require('node:test'); +const assert = require('node:assert/strict'); + +const { queryCustomUnmanagedTables } = require('../lib/query-metadata'); + +function fakeRequestReturning(rows, { paginate = false } = {}) { + let served = false; + return async ({ url }) => { + if (paginate && !served && !/page2/.test(url)) { + served = true; + return { + statusCode: 200, + body: JSON.stringify({ value: rows.slice(0, 1), '@odata.nextLink': 'https://x/page2' }), + }; + } + const body = paginate ? { value: rows.slice(1) } : { value: rows }; + return { statusCode: 200, body: JSON.stringify(body) }; + }; +} + +test('queryCustomUnmanagedTables keeps only custom + unmanaged tables (with schema/display)', async () => { + const req = fakeRequestReturning([ + { LogicalName: 'bp_permit', MetadataId: 'm1', SchemaName: 'bp_Permit', DisplayName: { UserLocalizedLabel: { Label: 'Permit' } }, IsCustomEntity: true, IsManaged: false }, + { LogicalName: 'bp_managed', MetadataId: 'm2', IsCustomEntity: true, IsManaged: true }, // managed -> dropped + { LogicalName: 'account', MetadataId: 'm3', IsCustomEntity: false, IsManaged: false }, // system -> dropped (not custom) + { LogicalName: 'bp_inspection', MetadataId: 'm4', SchemaName: 'bp_Inspection', IsCustomEntity: true, IsManaged: false }, // no DisplayName -> falls back to SchemaName + ]); + const out = await queryCustomUnmanagedTables('https://org.crm.dynamics.com/', 'tok', req); + assert.deepEqual(out, [ + { logicalName: 'bp_permit', metadataId: 'm1', schemaName: 'bp_Permit', displayName: 'Permit' }, + { logicalName: 'bp_inspection', metadataId: 'm4', schemaName: 'bp_Inspection', displayName: 'bp_Inspection' }, + ]); +}); + +test('queryCustomUnmanagedTables paginates via @odata.nextLink', async () => { + const req = fakeRequestReturning([ + { LogicalName: 'a_one', MetadataId: 'm1', IsCustomEntity: true, IsManaged: false }, + { LogicalName: 'a_two', MetadataId: 'm2', IsCustomEntity: true, IsManaged: false }, + ], { paginate: true }); + const out = await queryCustomUnmanagedTables('https://org.crm.dynamics.com', 'tok', req); + assert.deepEqual(out.map((t) => t.logicalName), ['a_one', 'a_two']); +}); diff --git a/plugins/power-pages/scripts/tests/query-table-relationships.test.js b/plugins/power-pages/scripts/tests/query-table-relationships.test.js new file mode 100644 index 000000000..7c5bd446f --- /dev/null +++ b/plugins/power-pages/scripts/tests/query-table-relationships.test.js @@ -0,0 +1,52 @@ +'use strict'; + +const test = require('node:test'); +const assert = require('node:assert/strict'); + +const { fetchTableRelationships } = require('../lib/query-table-relationships'); + +function router(map) { + // map: substring -> { statusCode, value } | throws if error:true + return async ({ url }) => { + for (const [needle, resp] of Object.entries(map)) { + if (url.includes(needle)) { + if (resp.error) return { error: resp.error }; + return { statusCode: resp.statusCode || 200, body: JSON.stringify({ value: resp.value || [] }) }; + } + } + return { statusCode: 200, body: JSON.stringify({ value: [] }) }; + }; +} + +test('fetchTableRelationships maps OneToMany + ManyToMany shapes', async () => { + const req = router({ + 'OneToManyRelationships': { + value: [{ SchemaName: 'bp_permit_step', ReferencedEntity: 'bp_permit', ReferencingEntity: 'bp_step', ReferencingAttribute: 'bp_permitid' }], + }, + 'ManyToManyRelationships': { + value: [{ SchemaName: 'bp_permit_tag', Entity1LogicalName: 'bp_permit', Entity2LogicalName: 'bp_tag' }], + }, + }); + const out = await fetchTableRelationships('https://org.crm.dynamics.com/', 'bp_permit', 'tok', req); + assert.deepEqual(out.oneToMany, [ + { schemaName: 'bp_permit_step', referencedEntity: 'bp_permit', referencingEntity: 'bp_step', referencingAttribute: 'bp_permitid' }, + ]); + assert.deepEqual(out.manyToMany, [ + { schemaName: 'bp_permit_tag', entity1: 'bp_permit', entity2: 'bp_tag' }, + ]); +}); + +test('fetchTableRelationships swallows ManyToMany errors (best-effort)', async () => { + const req = router({ + 'OneToManyRelationships': { value: [] }, + 'ManyToManyRelationships': { statusCode: 404, value: [] }, + }); + const out = await fetchTableRelationships('https://org.crm.dynamics.com', 'x_t', 'tok', req); + assert.deepEqual(out.manyToMany, []); + assert.deepEqual(out.oneToMany, []); +}); + +test('fetchTableRelationships propagates OneToMany errors', async () => { + const req = router({ 'OneToManyRelationships': { statusCode: 404, value: [] } }); + await assert.rejects(() => fetchTableRelationships('https://org.crm.dynamics.com', 'missing', 'tok', req), /HTTP 404/); +}); diff --git a/plugins/power-pages/scripts/tests/resolve-site-tables.test.js b/plugins/power-pages/scripts/tests/resolve-site-tables.test.js new file mode 100644 index 000000000..7dfebc315 --- /dev/null +++ b/plugins/power-pages/scripts/tests/resolve-site-tables.test.js @@ -0,0 +1,98 @@ +'use strict'; + +const test = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('fs'); +const os = require('os'); +const path = require('path'); + +const { collectReferencedEntityNames, scopeCustomTables } = require('../lib/resolve-site-tables'); + +function makeProject(t) { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'resolve-site-tables-')); + t.after(() => fs.rmSync(root, { recursive: true, force: true })); + return root; +} + +function writeTablePermission(root, fileBase, entityLogicalName, displayName) { + const dir = path.join(root, '.powerpages-site', 'table-permissions'); + fs.mkdirSync(dir, { recursive: true }); + fs.writeFileSync( + path.join(dir, `${fileBase}.tablepermission.yml`), + [ + 'adx_entitypermission_webrole:', + '- ad89f5ee-8665-f111-a826-6045bd00fdda', + 'append: true', + 'appendto: true', + 'create: true', + 'delete: true', + `entitylogicalname: ${entityLogicalName}`, + `entityname: ${displayName}`, + 'id: f03fefed-8665-f111-a826-000d3a597e6a', + 'read: true', + 'scope: 756150000', + 'write: true', + '', + ].join('\n'), + ); +} + +test('collectReferencedEntityNames: extracts entitylogicalname from table permissions (EDM bp_* fixture)', (t) => { + const root = makeProject(t); + writeTablePermission(root, 'Admin-to-Permits', 'bp_defaultapplication', 'Admin to Permits'); + writeTablePermission(root, 'Permit-Steps', 'bp_permitstep', 'Permit Steps'); + writeTablePermission(root, 'Notes-Global', 'annotation', 'Notes Global'); // standard table + + const { names, available, sources } = collectReferencedEntityNames({ projectRoot: root }); + assert.equal(available, true); + assert.equal(sources.tablePermissions, 3); + assert.ok(names.has('bp_defaultapplication')); + assert.ok(names.has('bp_permitstep')); + assert.ok(names.has('annotation')); + // entityname (display label) must NOT be added. + assert.ok(!names.has('admin to permits')); +}); + +test('collectReferencedEntityNames: unions datamodel manifest entities', (t) => { + const root = makeProject(t); + writeTablePermission(root, 'Permit', 'bp_permit', 'Permit'); + fs.writeFileSync( + path.join(root, '.datamodel-manifest.json'), + JSON.stringify({ entities: [{ logicalName: 'new_extra' }, { logicalName: 'bp_permit' }] }), + ); + const { names, sources } = collectReferencedEntityNames({ projectRoot: root }); + assert.ok(names.has('bp_permit')); + assert.ok(names.has('new_extra')); + assert.ok(sources.manifest >= 1); +}); + +test('collectReferencedEntityNames: available=false when no .powerpages-site and no manifest', (t) => { + const root = makeProject(t); + const { names, available } = collectReferencedEntityNames({ projectRoot: root }); + assert.equal(available, false); + assert.equal(names.size, 0); +}); + +test('scopeCustomTables: keeps only referenced custom tables; drops unreferenced + standard', (t) => { + const root = makeProject(t); + writeTablePermission(root, 'Permit', 'bp_permit', 'Permit'); + writeTablePermission(root, 'Inspection', 'BP_Inspection', 'Inspection'); // mixed case + writeTablePermission(root, 'Notes', 'annotation', 'Notes'); // standard + + const { names } = collectReferencedEntityNames({ projectRoot: root }); + + // Simulate the env's custom-unmanaged tables (the old prefix dump would return all of these). + const customUnmanaged = [ + { logicalName: 'bp_permit', metadataId: 'm1' }, + { logicalName: 'bp_inspection', metadataId: 'm2' }, + { logicalName: 'new_unrelated1', metadataId: 'm3' }, // not referenced -> dropped + { logicalName: 'new_unrelated2', metadataId: 'm4' }, // not referenced -> dropped + ]; + const scoped = scopeCustomTables(names, customUnmanaged).map((t2) => t2.logicalName).sort(); + assert.deepEqual(scoped, ['bp_inspection', 'bp_permit']); + // 'annotation' is referenced but not in the custom-unmanaged list -> naturally excluded. +}); + +test('scopeCustomTables: empty referenced set -> empty (never a prefix dump)', () => { + assert.deepEqual(scopeCustomTables(new Set(), [{ logicalName: 'new_x' }]), []); +}); diff --git a/plugins/power-pages/scripts/tests/validation-helpers.test.js b/plugins/power-pages/scripts/tests/validation-helpers.test.js index 9b7e2dc87..d48411ca8 100644 --- a/plugins/power-pages/scripts/tests/validation-helpers.test.js +++ b/plugins/power-pages/scripts/tests/validation-helpers.test.js @@ -73,3 +73,28 @@ test('findProjectRoot: returns null when neither marker is present', (t) => { assert.equal(findProjectRoot(root), null); }); +// --- odataGet / odataGetAll (shared pagination) ------------------------------ + +test('odataGetAll follows @odata.nextLink and aggregates all pages', async () => { + const { odataGetAll } = require(helpersPath); + const pages = { + 'https://x/api/data/v9.2/things': { value: [{ id: 1 }, { id: 2 }], '@odata.nextLink': 'https://x/page2' }, + 'https://x/page2': { value: [{ id: 3 }] }, + }; + const fakeRequest = async ({ url }) => ({ statusCode: 200, body: JSON.stringify(pages[url]) }); + const rows = await odataGetAll('https://x/api/data/v9.2/things', 'tok', fakeRequest); + assert.deepEqual(rows.map((r) => r.id), [1, 2, 3]); +}); + +test('odataGet throws on non-2xx', async () => { + const { odataGet } = require(helpersPath); + const fakeRequest = async () => ({ statusCode: 404, body: 'not found' }); + await assert.rejects(() => odataGet('https://x/y', 'tok', fakeRequest), /HTTP 404/); +}); + +test('odataGet throws on transport error', async () => { + const { odataGet } = require(helpersPath); + const fakeRequest = async () => ({ error: 'ECONNRESET' }); + await assert.rejects(() => odataGet('https://x/y', 'tok', fakeRequest), /OData request failed/); +}); + diff --git a/plugins/power-pages/skills/audit-permissions/scripts/query-table-relationships.js b/plugins/power-pages/skills/audit-permissions/scripts/query-table-relationships.js index a5bdf93f3..ca98582dc 100644 --- a/plugins/power-pages/skills/audit-permissions/scripts/query-table-relationships.js +++ b/plugins/power-pages/skills/audit-permissions/scripts/query-table-relationships.js @@ -1,19 +1,21 @@ #!/usr/bin/env node +// Thin CLI wrapper over scripts/lib/query-table-relationships.js. // Queries Dataverse for one-to-many relationships on a given table. // Returns JSON array of { schemaName, referencedEntity, referencingEntity, referencingAttribute }. // // Usage: // node query-table-relationships.js --envUrl --table // -// Output (stdout): JSON array -// [{ "schemaName": "cr4fc_order_orderitem", "referencedEntity": "cr4fc_order", "referencingEntity": "cr4fc_orderitem", "referencingAttribute": "cr4fc_orderid" }] +// Output (stdout): JSON array (OneToMany relationships only — the audit-permissions +// relationship-scope validation consumes schemaName + referencedEntity). // // Exit codes: // 0 = success (JSON on stdout) // 1 = error (message on stderr) -const { getAuthToken, makeRequest } = require('../../../scripts/lib/validation-helpers'); +const { getAuthToken } = require('../../../scripts/lib/validation-helpers'); +const { fetchTableRelationships } = require('../../../scripts/lib/query-table-relationships'); const args = process.argv.slice(2); function getArg(name) { @@ -37,29 +39,10 @@ if (!envUrl || !table) { } try { - const result = await makeRequest({ - url: `${envUrl}/api/data/v9.2/EntityDefinitions(LogicalName='${table}')/OneToManyRelationships?$select=SchemaName,ReferencedEntity,ReferencingEntity,ReferencingAttribute`, - headers: { - Authorization: `Bearer ${token}`, - Accept: 'application/json', - }, - timeout: 15000, - }); - - if (result.error || result.statusCode !== 200) { - process.stderr.write(`API error (${result.statusCode}): ${result.error || result.body}\n`); - process.exit(1); - } - - const parsed = JSON.parse(result.body); - const rels = (parsed.value || []).map(r => ({ - schemaName: r.SchemaName, - referencedEntity: r.ReferencedEntity, - referencingEntity: r.ReferencingEntity, - referencingAttribute: r.ReferencingAttribute, - })); - - process.stdout.write(JSON.stringify(rels, null, 2) + '\n'); + // OneToMany errors propagate here (preserves the original exit-1-on-API-error + // behavior); ManyToMany is best-effort inside the lib and unused by this CLI. + const { oneToMany } = await fetchTableRelationships(envUrl, table, token); + process.stdout.write(JSON.stringify(oneToMany, null, 2) + '\n'); } catch (err) { process.stderr.write(`Request failed: ${err.message}\n`); process.exit(1); diff --git a/plugins/power-pages/skills/setup-solution/SKILL.md b/plugins/power-pages/skills/setup-solution/SKILL.md index 2c44aeb39..e46d11a3e 100644 --- a/plugins/power-pages/skills/setup-solution/SKILL.md +++ b/plugins/power-pages/skills/setup-solution/SKILL.md @@ -304,14 +304,18 @@ GET {envUrl}/api/data/v9.2/powerpagesitelanguages?$filter=_powerpagesiteid_value ``` Store all language IDs. -**D. Dataverse tables** — always discover from the environment, don't rely on a manifest file alone: +**D. Dataverse tables** — discover the tables the **site actually references**, NOT every table sharing the publisher prefix. -1. Read `.datamodel-manifest.json` if present (for the known list of tables created by `setup-datamodel`) -2. **Also** query the environment directly for all custom unmanaged tables, filtering by the publisher prefix: -``` -GET {envUrl}/api/data/v9.2/EntityDefinitions?$select=LogicalName,MetadataId,IsManaged,IsCustomEntity +> **Why not publisher prefix:** prefix-matching over-counts catastrophically with a shared/default publisher (`new_`, env default) — a 6-table site can match 22 unrelated tables — and it also *misses* the site's real tables when they come from a different prefix (e.g. a `bp_*` template under an `edm` publisher). The authoritative signal (SME-confirmed) is the site's **table permissions**: "If a table is used in the site there will be permissions for it." + +Run the shared discovery helper, which scopes custom tables to the site's table permissions (+ datamodel manifest) intersected with the env's custom-unmanaged tables: +```bash +node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/discover-site-components.js" \ + --envUrl "{envUrl}" --token "{token}" --siteId "{websiteRecordId}" \ + --projectRoot "." \ + {if .datamodel-manifest.json elsewhere: --datamodelManifest ""} ``` -Filter client-side: `IsCustomEntity === true && IsManaged === false`. Group by publisher prefix (characters before first `_`). Present only tables whose prefix matches the site publisher — or if no prefix match, present all custom unmanaged tables and let the user decide. +Use the returned `customTables[]` (each `{ id, logicalName, schemaName, displayName }` — `id` is the MetadataId for the `AddSolutionComponent` call). This is already the correct, site-scoped list — do **not** re-filter by prefix. > **Important note on tables**: Dataverse solutions carry **schema only** — entity definitions, columns, relationships, forms, and views. Table **data/records** do NOT travel with the solution. If the target environment needs seed/reference data, that requires a separate data migration step. @@ -628,11 +632,13 @@ If both `missing.powerpagecomponents` (after filtering) and `missing.siteLanguag **This is the key decision point.** Build a full manifest of everything that will be added and present it to the user before writing anything. -If custom tables were discovered, ask via `AskUserQuestion` with `multiSelect: true` **before** showing the final manifest: -- First option: **"Include all N tables (Recommended)"** — pre-selected default +If custom tables were discovered (the site-referenced set from step D), ask via `AskUserQuestion` with `multiSelect: true` **before** showing the final manifest: +- First option: **"Include all N referenced tables (Recommended)"** — pre-selected default. N is the count of tables the site actually references (not an env-wide prefix list). - Then one option per table: `{logicalName} ({DisplayName})` - Last option: **"Exclude all tables"** +> The default list is already scoped to the site's real tables (step D). If the user knows of an additional table the site needs that has no permission yet, they can add it manually — but the default must never be a publisher-prefix dump. + Present as a structured summary: ``` From be4a70de9593c07798c240e362f913967d520cad Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 14:41:54 +0530 Subject: [PATCH 20/38] Fix FFD bin-packing under-allocation that overflowed the schema-attr cap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit deriveDomainsByCapacity seeded the packer's bin count from a LOWER bound (max(ceil(tables/maxTableCount), ceil(attrs/maxSchemaAttrs))), so when independent (no-edge) clusters fragment, FFD ran out of bins and dropped the non-fitting cluster into the least-loaded bucket — overflowing it past maxSchemaAttrs with no warning (the oversized-cluster guard only checks per- cluster table COUNT, not attrs). Verified repro: 4 independent 8000-attr tables, maxSchemaAttrs 15000 -> seed n=3 -> one bucket holds 16000 attrs. Seed the packer with the maximum permitted bins instead (one per cluster, capped at maxSchemaSplitSolutions). FFD still consolidates — clusters that fit together share a bin and empty bins are dropped, so the solution count stays minimal — but a cluster that fits nowhere opens a NEW bin rather than overflowing. The existing 16000-attr/2-solution test is unchanged (FFD still consolidates); added a regression test for the 4-independent-table overflow case. 1211 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../scripts/lib/compute-split-plan.js | 16 +++++---- .../scripts/tests/compute-split-plan.test.js | 33 +++++++++++++++++++ 2 files changed, 42 insertions(+), 7 deletions(-) diff --git a/plugins/power-pages/scripts/lib/compute-split-plan.js b/plugins/power-pages/scripts/lib/compute-split-plan.js index f59028400..82305136e 100644 --- a/plugins/power-pages/scripts/lib/compute-split-plan.js +++ b/plugins/power-pages/scripts/lib/compute-split-plan.js @@ -427,14 +427,16 @@ function deriveDomainsByCapacity(estimate, thresholds) { if (tables.length === 0) return [{ name: 'Tables', tableLogicalNames: [] }]; const clusters = buildTableClusters(tables, estimate.tableRelationships || []); - const totalAttrs = tables.reduce((s, t) => s + t.attributeCount, 0); const ceiling = (thresholds && thresholds.maxSchemaSplitSolutions) || 8; - let n = Math.max( - 1, - Math.ceil(tables.length / thresholds.maxTableCount), - Math.ceil(totalAttrs / Math.max(thresholds.maxSchemaAttrs, 1)), - ); - n = Math.min(n, ceiling, clusters.length); + // Seed the packer with the maximum permitted bins (one per cluster, capped at + // maxSchemaSplitSolutions). First-fit-decreasing still consolidates — clusters + // that fit together share a bin and the empty bins are dropped, so the final + // count stays minimal — but a cluster that fits nowhere lands in a NEW bin + // instead of overflowing an existing one. Seeding from a lower bound + // (ceil(tables/maxTable), ceil(attrs/maxAttr)) under-allocated bins and let + // independent attr-heavy clusters bust maxSchemaAttrs in the least-loaded + // bucket, unwarned (the oversized guard only catches per-cluster table count). + const n = Math.min(clusters.length, ceiling); const buckets = packClusters(clusters, n, thresholds); const multi = buckets.length > 1; diff --git a/plugins/power-pages/scripts/tests/compute-split-plan.test.js b/plugins/power-pages/scripts/tests/compute-split-plan.test.js index 48c92c03a..abf985a37 100644 --- a/plugins/power-pages/scripts/tests/compute-split-plan.test.js +++ b/plugins/power-pages/scripts/tests/compute-split-plan.test.js @@ -205,6 +205,39 @@ test('computeSplitPlan Strategy 3 packs tables by capacity (no domains configure assert.equal(tableSolutions.length, 2, '16000 attrs / 15000 cap -> 2 Table solutions, not one-per-table'); }); +test('computeSplitPlan Strategy 3 never overflows the attr cap when independent clusters fragment', () => { + // Regression for the FFD under-allocation bug: 4 INDEPENDENT (no-edge) tables of + // 8000 attrs each = 32000 total. Seeding the packer from the lower bound + // ceil(32000/15000)=3 gave only 3 bins, so the 4th cluster fell into the + // least-loaded bucket -> 16000 attrs (> 15000 cap), unwarned. The packer must + // instead open a 4th bin (clusters.length permits it) so no bucket busts the cap. + const result = computeSplitPlan({ + estimate: baseEstimate({ + tableCount: 4, + schemaAttrCount: 32000, + tables: [ + { logicalName: 'tst_alpha', attributeCount: 8000 }, + { logicalName: 'tst_beta', attributeCount: 8000 }, + { logicalName: 'tst_gamma', attributeCount: 8000 }, + { logicalName: 'tst_delta', attributeCount: 8000 }, + ], + tableRelationships: [], // no edges -> 4 singleton clusters + }), + config: baseConfig(), + meta: { baseName: 'Test', siteName: 'Test Site' }, + }); + assert.equal(result.splitStrategy, 'strategy-3-schema-segmentation'); + const tableSolutions = result.proposedSolutions.filter( + (s) => Array.isArray(s.componentTypes) && s.componentTypes.length === 1 && s.componentTypes[0] === 'Table', + ); + // 4 independent 8000-attr tables -> 4 single-table solutions (each 8000 < 15000), + // NOT 3 with one 16000-attr overflow bucket. + assert.equal(tableSolutions.length, 4, '4 independent 8000-attr tables -> 4 Table solutions (no attr-cap overflow)'); + for (const s of tableSolutions) { + assert.equal(s.tableLogicalNames.length, 1, `${s.uniqueName} must hold exactly one table — no bucket over the attr cap`); + } +}); + test('computeSplitPlan additive Strategy 4 prepends EnvVars solution', () => { const result = computeSplitPlan({ estimate: baseEstimate({ totalSizeMB: 142, webFilesAggregateMB: 110, envVarCount: 800 }), From e55e06d4549b2913ceea926800484de6e93c233c Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Wed, 17 Jun 2026 18:16:00 +0530 Subject: [PATCH 21/38] Wire --projectRoot into the remaining discover-site-components consumers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The site-referenced table discovery (this PR) made discover-site-components return customTables (and thus missing.customTables) ONLY when --projectRoot (or --datamodelManifest) is passed — without a local signal it returns [] rather than the old publisher-prefix dump. setup-solution Step D was updated to pass --projectRoot, but three other consumers that read missing.customTables were missed, so their "custom tables missing from the solution" completeness check silently reported 0 for every site: - export-solution Phase 2.5 (pre-export completeness) - plan-alm Phase 1 (pre-plan completeness) - deploy-pipeline Phase 3.5 (pre-sync completeness) Added --projectRoot "." to all three so the check is restored with the correct site-scoped count. (setup-solution Phase 5.4b/5.4c calls consume missing.envVars / missing.powerpagecomponents / siteLanguages, not customTables, so they're correct as-is with --publisherPrefix/--solutionId.) 1211 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- plugins/power-pages/skills/deploy-pipeline/SKILL.md | 3 ++- plugins/power-pages/skills/export-solution/SKILL.md | 3 ++- plugins/power-pages/skills/plan-alm/SKILL.md | 3 ++- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/plugins/power-pages/skills/deploy-pipeline/SKILL.md b/plugins/power-pages/skills/deploy-pipeline/SKILL.md index 7465cc0c9..f4635a15d 100644 --- a/plugins/power-pages/skills/deploy-pipeline/SKILL.md +++ b/plugins/power-pages/skills/deploy-pipeline/SKILL.md @@ -300,7 +300,8 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/discover-site-components.js" \ --envUrl "{devEnvUrl}" --token "{DEV_TOKEN}" \ --siteId "{websiteRecordId from .solution-manifest.json}" \ --publisherPrefix "{publisherPrefix from .solution-manifest.json}" \ - --solutionId "{solutionId from .solution-manifest.json}" + --solutionId "{solutionId from .solution-manifest.json}" \ + --projectRoot "." ``` Parse stdout and evaluate `missing.*`. **Before doing anything else**, capture the **pre-sync state** so a post-sync re-confirmation gate can show what changed: diff --git a/plugins/power-pages/skills/export-solution/SKILL.md b/plugins/power-pages/skills/export-solution/SKILL.md index 489796e7d..409409e37 100644 --- a/plugins/power-pages/skills/export-solution/SKILL.md +++ b/plugins/power-pages/skills/export-solution/SKILL.md @@ -158,7 +158,8 @@ node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/discover-site-components.js" \ --envUrl "{envUrl}" --token "{token}" \ --siteId "{websiteRecordId}" \ --publisherPrefix "{publisherPrefix from .solution-manifest.json}" \ - --solutionId "{solutionId}" + --solutionId "{solutionId}" \ + --projectRoot "." ``` Parse stdout and evaluate `missing`. **Before doing anything else**, capture the **pre-sync state** so a post-sync re-confirmation gate can show what changed: diff --git a/plugins/power-pages/skills/plan-alm/SKILL.md b/plugins/power-pages/skills/plan-alm/SKILL.md index c36b2dd1d..ed7a34b32 100644 --- a/plugins/power-pages/skills/plan-alm/SKILL.md +++ b/plugins/power-pages/skills/plan-alm/SKILL.md @@ -242,7 +242,8 @@ Steps: --envUrl "{envUrl}" --token "{token}" \ --siteId "{websiteRecordId from powerpages.config.json}" \ --publisherPrefix "{solutionManifest.publisher.prefix}" \ - --solutionId "{solutionManifest.solution.solutionId}" + --solutionId "{solutionManifest.solution.solutionId}" \ + --projectRoot "." ``` Parse stdout and evaluate `missing.*`: From bb6f7f72e556e92cbb0af54218e293bda3ae4d9c Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Thu, 18 Jun 2026 17:03:01 +0530 Subject: [PATCH 22/38] Address #193 review: attr-cap overflow warning + componentCount proxy + 4 more MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Substantive (both reviews flagged): - **maxSchemaAttrs overflow now warned at the ceiling boundary** (compute-split-plan.js). Commit 2 fixed the under-ceiling case (seed n=min(clusters,ceiling)); at the maxSchemaSplitSolutions ceiling (>8 independent attr-heavy clusters) the FFD least-loaded fallback can still co-locate clusters and bust maxSchemaAttrs. The oversized guard only checked table COUNT — added a companion attr-cap warning (summed attributeCount > maxSchemaAttrs). Regression test with 9×14000-attr clusters. - **Table-domain componentCount is now a schema-component proxy** (sum(attributeCount) + 1/table), not the table count. Counting 1/table severely undercounts solution components and could let an over-cap Table solution slip past validateSplits (and distorted the Site solution's subtracted count). Test asserts 500+300+2 = 802. Copilot inline: - **odataGetAll FAILS CLOSED**: throws if it hits maxPages with @odata.nextLink still present, instead of silently returning a truncated set (wrong ALM counts). Test added. - **resolve-site-tables: sources.tablePermissions counts permission FILES**, not parsed records — a malformed file no longer makes a real site look manifest-only/unavailable (which drives tableCountScope). Test with a malformed file. - **Renamed local odataGet → odataGetPath** in estimate-solution-size.js to avoid the name collision with validation-helpers' shared odataGet (different arg order). Minor: - **discoverTableRelationships now uses bounded concurrency (5)** instead of fully serial (~2 calls/table; a 34-table site was 68 sequential round-trips). Edge assembly stays sequential for deterministic dedup. - AGENTS.md: corrected the packing claim (both cap-exceeding cases now warn, not "always under the caps"). Deferred (tracked as follow-up): renderer doesn't surface tableCountScope:'unavailable' distinctly — edge-case (plan-alm always passes --projectRoot) + touches the merged #191 renderer; bundling with the renderer/terminology follow-up. 1228 tests pass, alm-lint 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) --- plugins/power-pages/AGENTS.md | 2 +- .../scripts/lib/compute-split-plan.js | 44 ++++++++++++++++--- .../scripts/lib/estimate-solution-size.js | 44 +++++++++++++------ .../scripts/lib/resolve-site-tables.js | 9 +++- .../scripts/lib/validation-helpers.js | 15 ++++++- .../scripts/tests/compute-split-plan.test.js | 40 +++++++++++++++++ .../scripts/tests/resolve-site-tables.test.js | 15 +++++++ .../scripts/tests/validation-helpers.test.js | 14 ++++++ 8 files changed, 161 insertions(+), 22 deletions(-) diff --git a/plugins/power-pages/AGENTS.md b/plugins/power-pages/AGENTS.md index dc28b330d..afdd2d50e 100644 --- a/plugins/power-pages/AGENTS.md +++ b/plugins/power-pages/AGENTS.md @@ -205,7 +205,7 @@ Shared lib modules live at `scripts/lib/` and are imported by other scripts via - `scripts/lib/alm-thresholds.js`: Central default threshold constants for the split decision tree. Loads optional `.alm-config.json` from project root and merges over defaults. Exports `DEFAULTS`, `DEFAULT_CONFIG`, `loadConfig(projectRoot)`, `classifyTier(value, greenUpperExclusive, yellowUpperExclusive)`, `deepMerge(target, source)`. Used by `estimate-solution-size.js` and `compute-split-plan.js`. - `scripts/lib/estimate-solution-size.js`: Estimates solution size + component counts by querying Dataverse. Args: `--envUrl`, `--websiteRecordId`, `--token` (opt), `--publisherPrefix` (opt), `--siteName` (opt), `--solutionId` (opt — scopes env var count to the target solution; without it falls back to a publisher-prefix tenant-wide query that overcounts when prefix is shared), `--datamodelManifest` (opt), `--projectRoot` (opt — enables disk cross-check: walks the local build-output directory (`dist/`, `public-output/`, `build/`, `.output/`) and surfaces the byte total). Output: `{ totalSizeMB, componentCountSiteTotal, componentCountSiteActionable, componentCountInSolution, orphansOnSite, tableCount, tableCountScope, schemaAttrCount, webFilesAggregateMB, webFilesIndividual[], webFileCount, webFileSampleSize, webFilesDiskMeasuredMB, webFilesDiskMeasuredPath, webFilesDiskFileCount, cloudFlowCount, botCount, envVarCount, envVarCountScope, envVarCountTenantWide, mediaRatio, siteType, tables[], tableRelationships[], breakdown, estimationMethod, estimationAccuracyPct, truncationSuspected, truncationWarnings, ppcGroundTruthCount }`. **Table discovery is site-referenced, NOT publisher-prefix:** `tableCount`/`tables[]` are scoped to the custom tables the site actually references — its `.powerpages-site/table-permissions/` (+ datamodel manifest) intersected with the env's custom-unmanaged tables (via `resolve-site-tables.js` + `query-metadata.js`). `tableCountScope` ∈ `"site-referenced" | "manifest-only" | "unavailable"` (the last → 0 tables, never an env-wide prefix dump). `--publisherPrefix` now scopes ONLY the env var count, not tables. `tableRelationships[]` are `[a,b]` dependency edges (lookups + N:N, via `query-table-relationships.js`) among the scoped tables, consumed by `compute-split-plan.js` to cluster related tables into the same solution. Metadata-based estimation with ±15% caveat. Web-file size is measured via stratified sample (first 50 + middle 50 + last 50, cap 150) scaled to full count. Disk fields are null unless `--projectRoot` was passed AND a build-output directory was found. Truncation canaries fire when Dataverse pagination disagrees with `@odata.count`, when ppcs land on a page-size boundary, when sampled average bytes/file < 1 KB at scale, or when the disk total exceeds the Dataverse total by >2× — any signal flips `truncationSuspected: true` with a per-cause `truncationWarnings[]` entry. Used by `plan-alm` Phase 1 Step 10. -- `scripts/lib/compute-split-plan.js`: Runs the split decision tree against a size-estimate blob. Args: `--estimate `, `--projectRoot` (opt — for `.alm-config.json` overrides), `--siteName` (opt), `--publisherPrefix` (opt). Output: `{ sizeAnalysis, assetAdvisory, splitStrategy, appliedStrategies, compositeSubPartitioned, proposedSolutions[], recommendations[], truncationSuspected, truncationWarnings }`. Evaluates strategies in priority order: Strategy 3 (Schema Segmentation) → Strategy 1 (Layer Split) → Strategy 2 (Change-Frequency) → Strategy 4 (Config Isolation). **Schema Segmentation is dependency-aware + capacity-bounded:** it builds connected-component clusters from `estimate.tableRelationships` (union-find), then bin-packs whole clusters (never splitting a relationship) into the fewest solutions that keep each under `maxTableCount`/`maxSchemaAttrs`, capped at `maxSchemaSplitSolutions` (default 8). This replaced the old one-solution-per-table-name-stem heuristic that produced ~one solution per table. An indivisible cluster over the cap stays whole and raises an oversized-cluster `recommendations[]` warning. The split trigger + thresholds are unchanged — only the packing. Strategy 4 stacks additively. Strategy 1 also runs a composite sub-partition pass: when Core still exceeds the size OR component-count cap after Web Assets are peeled off, Core is replaced with change-frequency-shaped sub-children (`_Foundation`/`_Config`/`_Content`, plus `_Integration` whenever the parent had any flows or bots — coverage takes priority over the `changeFreqMinFlows` heuristic, which governs only the TOP-LEVEL strategy choice). When additive Strategy 4 is firing concurrently (a top-level `_EnvVars` solution), `_Config` drops `Environment Variable` from its componentTypes to avoid double-claim; when it isn't, `_Config` absorbs env vars so they have an owner. Sub-partitioning sets `compositeSubPartitioned: true` and appends `composite-sub-partition` to `appliedStrategies`. `validateSplits` checks BOTH the size AND component-count cap per split (skipping `isFutureBuffer` solutions). Supports `.alm-config.json` overrides including `strategyOverride` to bypass the tree. See `solution-splitting-logic.md` spec in design docs for full logic. +- `scripts/lib/compute-split-plan.js`: Runs the split decision tree against a size-estimate blob. Args: `--estimate `, `--projectRoot` (opt — for `.alm-config.json` overrides), `--siteName` (opt), `--publisherPrefix` (opt). Output: `{ sizeAnalysis, assetAdvisory, splitStrategy, appliedStrategies, compositeSubPartitioned, proposedSolutions[], recommendations[], truncationSuspected, truncationWarnings }`. Evaluates strategies in priority order: Strategy 3 (Schema Segmentation) → Strategy 1 (Layer Split) → Strategy 2 (Change-Frequency) → Strategy 4 (Config Isolation). **Schema Segmentation is dependency-aware + capacity-bounded:** it builds connected-component clusters from `estimate.tableRelationships` (union-find), then bin-packs whole clusters (never splitting a relationship) into the fewest solutions that keep each under `maxTableCount`/`maxSchemaAttrs` **where possible** — capped at `maxSchemaSplitSolutions` (default 8). This replaced the old one-solution-per-table-name-stem heuristic that produced ~one solution per table. Two cases CAN exceed a per-solution cap, and BOTH raise an `recommendations[]` warning rather than failing silently: (a) an indivisible dependency cluster larger than `maxTableCount` stays whole (oversized-cluster table-count warning); (b) when MORE than `maxSchemaSplitSolutions` independent attr-heavy clusters must share the capped solution count, the FFD least-loaded fallback co-locates clusters and a solution's summed columns exceed `maxSchemaAttrs` (oversized-schema attr-cap warning). The split trigger + thresholds are unchanged — only the packing. Strategy 4 stacks additively. Strategy 1 also runs a composite sub-partition pass: when Core still exceeds the size OR component-count cap after Web Assets are peeled off, Core is replaced with change-frequency-shaped sub-children (`_Foundation`/`_Config`/`_Content`, plus `_Integration` whenever the parent had any flows or bots — coverage takes priority over the `changeFreqMinFlows` heuristic, which governs only the TOP-LEVEL strategy choice). When additive Strategy 4 is firing concurrently (a top-level `_EnvVars` solution), `_Config` drops `Environment Variable` from its componentTypes to avoid double-claim; when it isn't, `_Config` absorbs env vars so they have an owner. Sub-partitioning sets `compositeSubPartitioned: true` and appends `composite-sub-partition` to `appliedStrategies`. `validateSplits` checks BOTH the size AND component-count cap per split (skipping `isFutureBuffer` solutions). Supports `.alm-config.json` overrides including `strategyOverride` to bypass the tree. See `solution-splitting-logic.md` spec in design docs for full logic. - `scripts/lib/resolve-site-tables.js`: Single source of truth for "which custom tables does this site actually use." `collectReferencedEntityNames({ projectRoot, datamodelManifestPath })` reads `.powerpages-site/table-permissions/*.tablepermission.yml` (`entitylogicalname`, via `powerpages-config.js → loadTablePermissions`) + the datamodel manifest → `{ names:Set, available, sources }`. `scopeCustomTables(referencedNames, customUnmanagedTables)` intersects that set with the env's custom-unmanaged tables. SME-confirmed: table permissions are the complete signal ("if a table is used in the site there will be permissions for it"), so forms/lists are NOT scanned. Used by `estimate-solution-size.js` and `discover-site-components.js` to replace the publisher-prefix table dump. - `scripts/lib/query-metadata.js`: `queryCustomUnmanagedTables(envUrl, token, makeRequest?)` → `[{ logicalName, metadataId, schemaName, displayName }]` (the single `EntityDefinitions?$filter=IsCustomEntity` query, `IsManaged===false` filtered). Consolidates the formerly-triplicated custom-table query (estimator, discover-site-components, setup-solution). Reuses `odataGetAll` from `validation-helpers.js`. - `scripts/lib/query-table-relationships.js`: `fetchTableRelationships(envUrl, table, token, makeRequest?)` → `{ oneToMany[], manyToMany[] }`. Extracted from `skills/audit-permissions/scripts/query-table-relationships.js` (now a thin CLI wrapper over this lib) and extended with ManyToMany. OneToMany errors propagate; ManyToMany is best-effort. Used by the estimator to build `tableRelationships[]` and by audit-permissions for relationship-scope validation. diff --git a/plugins/power-pages/scripts/lib/compute-split-plan.js b/plugins/power-pages/scripts/lib/compute-split-plan.js index 82305136e..8a41d8ad5 100644 --- a/plugins/power-pages/scripts/lib/compute-split-plan.js +++ b/plugins/power-pages/scripts/lib/compute-split-plan.js @@ -312,6 +312,19 @@ function partitionBySchema(estimate, meta, config) { const breakdownAvailable = estimate.breakdown && Number.isFinite(Number(estimate.breakdown.tables)); const domainDescSuffix = breakdownAvailable ? '' : ' (rough estimate — breakdown unavailable)'; + // Per-table attribute counts → a schema-component PROXY. A table contributes far + // more than one solution component (the entity + every column/relationship), so + // counting 1-per-table severely undercounts and lets an over-cap Table solution + // slip past validateSplits' maxComponentCount check (and distorts the Site + // solution's count, which subtracts the domain counts). Proxy = sum(attributeCount) + // + 1 per table (the entity component). attributeCount comes from estimate.tables[]. + const attrByTable = new Map( + (Array.isArray(estimate.tables) ? estimate.tables : []) + .map((t) => [t && t.logicalName, (t && t.attributeCount) || 0]), + ); + const schemaComponentProxy = (names) => + (names || []).reduce((sum, n) => sum + (attrByTable.get(n) || 0), 0) + (names ? names.length : 0); + const domainSolutions = explicitDomains.map((dom, i) => ({ uniqueName: `${meta.baseName}_${sanitizeDomainName(dom.name)}`, displayName: `${meta.siteName} — ${dom.name}`, @@ -319,11 +332,11 @@ function partitionBySchema(estimate, meta, config) { componentTypes: ['Table'], description: `Schema domain: ${dom.name}. Tables: ${(dom.tableLogicalNames || []).join(', ') || '(derived)'}${domainDescSuffix}`, sizeMB: round(sizePerDomain), - // A Table domain's component count IS its table count when known (each table - // is one Entity solution component). Falls back to an even attr-share split - // only for explicit domains that didn't list their tables. + // Schema-component proxy when the domain's tables are known (sum of columns + + // 1/table); falls back to an even attr-share split only for explicit domains + // that didn't list their tables. componentCount: (dom.tableLogicalNames && dom.tableLogicalNames.length > 0) - ? dom.tableLogicalNames.length + ? schemaComponentProxy(dom.tableLogicalNames) : Math.ceil((estimate.schemaAttrCount || 0) / domainCount), components: [], tableLogicalNames: dom.tableLogicalNames || [], @@ -796,6 +809,26 @@ function computeSplitPlan({ estimate, config, meta }) { type: 'warning', message: `Solution ${s.uniqueName} holds ${s.tableLogicalNames.length} related tables — above the ${config.thresholds.maxTableCount}-per-solution cap — because they form one dependency cluster that cannot be split without breaking a relationship. Consider denormalizing the schema or raising maxTableCount in .alm-config.json.`, })); + // Oversized-SCHEMA guard (companion to the table-count guard above): a Table + // solution whose summed column count exceeds maxSchemaAttrs. This fires at the + // `maxSchemaSplitSolutions` ceiling — when MORE than that many independent + // attr-heavy table clusters must share the capped number of split solutions, the + // FFD packer's least-loaded fallback co-locates clusters and a bucket busts the + // column cap. Without this, the overflow is silent (the table-count guard alone + // misses it). attributeCount comes from estimate.tables[]. + const attrByTable = new Map( + (Array.isArray(estimate.tables) ? estimate.tables : []) + .map((t) => [t && t.logicalName, (t && t.attributeCount) || 0]), + ); + const solutionSchemaAttrs = (s) => + (s.tableLogicalNames || []).reduce((sum, n) => sum + (attrByTable.get(n) || 0), 0); + const oversizedAttrWarnings = proposedSolutions + .filter((s) => Array.isArray(s.tableLogicalNames) && s.tableLogicalNames.length > 0 && + solutionSchemaAttrs(s) > config.thresholds.maxSchemaAttrs) + .map((s) => ({ + type: 'warning', + message: `Solution ${s.uniqueName} holds tables totaling ${solutionSchemaAttrs(s)} columns — above the ${config.thresholds.maxSchemaAttrs}-column per-solution cap. This happens when more than ${config.thresholds.maxSchemaSplitSolutions} independent attr-heavy table clusters must share the capped number of schema-split solutions. Consider raising maxSchemaSplitSolutions (or maxSchemaAttrs) in .alm-config.json, or denormalizing the widest tables.`, + })); // Surface estimator-side truncation warnings as `recommendations[]` entries // so the rendered plan shows them inline. These get the `error` type because // a truncated input is more dangerous than a normal split-decision warning @@ -808,7 +841,8 @@ function computeSplitPlan({ estimate, config, meta }) { const recommendations = truncationRecs .concat(buildRecommendations(estimate, strategy, config)) .concat(splitWarnings) - .concat(oversizedClusterWarnings); + .concat(oversizedClusterWarnings) + .concat(oversizedAttrWarnings); const appliedStrategies = [strategy.primary]; if (strategy.additive) appliedStrategies.push('strategy-4-config-isolation'); diff --git a/plugins/power-pages/scripts/lib/estimate-solution-size.js b/plugins/power-pages/scripts/lib/estimate-solution-size.js index 911146e07..4a29b7cfe 100644 --- a/plugins/power-pages/scripts/lib/estimate-solution-size.js +++ b/plugins/power-pages/scripts/lib/estimate-solution-size.js @@ -91,7 +91,11 @@ const ODATA_MAX_PAGE_SIZE = 5000; // not a normal-case truncation. const PAGINATION_SAFETY_CAP = 100; -async function odataGet(envUrl, path, token) { +// NB: this is the local path-based helper `odataGetPath(envUrl, path, token)` — it +// builds the v9.2 URL from a relative path. Distinct from validation-helpers.js's +// shared `odataGet(url, token, request)` (absolute URL, different arg order); the +// rename avoids a silent breakage if a future edit destructures the shared one here. +async function odataGetPath(envUrl, path, token) { const url = path.startsWith('http') ? path : `${envUrl}/api/data/v9.2/${path.replace(/^\//, '')}`; const res = await helpers.makeRequest({ url, @@ -129,7 +133,7 @@ async function collectPaginated(envUrl, path, token, maxPages = PAGINATION_SAFET const items = []; let pagesFetched = 0; for (let p = 0; p < maxPages && next; p++) { - const page = await odataGet(envUrl, next, token); + const page = await odataGetPath(envUrl, next, token); if (Array.isArray(page.value)) items.push(...page.value); next = page['@odata.nextLink'] || null; pagesFetched += 1; @@ -259,7 +263,7 @@ async function countOData(envUrl, entity, filter, token) { try { const filterPart = filter ? `&$filter=${filter}` : ''; const countPath = `${entity}?$count=true&$top=1${filterPart}`; - const page = await odataGet(envUrl, countPath, token); + const page = await odataGetPath(envUrl, countPath, token); const n = page['@odata.count']; return typeof n === 'number' ? n : null; } catch { @@ -349,13 +353,27 @@ async function discoverTableRelationships(envUrl, tables, token) { seen.add(key); edges.push(a < b ? [a, b] : [b, a]); }; - for (const t of tables) { - let rel; - try { - rel = await fetchTableRelationships(envUrl, t.logicalName, token); - } catch { - continue; // inaccessible table — skip its edges + // Fetch each table's relationships with BOUNDED CONCURRENCY (~2 OData calls per + // table; a 34-table site is 68 round-trips — serial is slow at plan time). Edge + // assembly stays sequential, in table order, so dedup is deterministic. + const CONCURRENCY = 5; + const results = new Array(tables.length).fill(null); + let nextIdx = 0; + async function worker() { + while (nextIdx < tables.length) { + const i = nextIdx++; + try { + results[i] = await fetchTableRelationships(envUrl, tables[i].logicalName, token); + } catch { + results[i] = null; // inaccessible table — skip its edges + } } + } + await Promise.all( + Array.from({ length: Math.min(CONCURRENCY, tables.length) }, () => worker()), + ); + for (const rel of results) { + if (!rel) continue; for (const e of rel.oneToMany) addEdge(e.referencedEntity, e.referencingEntity); for (const e of rel.manyToMany) addEdge(e.entity1, e.entity2); } @@ -366,7 +384,7 @@ async function countAttributesForTables(envUrl, tables, token) { let total = 0; for (const t of tables) { try { - const page = await odataGet( + const page = await odataGetPath( envUrl, `EntityDefinitions(LogicalName='${t.logicalName}')/Attributes?$select=LogicalName&$top=1000`, token, @@ -473,7 +491,7 @@ function classifyPPCs(ppcs) { } async function measureWebFiles(envUrl, webFiles, token) { - // Uses odataGet directly (single-row fetch each, no pagination needed). + // Uses odataGetPath directly (single-row fetch each, no pagination needed). const individual = []; let aggregateBytes = 0; let imgOrFontBytes = 0; @@ -481,7 +499,7 @@ async function measureWebFiles(envUrl, webFiles, token) { for (const wf of webFiles) { const id = wf.powerpagecomponentid; try { - const rec = await odataGet( + const rec = await odataGetPath( envUrl, `powerpagecomponents(${id})?$select=name,powerpagecomponentid,content`, token, @@ -978,7 +996,7 @@ async function estimateSolutionSize({ envUrl, websiteRecordId, token, publisherP const LEGACY_BOUNDARIES = [500, 1000, 2000]; if (LEGACY_BOUNDARIES.includes(ppcs.length)) { truncationWarnings.push( - `ppcs.length is exactly ${ppcs.length} — a historical paging boundary from an older $top value. Suggests the \`Prefer: odata.maxpagesize\` header has regressed; verify odataGet still sends it.`, + `ppcs.length is exactly ${ppcs.length} — a historical paging boundary from an older $top value. Suggests the \`Prefer: odata.maxpagesize\` header has regressed; verify odataGetPath still sends it.`, ); } diff --git a/plugins/power-pages/scripts/lib/resolve-site-tables.js b/plugins/power-pages/scripts/lib/resolve-site-tables.js index 043f35322..e58ce46ec 100644 --- a/plugins/power-pages/scripts/lib/resolve-site-tables.js +++ b/plugins/power-pages/scripts/lib/resolve-site-tables.js @@ -49,13 +49,20 @@ function collectReferencedEntityNames({ projectRoot, datamodelManifestPath } = { const dir = path.join(projectRoot, '.powerpages-site', 'table-permissions'); if (fs.existsSync(dir)) { sawTablePermissionsDir = true; + // Count the permission FILES present — `sources.tablePermissions` must mean + // "the site has table-permission files" (a reliable site-referenced signal, + // consumed by `tableCountScope`), NOT "we parsed at least one record". A + // temporarily-malformed file must not make a real site look manifest-only. + try { + sources.tablePermissions = fs.readdirSync(dir) + .filter((f) => /\.tablepermission\.yml$/i.test(f)).length; + } catch { /* keep 0 */ } let records = []; try { records = loadTablePermissions(dir); } catch { records = []; } for (const r of records) { const name = r && r.entitylogicalname; // NOT entityname (that's the display label) if (typeof name === 'string' && name.trim()) { names.add(name.trim().toLowerCase()); - sources.tablePermissions += 1; } } } diff --git a/plugins/power-pages/scripts/lib/validation-helpers.js b/plugins/power-pages/scripts/lib/validation-helpers.js index 8a3b594cc..b9a097ee5 100644 --- a/plugins/power-pages/scripts/lib/validation-helpers.js +++ b/plugins/power-pages/scripts/lib/validation-helpers.js @@ -265,7 +265,11 @@ async function odataGet(url, token, request = makeRequest) { /** * Follows `@odata.nextLink`, aggregating every page's `value[]` into one array. - * `maxPages` is a runaway-loop safety cap (100 × 5000 ≈ 500K rows). + * `maxPages` is a runaway-loop safety cap (100 × 5000 ≈ 500K rows). FAILS CLOSED: + * if the cap is reached while `@odata.nextLink` is still present, throws rather than + * silently returning a truncated set — a partial result would produce wrong + * table/env-var counts for ALM sizing/splitting with no signal. Callers that want + * partial results must catch and downgrade accuracy intentionally. * @param {string} url - absolute starting URL * @param {string} token - bearer token * @param {Function} [request=makeRequest] - injectable for tests @@ -275,11 +279,18 @@ async function odataGet(url, token, request = makeRequest) { async function odataGetAll(url, token, request = makeRequest, maxPages = 100) { const out = []; let next = url; - for (let p = 0; p < maxPages && next; p++) { + let p = 0; + for (; p < maxPages && next; p++) { const page = await odataGet(next, token, request); if (Array.isArray(page.value)) out.push(...page.value); next = page['@odata.nextLink'] || null; } + if (next) { + throw new Error( + `odataGetAll hit the ${maxPages}-page cap with @odata.nextLink still present ` + + `(${out.length} rows so far) — result would be truncated. Raise maxPages or narrow the query.`, + ); + } return out; } diff --git a/plugins/power-pages/scripts/tests/compute-split-plan.test.js b/plugins/power-pages/scripts/tests/compute-split-plan.test.js index abf985a37..34ea3741e 100644 --- a/plugins/power-pages/scripts/tests/compute-split-plan.test.js +++ b/plugins/power-pages/scripts/tests/compute-split-plan.test.js @@ -238,6 +238,46 @@ test('computeSplitPlan Strategy 3 never overflows the attr cap when independent } }); +test('computeSplitPlan WARNS when >maxSchemaSplitSolutions independent attr-heavy clusters bust the attr cap (ceiling boundary)', () => { + // 9 INDEPENDENT (no-edge) 14000-attr tables, ceiling=8: the packer can open at + // most 8 buckets, so FFD's least-loaded fallback co-locates 2 tables in one + // solution -> 28000 attrs > maxSchemaAttrs(15000). The table-count guard misses + // it (each solution has <=2 tables, well under maxTableCount). The attr guard + // must surface it. (Regression for the ceiling-boundary silent overflow.) + const tables = Array.from({ length: 9 }, (_, i) => ({ logicalName: `tst_t${i}`, attributeCount: 14000 })); + const result = computeSplitPlan({ + estimate: baseEstimate({ tableCount: 9, schemaAttrCount: 9 * 14000, tables, tableRelationships: [] }), + config: baseConfig(), + meta: { baseName: 'Test', siteName: 'Test Site' }, + }); + assert.equal(result.splitStrategy, 'strategy-3-schema-segmentation'); + const attrWarn = (result.recommendations || []).find((r) => /columns — above the .* per-solution cap/i.test(r.message || '')); + assert.ok(attrWarn, 'expected an oversized-schema (attr-cap) warning when a Table solution exceeds maxSchemaAttrs at the ceiling'); +}); + +test('computeSplitPlan: a Table domain componentCount is a schema-component proxy (sum attrs + 1/table), not the table count', () => { + // Counting 1-per-table severely undercounts solution components and can let an + // over-cap solution slip past validateSplits. Proxy = sum(attributeCount) + 1/table. + const result = computeSplitPlan({ + // tableCount:34 makes the schema "red" so Strategy 3 fires; the proxy reads the + // per-table attributeCount from tables[] (500 + 300), independent of the global count. + estimate: baseEstimate({ + tableCount: 34, + schemaAttrCount: 32000, + tables: [ + { logicalName: 'tst_product', attributeCount: 500 }, + { logicalName: 'tst_category', attributeCount: 300 }, + ], + }), + config: baseConfig({ domains: [{ name: 'Catalog', tableLogicalNames: ['tst_product', 'tst_category'] }] }), + meta: { baseName: 'Test', siteName: 'Test Site' }, + }); + assert.equal(result.splitStrategy, 'strategy-3-schema-segmentation'); + const catalog = result.proposedSolutions.find((s) => s.uniqueName === 'Test_Catalog'); + assert.ok(catalog, 'Catalog Table domain solution exists'); + assert.equal(catalog.componentCount, 802, 'sum(500+300) + 2 tables = 802 (proxy), not 2 (table count)'); +}); + test('computeSplitPlan additive Strategy 4 prepends EnvVars solution', () => { const result = computeSplitPlan({ estimate: baseEstimate({ totalSizeMB: 142, webFilesAggregateMB: 110, envVarCount: 800 }), diff --git a/plugins/power-pages/scripts/tests/resolve-site-tables.test.js b/plugins/power-pages/scripts/tests/resolve-site-tables.test.js index 7dfebc315..f806f5b29 100644 --- a/plugins/power-pages/scripts/tests/resolve-site-tables.test.js +++ b/plugins/power-pages/scripts/tests/resolve-site-tables.test.js @@ -96,3 +96,18 @@ test('scopeCustomTables: keeps only referenced custom tables; drops unreferenced test('scopeCustomTables: empty referenced set -> empty (never a prefix dump)', () => { assert.deepEqual(scopeCustomTables(new Set(), [{ logicalName: 'new_x' }]), []); }); + +test('collectReferencedEntityNames: sources.tablePermissions reflects FILE existence even when files are unparseable', (t) => { + const root = makeProject(t); + const dir = path.join(root, '.powerpages-site', 'table-permissions'); + fs.mkdirSync(dir, { recursive: true }); + // A malformed permission file — present, but yields no entitylogicalname. Without + // counting files up-front, sources.tablePermissions would be 0 and a real site + // would be misclassified manifest-only/unavailable by tableCountScope. + fs.writeFileSync(path.join(dir, 'broken.tablepermission.yml'), ':\n not: [valid yaml }}}\n'); + + const { names, available, sources } = collectReferencedEntityNames({ projectRoot: root }); + assert.equal(available, true, 'the table-permissions dir exists → available'); + assert.equal(sources.tablePermissions, 1, 'counts the permission FILE, not parsed records (0 parsed here)'); + assert.equal(names.size, 0, 'no entity names parsed from the malformed file'); +}); diff --git a/plugins/power-pages/scripts/tests/validation-helpers.test.js b/plugins/power-pages/scripts/tests/validation-helpers.test.js index d48411ca8..0e739fe9a 100644 --- a/plugins/power-pages/scripts/tests/validation-helpers.test.js +++ b/plugins/power-pages/scripts/tests/validation-helpers.test.js @@ -86,6 +86,20 @@ test('odataGetAll follows @odata.nextLink and aggregates all pages', async () => assert.deepEqual(rows.map((r) => r.id), [1, 2, 3]); }); +test('odataGetAll FAILS CLOSED: throws when it hits maxPages with @odata.nextLink still present', async () => { + const { odataGetAll } = require(helpersPath); + // Every page advertises a nextLink → never terminates → hits the page cap. + // Must throw rather than silently return a truncated set (wrong ALM counts). + const fakeRequest = async () => ({ + statusCode: 200, + body: JSON.stringify({ value: [{ id: 1 }], '@odata.nextLink': 'https://x/next' }), + }); + await assert.rejects( + () => odataGetAll('https://x/start', 'tok', fakeRequest, 3), + /page cap.*nextLink|truncated/i, + ); +}); + test('odataGet throws on non-2xx', async () => { const { odataGet } = require(helpersPath); const fakeRequest = async () => ({ statusCode: 404, body: 'not found' }); From 91be9fd84725be14f648a9c8c78d4797aebe30a9 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 18 Jun 2026 11:46:56 +0000 Subject: [PATCH 23/38] Resolve merge conflicts with origin/users/nityagi/table-discovery-fix (clean) --- .../power-pages/.claude-plugin/plugin.json | 4 - plugins/power-pages/AGENTS.md | 8 -- .../scripts/check-activation-status.js | 48 --------- .../scripts/lib/compute-split-plan.js | 15 --- .../scripts/lib/detect-project-context.js | 36 ------- .../scripts/lib/estimate-solution-size.js | 10 -- .../scripts/lib/refresh-alm-plan-data.js | 6 -- .../scripts/lib/resolve-site-tables.js | 7 -- .../scripts/lib/validation-helpers.js | 18 ---- .../scripts/tests/compute-split-plan.test.js | 3 - .../tests/detect-project-context.test.js | 9 -- .../tests/refresh-alm-plan-data.test.js | 97 ------------------- .../scripts/tests/resolve-site-tables.test.js | 3 - .../scripts/tests/validation-helpers.test.js | 3 - 14 files changed, 267 deletions(-) diff --git a/plugins/power-pages/.claude-plugin/plugin.json b/plugins/power-pages/.claude-plugin/plugin.json index f6342b3f2..f3fa3f687 100644 --- a/plugins/power-pages/.claude-plugin/plugin.json +++ b/plugins/power-pages/.claude-plugin/plugin.json @@ -1,10 +1,6 @@ { "name": "power-pages", -<<<<<<< HEAD "version": "2.5.0", -======= - "version": "2.4.0", ->>>>>>> origin/users/nityagi/table-discovery-fix "description": "Create and deploy Power Pages sites using modern development approaches. Supports code sites (SPAs) with React, Angular, Vue, or Astro. Includes ALM orchestration (plan-alm) with a solution-splitting decision tree, per-solution pipelines, Azure Blob asset advisory, manifest schema v2 for multi-solution deployments, and force-link remediation for cross-host pipeline migrations.", "author": { "name": "Microsoft", diff --git a/plugins/power-pages/AGENTS.md b/plugins/power-pages/AGENTS.md index 6667518d5..c10f0036c 100644 --- a/plugins/power-pages/AGENTS.md +++ b/plugins/power-pages/AGENTS.md @@ -198,11 +198,7 @@ Shared lib modules live at `scripts/lib/` and are imported by other scripts via #### ALM Prerequisites & Context - `scripts/lib/verify-alm-prerequisites.js`: Verifies all prerequisites for ALM skills — PAC CLI installed + authenticated (`pac env who`), Azure CLI installed + logged in, Dataverse API reachable (`WhoAmI`). Args: `--envUrl` (opt, overrides env from PAC CLI), `--require-manifest` (fails if `.solution-manifest.json` not found). Output: `{ envUrl, token, userId, organizationId, tenantId }`. Exit 0 on success, exit 1 on any failure. Used by `setup-solution`, `export-solution`, `import-solution`, `setup-pipeline`, `deploy-pipeline`, `plan-alm`. -<<<<<<< HEAD -- `scripts/lib/detect-project-context.js`: Reads Power Pages project context from the project root. Resolves site identity in order: (1) `powerpages.config.json` → `siteType: "code"` (SPA sites); (2) `.powerpages-site/website.yml` → `siteType: "data-model"` (standard/enhanced data-model "EDM" sites, which have **no** `powerpages.config.json` — `id`→`websiteRecordId`, `name`→`siteName`, `environmentUrl: null` since the local files carry no env URL). Also reads `.solution-manifest.json` and `.datamodel-manifest.json`. Args: `--projectRoot` (opt, auto-discovered from cwd if omitted). Output: `{ projectRoot, siteType, siteName, websiteRecordId, environmentUrl, solutionManifest, datamodelManifest }`. Exit 0 on success, exit 1 only if **neither** `powerpages.config.json` nor `.powerpages-site/website.yml` is found. Note: `findProjectRoot` (in `validation-helpers.js`) likewise treats a `.powerpages-site/` directory as a project-root marker, not just `powerpages.config.json`, so data-model sites are discoverable. -======= - `scripts/lib/detect-project-context.js`: Reads Power Pages project context from the project root. The `siteType` discriminator is the **build axis** — code/SPA vs declarative (design-studio) site — NOT the Dataverse data-model axis (a declarative site can be on the standard OR enhanced data model; both download to a `.powerpages-site/` tree). `siteType: "data-model"` is the (compat-named) declarative bucket; a future pass may rename it `"declarative"`. Resolves identity in order: (1) `powerpages.config.json` → `siteType: "code"` (code/SPA sites); (2) `.powerpages-site/` → `siteType: "data-model"` (declarative design-studio sites — standard or enhanced data model — which have **no** `powerpages.config.json`). The **authoritative declarative marker is the `.powerpages-site/.portalconfig/` directory** (only declarative sites have it); `website.yml` is the identity source (`id`→`websiteRecordId`, `name`→`siteName`) but is NOT a reliable declarative signal alone because **both** site types carry it. `environmentUrl: null` for declarative sites (no env URL in the local files — callers re-confirm via `pac env who`). Also reads `.solution-manifest.json` and `.datamodel-manifest.json`. Args: `--projectRoot` (opt). Output: `{ projectRoot, siteType, siteName, websiteRecordId, environmentUrl, solutionManifest, datamodelManifest }`. Exit 0 on success, exit 1 only if neither `powerpages.config.json` nor a `.powerpages-site/` (`.portalconfig/`/`website.yml`) marker is found. Note: `findProjectRoot` (in `validation-helpers.js`) likewise treats a `.powerpages-site/` directory as a project-root marker. ->>>>>>> origin/users/nityagi/table-discovery-fix - `scripts/lib/alm-paths.js`: Single source of truth for ALM artifact paths. Exports `ALM_DIR` (always `docs/alm`), `FILE_NAMES` (frozen object mapping logical key → filename for all 14 ALM artifacts), `almDir(projectRoot) → path`, `almPath(projectRoot, key) → path`, `ensureAlmDir(projectRoot) → path` (mkdir -p idempotent). Every ALM-only state file (5 plan/decision JSONs + 9 `last-*.json` skill-run markers including `last-export.json`) writes under `/docs/alm/`. **Always resolve through this helper** — never inline a raw `docs/alm/...` path in a script. Files intentionally NOT moved here (and not in `FILE_NAMES`): `.solution-manifest.json`, `.datamodel-manifest.json`, `.alm-config.json`, `.alm-deferred`, `deployment-settings.json`. Adding a new ALM marker means adding its key + filename to `FILE_NAMES` first; `almPath` throws on unknown keys to catch typos at call-site. - `scripts/lib/check-alm-plan.js`: Phase 0 gate helper used by every ALM skill to detect (a) whether an ALM plan exists for this project, (b) whether the user has explicitly deferred ALM via the `.alm-deferred` marker, and (c) whether an existing plan is stale (the source solution was modified after the plan was generated). Args: `--projectRoot`, `--envUrl` (opt — required for staleness check), `--token` (opt), `--solutionId` (opt — required for staleness check). Output: `{ exists, deferred, deferral, planPath, htmlPath, stale, staleness: { reason, detail }, generatedAt, planStatus, solution: {...} }`. Without env/solution context the helper does an existence-only check; with them it queries Dataverse for `solutions(solutionId)?$select=modifiedon` and compares against `planData.generatedAt`. Used by `setup-solution`, `setup-pipeline`, `deploy-pipeline`, `export-solution`, `import-solution`, `configure-env-variables`, `ensure-pipelines-host`, `force-link-environment` Phase 0 gates — the "fail closed when no plan" pattern. **PLAN_STATUS lifecycle — promotes `Approved` → `In Execution`:** plan-alm is plan-only and leaves the plan `Approved`; this helper performs the `Approved` → `In Execution` transition (and writes the first heartbeat) the **first time an execution skill's Phase 0 runs** — it is the only thing that sets `In Execution`, so without it the heartbeat/active-chain machinery (multi-hour-deploy `stale-heartbeat` reclassification) never engages. Gated on heartbeat-write: read-only callers pass `--no-heartbeat` (plan-alm's own deferral check, audits, tests) and are never promoted. The terminal `In Execution` → `Completed` transition is owned by `refresh-alm-plan-data.js` (completion evaluator). - `scripts/lib/resolve-target-solution.js`: Resolves "which solution should this new Dataverse record land in?" Implements the strict 3-step order from the ALM-aware-by-default principle: (1) explicit `--solutionUniqueName` (or equivalent caller arg) wins; (2) `.solution-manifest.json` in the project root; (3) neither → throw `NoSolutionConfiguredError`. **The module NEVER auto-picks from Dataverse** — interactive prompt UX is the caller's responsibility (catch the error, present an `AskUserQuestion` list, re-invoke with `explicit` populated). Callers that need to confirm the solution still exists in Dataverse can pass `verifyExists: true`; the module then enriches the result with `{ solutionId, version, ismanaged }`. Component-creation scripts must require this helper and pass through `--solutionUniqueName` so records land in the user's solution instead of `Default`. @@ -211,11 +207,7 @@ Shared lib modules live at `scripts/lib/` and are imported by other scripts via - `scripts/lib/alm-thresholds.js`: Central default threshold constants for the split decision tree. Loads optional `.alm-config.json` from project root and merges over defaults. Exports `DEFAULTS`, `DEFAULT_CONFIG`, `loadConfig(projectRoot)`, `classifyTier(value, greenUpperExclusive, yellowUpperExclusive)`, `deepMerge(target, source)`. Used by `estimate-solution-size.js` and `compute-split-plan.js`. - `scripts/lib/estimate-solution-size.js`: Estimates solution size + component counts by querying Dataverse. Args: `--envUrl`, `--websiteRecordId`, `--token` (opt), `--publisherPrefix` (opt), `--siteName` (opt), `--solutionId` (opt — scopes env var count to the target solution; without it falls back to a publisher-prefix tenant-wide query that overcounts when prefix is shared), `--datamodelManifest` (opt), `--projectRoot` (opt — enables disk cross-check: walks the local build-output directory (`dist/`, `public-output/`, `build/`, `.output/`) and surfaces the byte total). Output: `{ totalSizeMB, componentCountSiteTotal, componentCountSiteActionable, componentCountInSolution, orphansOnSite, tableCount, tableCountScope, schemaAttrCount, webFilesAggregateMB, webFilesIndividual[], webFileCount, webFileSampleSize, webFilesDiskMeasuredMB, webFilesDiskMeasuredPath, webFilesDiskFileCount, cloudFlowCount, botCount, envVarCount, envVarCountScope, envVarCountTenantWide, mediaRatio, siteType, tables[], tableRelationships[], breakdown, estimationMethod, estimationAccuracyPct, truncationSuspected, truncationWarnings, ppcGroundTruthCount }`. **Table discovery is site-referenced, NOT publisher-prefix:** `tableCount`/`tables[]` are scoped to the custom tables the site actually references — its `.powerpages-site/table-permissions/` (+ datamodel manifest) intersected with the env's custom-unmanaged tables (via `resolve-site-tables.js` + `query-metadata.js`). `tableCountScope` ∈ `"site-referenced" | "manifest-only" | "unavailable"` (the last → 0 tables, never an env-wide prefix dump). `--publisherPrefix` now scopes ONLY the env var count, not tables. `tableRelationships[]` are `[a,b]` dependency edges (lookups + N:N, via `query-table-relationships.js`) among the scoped tables, consumed by `compute-split-plan.js` to cluster related tables into the same solution. Metadata-based estimation with ±15% caveat. Web-file size is measured via stratified sample (first 50 + middle 50 + last 50, cap 150) scaled to full count. Disk fields are null unless `--projectRoot` was passed AND a build-output directory was found. Truncation canaries fire when Dataverse pagination disagrees with `@odata.count`, when ppcs land on a page-size boundary, when sampled average bytes/file < 1 KB at scale, or when the disk total exceeds the Dataverse total by >2× — any signal flips `truncationSuspected: true` with a per-cause `truncationWarnings[]` entry. Used by `plan-alm` Phase 1 Step 10. -<<<<<<< HEAD -- `scripts/lib/compute-split-plan.js`: Runs the split decision tree against a size-estimate blob. Args: `--estimate `, `--projectRoot` (opt — for `.alm-config.json` overrides), `--siteName` (opt), `--publisherPrefix` (opt). Output: `{ sizeAnalysis, assetAdvisory, splitStrategy, appliedStrategies, compositeSubPartitioned, proposedSolutions[], recommendations[], truncationSuspected, truncationWarnings }`. Evaluates strategies in priority order: Strategy 3 (Schema Segmentation) → Strategy 1 (Layer Split) → Strategy 2 (Change-Frequency) → Strategy 4 (Config Isolation). **Schema Segmentation is dependency-aware + capacity-bounded:** it builds connected-component clusters from `estimate.tableRelationships` (union-find), then bin-packs whole clusters (never splitting a relationship) into the fewest solutions that keep each under `maxTableCount`/`maxSchemaAttrs`, capped at `maxSchemaSplitSolutions` (default 8). This replaced the old one-solution-per-table-name-stem heuristic that produced ~one solution per table. An indivisible cluster over the cap stays whole and raises an oversized-cluster `recommendations[]` warning. The split trigger + thresholds are unchanged — only the packing. Strategy 4 stacks additively. Strategy 1 also runs a composite sub-partition pass: when Core still exceeds the size OR component-count cap after Web Assets are peeled off, Core is replaced with change-frequency-shaped sub-children (`_Foundation`/`_Config`/`_Content`, plus `_Integration` whenever the parent had any flows or bots — coverage takes priority over the `changeFreqMinFlows` heuristic, which governs only the TOP-LEVEL strategy choice). When additive Strategy 4 is firing concurrently (a top-level `_EnvVars` solution), `_Config` drops `Environment Variable` from its componentTypes to avoid double-claim; when it isn't, `_Config` absorbs env vars so they have an owner. Sub-partitioning sets `compositeSubPartitioned: true` and appends `composite-sub-partition` to `appliedStrategies`. `validateSplits` checks BOTH the size AND component-count cap per split (skipping `isFutureBuffer` solutions). Supports `.alm-config.json` overrides including `strategyOverride` to bypass the tree. See `solution-splitting-logic.md` spec in design docs for full logic. -======= - `scripts/lib/compute-split-plan.js`: Runs the split decision tree against a size-estimate blob. Args: `--estimate `, `--projectRoot` (opt — for `.alm-config.json` overrides), `--siteName` (opt), `--publisherPrefix` (opt). Output: `{ sizeAnalysis, assetAdvisory, splitStrategy, appliedStrategies, compositeSubPartitioned, proposedSolutions[], recommendations[], truncationSuspected, truncationWarnings }`. Evaluates strategies in priority order: Strategy 3 (Schema Segmentation) → Strategy 1 (Layer Split) → Strategy 2 (Change-Frequency) → Strategy 4 (Config Isolation). **Schema Segmentation is dependency-aware + capacity-bounded:** it builds connected-component clusters from `estimate.tableRelationships` (union-find), then bin-packs whole clusters (never splitting a relationship) into the fewest solutions that keep each under `maxTableCount`/`maxSchemaAttrs` **where possible** — capped at `maxSchemaSplitSolutions` (default 8). This replaced the old one-solution-per-table-name-stem heuristic that produced ~one solution per table. Two cases CAN exceed a per-solution cap, and BOTH raise an `recommendations[]` warning rather than failing silently: (a) an indivisible dependency cluster larger than `maxTableCount` stays whole (oversized-cluster table-count warning); (b) when MORE than `maxSchemaSplitSolutions` independent attr-heavy clusters must share the capped solution count, the FFD least-loaded fallback co-locates clusters and a solution's summed columns exceed `maxSchemaAttrs` (oversized-schema attr-cap warning). The split trigger + thresholds are unchanged — only the packing. Strategy 4 stacks additively. Strategy 1 also runs a composite sub-partition pass: when Core still exceeds the size OR component-count cap after Web Assets are peeled off, Core is replaced with change-frequency-shaped sub-children (`_Foundation`/`_Config`/`_Content`, plus `_Integration` whenever the parent had any flows or bots — coverage takes priority over the `changeFreqMinFlows` heuristic, which governs only the TOP-LEVEL strategy choice). When additive Strategy 4 is firing concurrently (a top-level `_EnvVars` solution), `_Config` drops `Environment Variable` from its componentTypes to avoid double-claim; when it isn't, `_Config` absorbs env vars so they have an owner. Sub-partitioning sets `compositeSubPartitioned: true` and appends `composite-sub-partition` to `appliedStrategies`. `validateSplits` checks BOTH the size AND component-count cap per split (skipping `isFutureBuffer` solutions). Supports `.alm-config.json` overrides including `strategyOverride` to bypass the tree. See `solution-splitting-logic.md` spec in design docs for full logic. ->>>>>>> origin/users/nityagi/table-discovery-fix - `scripts/lib/resolve-site-tables.js`: Single source of truth for "which custom tables does this site actually use." `collectReferencedEntityNames({ projectRoot, datamodelManifestPath })` reads `.powerpages-site/table-permissions/*.tablepermission.yml` (`entitylogicalname`, via `powerpages-config.js → loadTablePermissions`) + the datamodel manifest → `{ names:Set, available, sources }`. `scopeCustomTables(referencedNames, customUnmanagedTables)` intersects that set with the env's custom-unmanaged tables. SME-confirmed: table permissions are the complete signal ("if a table is used in the site there will be permissions for it"), so forms/lists are NOT scanned. Used by `estimate-solution-size.js` and `discover-site-components.js` to replace the publisher-prefix table dump. - `scripts/lib/query-metadata.js`: `queryCustomUnmanagedTables(envUrl, token, makeRequest?)` → `[{ logicalName, metadataId, schemaName, displayName }]` (the single `EntityDefinitions?$filter=IsCustomEntity` query, `IsManaged===false` filtered). Consolidates the formerly-triplicated custom-table query (estimator, discover-site-components, setup-solution). Reuses `odataGetAll` from `validation-helpers.js`. - `scripts/lib/query-table-relationships.js`: `fetchTableRelationships(envUrl, table, token, makeRequest?)` → `{ oneToMany[], manyToMany[] }`. Extracted from `skills/audit-permissions/scripts/query-table-relationships.js` (now a thin CLI wrapper over this lib) and extended with ManyToMany. OneToMany errors propagate; ManyToMany is best-effort. Used by the estimator to build `tableRelationships[]` and by audit-permissions for relationship-scope validation. diff --git a/plugins/power-pages/scripts/check-activation-status.js b/plugins/power-pages/scripts/check-activation-status.js index a88d834da..1b3b2962c 100644 --- a/plugins/power-pages/scripts/check-activation-status.js +++ b/plugins/power-pages/scripts/check-activation-status.js @@ -43,53 +43,6 @@ function resolveSiteIdentity(projectRoot, deps = {}) { const _execSync = deps.execSync || execSync; const _readFileSync = deps.readFileSync || fs.readFileSync; -<<<<<<< HEAD -// --- Read site identity from powerpages.config.json (code/SPA sites) OR -// .powerpages-site/website.yml (data-model / enhanced data model "EDM" sites, -// which have no powerpages.config.json). website.yml carries both the site -// name and the website GUID, so EDM sites skip the pac-pages-list lookup below. --- -let siteName; -let websiteRecordId = null; -const configPath = findPath(projectRoot, 'powerpages.config.json'); -if (configPath) { - try { - const config = JSON.parse(fs.readFileSync(configPath, 'utf8')); - siteName = config.siteName; - websiteRecordId = config.websiteRecordId || null; - } catch { - output({ error: 'Failed to parse powerpages.config.json' }); - } -} else { - const websiteYmlPath = findPath(projectRoot, path.join('.powerpages-site', 'website.yml')); - const site = websiteYmlPath ? readWebsiteYml(websiteYmlPath) : null; - if (site) { - siteName = site.name; - websiteRecordId = site.id || null; - } -} -if (!siteName) { - output({ error: 'Site name not found — looked in powerpages.config.json and .powerpages-site/website.yml' }); -} - -// --- Get websiteRecordId from pac pages list (only when not already known, -// e.g. a code site whose config omitted it). EDM sites already have it from website.yml. --- -if (!websiteRecordId) try { - const pacOutput = execSync('pac pages list', { encoding: 'utf8', timeout: 15000 }); - // pac pages list outputs a table with columns. Find the row matching siteName. - // Column headers vary but Website Record ID is always a GUID column. - const lines = pacOutput.split(/\r?\n/).filter((l) => l.trim()); - for (const line of lines) { - // Skip header/separator lines - if (line.includes('----') || line.toLowerCase().includes('website name')) continue; - // Check if this line contains our site name (case-insensitive) - if (line.toLowerCase().includes(siteName.toLowerCase())) { - // Extract GUID from the line - const guidMatch = line.match(/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/i); - if (guidMatch) { - websiteRecordId = guidMatch[0]; - } - break; -======= let siteName = null; let websiteRecordId = null; let source = null; @@ -111,7 +64,6 @@ if (!websiteRecordId) try { siteName = site.name; websiteRecordId = site.id || null; source = 'website.yml'; ->>>>>>> origin/users/nityagi/table-discovery-fix } } diff --git a/plugins/power-pages/scripts/lib/compute-split-plan.js b/plugins/power-pages/scripts/lib/compute-split-plan.js index 233d842dd..8a41d8ad5 100644 --- a/plugins/power-pages/scripts/lib/compute-split-plan.js +++ b/plugins/power-pages/scripts/lib/compute-split-plan.js @@ -332,19 +332,11 @@ function partitionBySchema(estimate, meta, config) { componentTypes: ['Table'], description: `Schema domain: ${dom.name}. Tables: ${(dom.tableLogicalNames || []).join(', ') || '(derived)'}${domainDescSuffix}`, sizeMB: round(sizePerDomain), -<<<<<<< HEAD - // A Table domain's component count IS its table count when known (each table - // is one Entity solution component). Falls back to an even attr-share split - // only for explicit domains that didn't list their tables. - componentCount: (dom.tableLogicalNames && dom.tableLogicalNames.length > 0) - ? dom.tableLogicalNames.length -======= // Schema-component proxy when the domain's tables are known (sum of columns + // 1/table); falls back to an even attr-share split only for explicit domains // that didn't list their tables. componentCount: (dom.tableLogicalNames && dom.tableLogicalNames.length > 0) ? schemaComponentProxy(dom.tableLogicalNames) ->>>>>>> origin/users/nityagi/table-discovery-fix : Math.ceil((estimate.schemaAttrCount || 0) / domainCount), components: [], tableLogicalNames: dom.tableLogicalNames || [], @@ -817,8 +809,6 @@ function computeSplitPlan({ estimate, config, meta }) { type: 'warning', message: `Solution ${s.uniqueName} holds ${s.tableLogicalNames.length} related tables — above the ${config.thresholds.maxTableCount}-per-solution cap — because they form one dependency cluster that cannot be split without breaking a relationship. Consider denormalizing the schema or raising maxTableCount in .alm-config.json.`, })); -<<<<<<< HEAD -======= // Oversized-SCHEMA guard (companion to the table-count guard above): a Table // solution whose summed column count exceeds maxSchemaAttrs. This fires at the // `maxSchemaSplitSolutions` ceiling — when MORE than that many independent @@ -839,7 +829,6 @@ function computeSplitPlan({ estimate, config, meta }) { type: 'warning', message: `Solution ${s.uniqueName} holds tables totaling ${solutionSchemaAttrs(s)} columns — above the ${config.thresholds.maxSchemaAttrs}-column per-solution cap. This happens when more than ${config.thresholds.maxSchemaSplitSolutions} independent attr-heavy table clusters must share the capped number of schema-split solutions. Consider raising maxSchemaSplitSolutions (or maxSchemaAttrs) in .alm-config.json, or denormalizing the widest tables.`, })); ->>>>>>> origin/users/nityagi/table-discovery-fix // Surface estimator-side truncation warnings as `recommendations[]` entries // so the rendered plan shows them inline. These get the `error` type because // a truncated input is more dangerous than a normal split-decision warning @@ -852,12 +841,8 @@ function computeSplitPlan({ estimate, config, meta }) { const recommendations = truncationRecs .concat(buildRecommendations(estimate, strategy, config)) .concat(splitWarnings) -<<<<<<< HEAD - .concat(oversizedClusterWarnings); -======= .concat(oversizedClusterWarnings) .concat(oversizedAttrWarnings); ->>>>>>> origin/users/nityagi/table-discovery-fix const appliedStrategies = [strategy.primary]; if (strategy.additive) appliedStrategies.push('strategy-4-config-isolation'); diff --git a/plugins/power-pages/scripts/lib/detect-project-context.js b/plugins/power-pages/scripts/lib/detect-project-context.js index 3c9a4e10d..6c4174f41 100644 --- a/plugins/power-pages/scripts/lib/detect-project-context.js +++ b/plugins/power-pages/scripts/lib/detect-project-context.js @@ -1,19 +1,6 @@ #!/usr/bin/env node // Reads Power Pages project context files from the project root. -<<<<<<< HEAD -// Locates powerpages.config.json (code/SPA sites) OR .powerpages-site/website.yml -// (data-model config sites, standard and enhanced data model), plus -// .solution-manifest.json and .datamodel-manifest.json. -// -// Site identity resolution order (first match wins): -// 1. powerpages.config.json -> siteType "code" (SPA sites; has siteName, -// websiteRecordId, environmentUrl) -// 2. .powerpages-site/website.yml -> siteType "data-model" (enhanced/standard -// data-model sites from `pac pages download`; the -// YAML carries `id` and `name`, but no environment URL — -// callers re-confirm the env via `pac env who`) -======= // Locates powerpages.config.json (code/SPA sites) OR a .powerpages-site/ config tree // (declarative "data-model" sites — Power Pages design-studio sites), plus // .solution-manifest.json and .datamodel-manifest.json. @@ -35,7 +22,6 @@ // via `pac env who`. `.portalconfig/` is the positive // declarative marker; BOTH site types carry website.yml, // so it isn't a reliable "declarative" signal alone.) ->>>>>>> origin/users/nityagi/table-discovery-fix // // Usage: node detect-project-context.js [--projectRoot ] // @@ -97,8 +83,6 @@ function readWebsiteYml(filePath) { if (!m) continue; const key = m[1]; let value = m[2].trim(); -<<<<<<< HEAD -======= // Strip an inline YAML comment ( ` #...` preceded by whitespace) on UNQUOTED // values — `id: abc # note` -> `abc`. A `#` inside quotes, or with no leading // space, is left intact. (pac-downloaded website.yml is flat + uncommented, so @@ -109,7 +93,6 @@ function readWebsiteYml(filePath) { if (!looksQuoted) { value = value.replace(/\s+#.*$/, '').trim(); } ->>>>>>> origin/users/nityagi/table-discovery-fix // Strip surrounding quotes a YAML writer may add. if ((value.startsWith('"') && value.endsWith('"')) || (value.startsWith("'") && value.endsWith("'"))) { @@ -120,8 +103,6 @@ function readWebsiteYml(filePath) { return (out.id || out.name) ? out : null; } -<<<<<<< HEAD -======= function isDirectory(p) { try { return fs.statSync(p).isDirectory(); @@ -130,7 +111,6 @@ function isDirectory(p) { } } ->>>>>>> origin/users/nityagi/table-discovery-fix function detectProjectContext(options = {}) { const startDir = options.projectRoot || process.cwd(); const projectRoot = options.projectRoot @@ -166,16 +146,6 @@ function detectProjectContext(options = {}) { }; } -<<<<<<< HEAD - // 2. Data-model (standard/enhanced) site — identity comes from - // .powerpages-site/website.yml (`id` -> websiteRecordId, `name` -> siteName). - // There is no environment URL in the local files; callers re-confirm via `pac env who`. - const websiteYmlPath = path.join(projectRoot, '.powerpages-site', 'website.yml'); - if (fs.existsSync(websiteYmlPath)) { - const site = readWebsiteYml(websiteYmlPath); - if (!site) { - throw new Error(`Could not read site id/name from: ${websiteYmlPath}`); -======= // 2. Declarative ("data-model") site — a Power Pages design-studio site // (`pac pages download`; standard or enhanced data model), as opposed to a // code/SPA site. The authoritative positive marker is the @@ -194,18 +164,12 @@ function detectProjectContext(options = {}) { if (!site) { throw new Error(`Could not read site id/name from: ${websiteYmlPath}`); } ->>>>>>> origin/users/nityagi/table-discovery-fix } return { projectRoot, siteType: 'data-model', -<<<<<<< HEAD - siteName: site.name || null, - websiteRecordId: site.id || null, -======= siteName: site ? (site.name || null) : null, websiteRecordId: site ? (site.id || null) : null, ->>>>>>> origin/users/nityagi/table-discovery-fix environmentUrl: null, solutionManifest, datamodelManifest, diff --git a/plugins/power-pages/scripts/lib/estimate-solution-size.js b/plugins/power-pages/scripts/lib/estimate-solution-size.js index 91a50266f..4a29b7cfe 100644 --- a/plugins/power-pages/scripts/lib/estimate-solution-size.js +++ b/plugins/power-pages/scripts/lib/estimate-solution-size.js @@ -353,15 +353,6 @@ async function discoverTableRelationships(envUrl, tables, token) { seen.add(key); edges.push(a < b ? [a, b] : [b, a]); }; -<<<<<<< HEAD - for (const t of tables) { - let rel; - try { - rel = await fetchTableRelationships(envUrl, t.logicalName, token); - } catch { - continue; // inaccessible table — skip its edges - } -======= // Fetch each table's relationships with BOUNDED CONCURRENCY (~2 OData calls per // table; a 34-table site is 68 round-trips — serial is slow at plan time). Edge // assembly stays sequential, in table order, so dedup is deterministic. @@ -383,7 +374,6 @@ async function discoverTableRelationships(envUrl, tables, token) { ); for (const rel of results) { if (!rel) continue; ->>>>>>> origin/users/nityagi/table-discovery-fix for (const e of rel.oneToMany) addEdge(e.referencedEntity, e.referencingEntity); for (const e of rel.manyToMany) addEdge(e.entity1, e.entity2); } diff --git a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js index e64d64cda..364625520 100644 --- a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js +++ b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js @@ -36,7 +36,6 @@ // finalize: // - PLAN_STATUS = "Completed" // -<<<<<<< HEAD // After every phase's step-sync, a completion evaluator flips PLAN_STATUS to // "Completed" (+ COMPLETED_AT) once all non-skip steps are completed and none // failed — so the last execution skill terminates the plan automatically, @@ -44,8 +43,6 @@ // Approved -> In Execution promotion that starts the lifecycle lives in // check-alm-plan.js (first execution-skill Phase 0). // -======= ->>>>>>> origin/users/nityagi/table-discovery-fix // stdout JSON includes `nextStep: { name, skill: string | null } | null` (when ok:true) — the first // still-pending checklist step and the slash command that runs it. Execution // skills echo this so the user knows the next step to invoke (user-driven @@ -1122,10 +1119,7 @@ function reconcile({ projectRoot, render, rendererPath }) { process.stderr.write(`[refresh-alm-plan-data] reconcile phase "${phase}" failed: ${e.message}\n`); } } -<<<<<<< HEAD evaluatePlanCompletion(planData); -======= ->>>>>>> origin/users/nityagi/table-discovery-fix fs.writeFileSync(dataPath, JSON.stringify(planData, null, 2), 'utf8'); let rendered = false; diff --git a/plugins/power-pages/scripts/lib/resolve-site-tables.js b/plugins/power-pages/scripts/lib/resolve-site-tables.js index 7853e6395..e58ce46ec 100644 --- a/plugins/power-pages/scripts/lib/resolve-site-tables.js +++ b/plugins/power-pages/scripts/lib/resolve-site-tables.js @@ -49,8 +49,6 @@ function collectReferencedEntityNames({ projectRoot, datamodelManifestPath } = { const dir = path.join(projectRoot, '.powerpages-site', 'table-permissions'); if (fs.existsSync(dir)) { sawTablePermissionsDir = true; -<<<<<<< HEAD -======= // Count the permission FILES present — `sources.tablePermissions` must mean // "the site has table-permission files" (a reliable site-referenced signal, // consumed by `tableCountScope`), NOT "we parsed at least one record". A @@ -59,17 +57,12 @@ function collectReferencedEntityNames({ projectRoot, datamodelManifestPath } = { sources.tablePermissions = fs.readdirSync(dir) .filter((f) => /\.tablepermission\.yml$/i.test(f)).length; } catch { /* keep 0 */ } ->>>>>>> origin/users/nityagi/table-discovery-fix let records = []; try { records = loadTablePermissions(dir); } catch { records = []; } for (const r of records) { const name = r && r.entitylogicalname; // NOT entityname (that's the display label) if (typeof name === 'string' && name.trim()) { names.add(name.trim().toLowerCase()); -<<<<<<< HEAD - sources.tablePermissions += 1; -======= ->>>>>>> origin/users/nityagi/table-discovery-fix } } } diff --git a/plugins/power-pages/scripts/lib/validation-helpers.js b/plugins/power-pages/scripts/lib/validation-helpers.js index c9a6fb0d4..b9a097ee5 100644 --- a/plugins/power-pages/scripts/lib/validation-helpers.js +++ b/plugins/power-pages/scripts/lib/validation-helpers.js @@ -93,18 +93,11 @@ function findPath(dir, target) { * * A project root is marked by EITHER: * - `powerpages.config.json` — code/SPA sites (`pac pages download-code-site`), OR -<<<<<<< HEAD - * - a `.powerpages-site/` directory — data-model config sites (`pac pages download`, - * standard or enhanced data model). These have NO `powerpages.config.json`. - * - * Code sites have both markers; data-model (e.g. enhanced data model) sites have only -======= * - a `.powerpages-site/` directory — declarative ("data-model") design-studio sites * (`pac pages download`; standard or enhanced data model). These have NO * `powerpages.config.json`. * * Code sites have both markers; declarative sites have only ->>>>>>> origin/users/nityagi/table-discovery-fix * `.powerpages-site/`. Checking for either makes root discovery work for both site types. * * @returns {string|null} Project root path, or null @@ -272,15 +265,11 @@ async function odataGet(url, token, request = makeRequest) { /** * Follows `@odata.nextLink`, aggregating every page's `value[]` into one array. -<<<<<<< HEAD - * `maxPages` is a runaway-loop safety cap (100 × 5000 ≈ 500K rows). -======= * `maxPages` is a runaway-loop safety cap (100 × 5000 ≈ 500K rows). FAILS CLOSED: * if the cap is reached while `@odata.nextLink` is still present, throws rather than * silently returning a truncated set — a partial result would produce wrong * table/env-var counts for ALM sizing/splitting with no signal. Callers that want * partial results must catch and downgrade accuracy intentionally. ->>>>>>> origin/users/nityagi/table-discovery-fix * @param {string} url - absolute starting URL * @param {string} token - bearer token * @param {Function} [request=makeRequest] - injectable for tests @@ -290,25 +279,18 @@ async function odataGet(url, token, request = makeRequest) { async function odataGetAll(url, token, request = makeRequest, maxPages = 100) { const out = []; let next = url; -<<<<<<< HEAD - for (let p = 0; p < maxPages && next; p++) { -======= let p = 0; for (; p < maxPages && next; p++) { ->>>>>>> origin/users/nityagi/table-discovery-fix const page = await odataGet(next, token, request); if (Array.isArray(page.value)) out.push(...page.value); next = page['@odata.nextLink'] || null; } -<<<<<<< HEAD -======= if (next) { throw new Error( `odataGetAll hit the ${maxPages}-page cap with @odata.nextLink still present ` + `(${out.length} rows so far) — result would be truncated. Raise maxPages or narrow the query.`, ); } ->>>>>>> origin/users/nityagi/table-discovery-fix return out; } diff --git a/plugins/power-pages/scripts/tests/compute-split-plan.test.js b/plugins/power-pages/scripts/tests/compute-split-plan.test.js index 905b22095..34ea3741e 100644 --- a/plugins/power-pages/scripts/tests/compute-split-plan.test.js +++ b/plugins/power-pages/scripts/tests/compute-split-plan.test.js @@ -236,8 +236,6 @@ test('computeSplitPlan Strategy 3 never overflows the attr cap when independent for (const s of tableSolutions) { assert.equal(s.tableLogicalNames.length, 1, `${s.uniqueName} must hold exactly one table — no bucket over the attr cap`); } -<<<<<<< HEAD -======= }); test('computeSplitPlan WARNS when >maxSchemaSplitSolutions independent attr-heavy clusters bust the attr cap (ceiling boundary)', () => { @@ -278,7 +276,6 @@ test('computeSplitPlan: a Table domain componentCount is a schema-component prox const catalog = result.proposedSolutions.find((s) => s.uniqueName === 'Test_Catalog'); assert.ok(catalog, 'Catalog Table domain solution exists'); assert.equal(catalog.componentCount, 802, 'sum(500+300) + 2 tables = 802 (proxy), not 2 (table count)'); ->>>>>>> origin/users/nityagi/table-discovery-fix }); test('computeSplitPlan additive Strategy 4 prepends EnvVars solution', () => { diff --git a/plugins/power-pages/scripts/tests/detect-project-context.test.js b/plugins/power-pages/scripts/tests/detect-project-context.test.js index f7b32d947..2a682cb36 100644 --- a/plugins/power-pages/scripts/tests/detect-project-context.test.js +++ b/plugins/power-pages/scripts/tests/detect-project-context.test.js @@ -30,15 +30,9 @@ test('detectProjectContext: code site (powerpages.config.json) reports siteType assert.equal(result.siteName, 'Code Site'); }); -<<<<<<< HEAD -test('detectProjectContext: enhanced data-model site resolves identity from .powerpages-site/website.yml', (t) => { - const projectRoot = createTempProject(t); - // No powerpages.config.json — this is an EDM / data-model config site. -======= test('detectProjectContext: declarative (data-model) site resolves identity from .powerpages-site/website.yml', (t) => { const projectRoot = createTempProject(t); // No powerpages.config.json — this is a declarative ("data-model") design-studio site. ->>>>>>> origin/users/nityagi/table-discovery-fix writeProjectFile( projectRoot, '.powerpages-site/website.yml', @@ -59,8 +53,6 @@ test('detectProjectContext: declarative (data-model) site resolves identity from assert.equal(result.environmentUrl, null); }); -<<<<<<< HEAD -======= test('detectProjectContext: .powerpages-site/.portalconfig/ is the positive declarative signal (even without website.yml)', (t) => { const projectRoot = createTempProject(t); // A declarative site identified by its .portalconfig/ marker, with no website.yml @@ -74,7 +66,6 @@ test('detectProjectContext: .powerpages-site/.portalconfig/ is the positive decl assert.equal(result.environmentUrl, null); }); ->>>>>>> origin/users/nityagi/table-discovery-fix test('detectProjectContext: config site wins over website.yml when both exist (code-site precedence)', (t) => { const projectRoot = createTempProject(t); writeProjectFile(projectRoot, 'powerpages.config.json', JSON.stringify({ diff --git a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js index 17099f55b..ab23aecea 100644 --- a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js +++ b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js @@ -1952,100 +1952,3 @@ test('reconcile: soft no-op when there is no plan', (t) => { assert.equal(result.ok, false); assert.equal(result.reason, 'no-plan'); }); -<<<<<<< HEAD - -// --- Gap 5: completion evaluator (In Execution -> Completed) ----------------- - -test('completion: a refresh that leaves every non-skip step completed flips PLAN_STATUS to Completed', (t) => { - const root = makeProject(t); - writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { - SITE_NAME: 'T', STRATEGY: 'pp-pipelines', PLAN_STATUS: 'In Execution', - validationRuns: { Staging: null }, - steps: [ - { name: 'Setup solution', status: 'completed' }, - { name: 'Deploy via pipeline to Staging', status: 'completed' }, - { name: 'Test site in Staging', status: 'pending' }, - ], - }); - // test-site flips the last pending step -> all done -> plan completes. - writeJson(path.join(root, 'docs', 'alm', 'last-test-site.json'), { runOutcome: 'passed', runAt: '2026-06-16T00:00:00.000Z' }); - - refresh({ projectRoot: root, phase: 'test-site', render: false, stageName: 'Staging' }); - - const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); - assert.equal(planData.PLAN_STATUS, 'Completed', 'all steps done -> Completed'); - assert.ok(planData.COMPLETED_AT, 'COMPLETED_AT stamped'); -}); - -test('completion: a still-pending step keeps PLAN_STATUS at In Execution', (t) => { - const root = makeProject(t); - writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { - SITE_NAME: 'T', STRATEGY: 'pp-pipelines', PLAN_STATUS: 'In Execution', - steps: [ - { name: 'Deploy via pipeline to Staging', status: 'pending' }, - { name: 'Deploy via pipeline to Production', status: 'pending' }, - ], - }); - writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { stageName: 'Staging', status: 'Succeeded', deployedAt: '2026-06-16T00:00:00.000Z' }); - - refresh({ projectRoot: root, phase: 'deploy-pipeline', render: false }); - - const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); - assert.equal(planData.steps[0].status, 'completed', 'Staging deploy flipped'); - assert.equal(planData.PLAN_STATUS, 'In Execution', 'Production still pending -> not Completed'); - assert.equal(planData.COMPLETED_AT, undefined); -}); - -test('completion: a failed step blocks completion even if all others are done', (t) => { - const root = makeProject(t); - writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { - SITE_NAME: 'T', STRATEGY: 'pp-pipelines', PLAN_STATUS: 'In Execution', - steps: [ - { name: 'Setup solution', status: 'completed' }, - { name: 'Deploy via pipeline to Staging', status: 'pending' }, - ], - }); - // A FAILED deploy marker flips the deploy step to 'failed', not 'completed'. - writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { stageName: 'Staging', status: 'Failed', deployedAt: '2026-06-16T00:00:00.000Z' }); - - refresh({ projectRoot: root, phase: 'deploy-pipeline', render: false }); - - const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); - assert.equal(planData.steps[1].status, 'failed'); - assert.equal(planData.PLAN_STATUS, 'In Execution', 'a failed step must NOT complete the plan'); -}); - -test('completion: skip:true steps are ignored when deciding completion', (t) => { - const root = makeProject(t); - writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { - SITE_NAME: 'T', STRATEGY: 'pp-pipelines', PLAN_STATUS: 'In Execution', - validationRuns: { Staging: null }, - steps: [ - { name: 'Deploy via pipeline to Staging', status: 'completed' }, - { name: 'Test site in Staging', status: 'pending', skip: true }, - ], - }); - // Re-run a no-op-ish refresh; the skipped Test step must not block completion. - writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { stageName: 'Staging', status: 'Succeeded', deployedAt: '2026-06-16T00:00:00.000Z' }); - - refresh({ projectRoot: root, phase: 'deploy-pipeline', render: false }); - - const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); - assert.equal(planData.PLAN_STATUS, 'Completed', 'only non-skip steps count -> Completed'); -}); - -test('completion: a Draft plan is never auto-completed', (t) => { - const root = makeProject(t); - writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { - SITE_NAME: 'T', STRATEGY: 'pp-pipelines', PLAN_STATUS: 'Draft', - steps: [{ name: 'Deploy via pipeline to Staging', status: 'completed' }], - }); - writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { stageName: 'Staging', status: 'Succeeded', deployedAt: '2026-06-16T00:00:00.000Z' }); - - refresh({ projectRoot: root, phase: 'deploy-pipeline', render: false }); - - const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); - assert.equal(planData.PLAN_STATUS, 'Draft', 'Draft is not a pre-terminal execution state'); -}); -======= ->>>>>>> origin/users/nityagi/table-discovery-fix diff --git a/plugins/power-pages/scripts/tests/resolve-site-tables.test.js b/plugins/power-pages/scripts/tests/resolve-site-tables.test.js index 6468ed238..f806f5b29 100644 --- a/plugins/power-pages/scripts/tests/resolve-site-tables.test.js +++ b/plugins/power-pages/scripts/tests/resolve-site-tables.test.js @@ -96,8 +96,6 @@ test('scopeCustomTables: keeps only referenced custom tables; drops unreferenced test('scopeCustomTables: empty referenced set -> empty (never a prefix dump)', () => { assert.deepEqual(scopeCustomTables(new Set(), [{ logicalName: 'new_x' }]), []); }); -<<<<<<< HEAD -======= test('collectReferencedEntityNames: sources.tablePermissions reflects FILE existence even when files are unparseable', (t) => { const root = makeProject(t); @@ -113,4 +111,3 @@ test('collectReferencedEntityNames: sources.tablePermissions reflects FILE exist assert.equal(sources.tablePermissions, 1, 'counts the permission FILE, not parsed records (0 parsed here)'); assert.equal(names.size, 0, 'no entity names parsed from the malformed file'); }); ->>>>>>> origin/users/nityagi/table-discovery-fix diff --git a/plugins/power-pages/scripts/tests/validation-helpers.test.js b/plugins/power-pages/scripts/tests/validation-helpers.test.js index 44b69c130..0e739fe9a 100644 --- a/plugins/power-pages/scripts/tests/validation-helpers.test.js +++ b/plugins/power-pages/scripts/tests/validation-helpers.test.js @@ -86,8 +86,6 @@ test('odataGetAll follows @odata.nextLink and aggregates all pages', async () => assert.deepEqual(rows.map((r) => r.id), [1, 2, 3]); }); -<<<<<<< HEAD -======= test('odataGetAll FAILS CLOSED: throws when it hits maxPages with @odata.nextLink still present', async () => { const { odataGetAll } = require(helpersPath); // Every page advertises a nextLink → never terminates → hits the page cap. @@ -102,7 +100,6 @@ test('odataGetAll FAILS CLOSED: throws when it hits maxPages with @odata.nextLin ); }); ->>>>>>> origin/users/nityagi/table-discovery-fix test('odataGet throws on non-2xx', async () => { const { odataGet } = require(helpersPath); const fakeRequest = async () => ({ statusCode: 404, body: 'not found' }); From 673ae8619e7048a87b57f67b88bf4b870de25aef Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Fri, 19 Jun 2026 16:14:47 +0530 Subject: [PATCH 24/38] Fix incomplete Open Plugins resolution in the merge (#194) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The "Merge main into branch" resolution (525405a5) left three problems that broke validate-repository-metadata + the env-var convention: - Restored the `.claude-plugin/plugin.json` legacy symlink (the merge deleted it → validator "missing legacy plugin manifest"). - marketplace.json power-pages version 2.4.0 -> 2.5.0 (it was left out of sync with .plugin/plugin.json's 2.5.0 → validator version-mismatch). - ensure-pipelines-host SKILL.md:906 (this PR's Phase 6 refresh call) ${CLAUDE_PLUGIN_ROOT} -> ${PLUGIN_ROOT} (post Open Plugins migration convention). validate-legacy-compatibility: metadata in sync. 1238 tests pass, alm-lint 0. Co-Authored-By: Claude Opus 4.8 (1M context) --- marketplace.json | 2 +- plugins/power-pages/.claude-plugin/plugin.json | 1 + plugins/power-pages/skills/ensure-pipelines-host/SKILL.md | 2 +- 3 files changed, 3 insertions(+), 2 deletions(-) create mode 120000 plugins/power-pages/.claude-plugin/plugin.json diff --git a/marketplace.json b/marketplace.json index 119399ebc..207f7a55e 100644 --- a/marketplace.json +++ b/marketplace.json @@ -13,7 +13,7 @@ "source": "./plugins/power-pages", "description": "Power Pages development and management plugin for Claude Code and GitHub Copilot", "category": "development", - "version": "2.4.0", + "version": "2.5.0", "license": "MIT", "tags": [ "power platform", diff --git a/plugins/power-pages/.claude-plugin/plugin.json b/plugins/power-pages/.claude-plugin/plugin.json new file mode 120000 index 000000000..a3eddf24e --- /dev/null +++ b/plugins/power-pages/.claude-plugin/plugin.json @@ -0,0 +1 @@ +../.plugin/plugin.json \ No newline at end of file diff --git a/plugins/power-pages/skills/ensure-pipelines-host/SKILL.md b/plugins/power-pages/skills/ensure-pipelines-host/SKILL.md index 413671853..f639b77ff 100644 --- a/plugins/power-pages/skills/ensure-pipelines-host/SKILL.md +++ b/plugins/power-pages/skills/ensure-pipelines-host/SKILL.md @@ -903,7 +903,7 @@ Follow the skill tracking instructions in the reference to record this skill's u **Refresh the ALM plan (if one exists):** ```bash -node "${CLAUDE_PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ +node "${PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ --projectRoot "." \ --phase ensure-pipelines-host \ --render From 1f5ddd20b6c778f5245fa4a569ffcc2ac3b6c852 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Mon, 22 Jun 2026 10:57:07 +0530 Subject: [PATCH 25/38] Address #194 review: DRY plan path + hook smoke test + completion-edge tests MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three minor findings from the /review 194 pass: 1. Hardcoded plan path — `docs/.alm-plan-data.json` / `docs/alm-plan.html` were inlined at four call-sites (the PostToolUse hook, check-alm-plan.js, and refresh-alm-plan-data.js reconcile + refresh). Centralize them in alm-paths.js as `planDataPath()` / `planHtmlPath()` (these artifacts live at the docs/ ROOT, not docs/alm/) and route all four callers through the helper so the path can't drift. + tests guarding the docs-root invariant. 2. Coverage gap — no automated test for the hook's reconcile spawn. Add run-skill-posttool-validation.test.js: the backstop heals a skipped refresh after an ALM skill, is skipped for non-ALM skills, and is exit-code-neutral (a blocking validator's status is unchanged whether or not the plan exists). 3. Completion edge — pin evaluatePlanCompletion's status gate: the defensive Approved → Completed fallback completes, a Draft plan never auto-completes, and a failed step blocks completion so a failed deploy can't look "done". Full suite: 1246 pass. legacy-compat + alm-lint: clean. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../hooks/run-skill-posttool-validation.js | 3 +- plugins/power-pages/scripts/lib/alm-paths.js | 21 +++ .../power-pages/scripts/lib/check-alm-plan.js | 5 +- .../scripts/lib/refresh-alm-plan-data.js | 10 +- .../scripts/tests/alm-paths.test.js | 19 ++- .../tests/refresh-alm-plan-data.test.js | 68 +++++++++ .../run-skill-posttool-validation.test.js | 139 ++++++++++++++++++ 7 files changed, 256 insertions(+), 9 deletions(-) create mode 100644 plugins/power-pages/scripts/tests/run-skill-posttool-validation.test.js diff --git a/plugins/power-pages/hooks/run-skill-posttool-validation.js b/plugins/power-pages/hooks/run-skill-posttool-validation.js index ee1bace99..df36ea2d7 100644 --- a/plugins/power-pages/hooks/run-skill-posttool-validation.js +++ b/plugins/power-pages/hooks/run-skill-posttool-validation.js @@ -8,6 +8,7 @@ const { getValidatorScript, isAlmPlanSkill, } = require('../scripts/lib/powerpages-hook-utils'); +const { planDataPath } = require('../scripts/lib/alm-paths'); const DEBUG = process.env.DEBUG === '1' || process.env.DEBUG === 'true'; @@ -62,7 +63,7 @@ process.stdin.on('end', () => { // changes the hook's exit code (the validator's status stands). Triggering on // any ALM skill (not just the marker's writer) catches a skip that surfaces only // when the NEXT ALM skill runs. Honors .alm-deferred + no-plan inside reconcile. - if (isAlmPlanSkill(skillName) && fs.existsSync(path.join(cwd, 'docs', '.alm-plan-data.json'))) { + if (isAlmPlanSkill(skillName) && fs.existsSync(planDataPath(cwd))) { try { const refreshPath = path.join(__dirname, '..', 'scripts', 'lib', 'refresh-alm-plan-data.js'); const rec = spawnSync(process.execPath, [refreshPath, '--projectRoot', cwd, '--reconcile', '--render'], { diff --git a/plugins/power-pages/scripts/lib/alm-paths.js b/plugins/power-pages/scripts/lib/alm-paths.js index b0546e081..2c09af8d3 100644 --- a/plugins/power-pages/scripts/lib/alm-paths.js +++ b/plugins/power-pages/scripts/lib/alm-paths.js @@ -83,10 +83,31 @@ function ensureAlmDir(projectRoot) { return dir; } +// The rendered plan + its backing JSON live at the `docs/` ROOT (NOT under +// `docs/alm/`): `docs/alm-plan.html` and `docs/.alm-plan-data.json`. Centralized +// here so the PostToolUse hook, check-alm-plan.js, and refresh-alm-plan-data.js +// can't drift on the path. +const PLAN_DATA_FILE = '.alm-plan-data.json'; +const PLAN_HTML_FILE = 'alm-plan.html'; + +function planDataPath(projectRoot) { + if (!projectRoot) throw new Error('planDataPath: projectRoot is required'); + return path.join(projectRoot, 'docs', PLAN_DATA_FILE); +} + +function planHtmlPath(projectRoot) { + if (!projectRoot) throw new Error('planHtmlPath: projectRoot is required'); + return path.join(projectRoot, 'docs', PLAN_HTML_FILE); +} + module.exports = { ALM_DIR, FILE_NAMES, + PLAN_DATA_FILE, + PLAN_HTML_FILE, almDir, almPath, ensureAlmDir, + planDataPath, + planHtmlPath, }; diff --git a/plugins/power-pages/scripts/lib/check-alm-plan.js b/plugins/power-pages/scripts/lib/check-alm-plan.js index 57f11524e..e379e6153 100644 --- a/plugins/power-pages/scripts/lib/check-alm-plan.js +++ b/plugins/power-pages/scripts/lib/check-alm-plan.js @@ -80,6 +80,7 @@ const fs = require('fs'); const path = require('path'); const helpers = require('./validation-helpers'); +const { planDataPath, planHtmlPath } = require('./alm-paths'); // Heartbeat window — how recent `lastInvocationAt` must be for the plan to count // as actively executing. 60 minutes is comfortably larger than the longest single @@ -185,8 +186,8 @@ function readDeferralLocal(projectRoot) { async function checkAlmPlan({ projectRoot, envUrl, token, solutionId, makeRequest, writeHeartbeat = true, now }) { if (!projectRoot) throw new Error('--projectRoot is required'); - const planPath = path.join(projectRoot, 'docs', '.alm-plan-data.json'); - const htmlPath = path.join(projectRoot, 'docs', 'alm-plan.html'); + const planPath = planDataPath(projectRoot); + const htmlPath = planHtmlPath(projectRoot); const nowMs = (typeof now === 'number') ? now : Date.now(); // Deferral marker check — runs first, regardless of plan presence. diff --git a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js index 364625520..27d912fe8 100644 --- a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js +++ b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js @@ -56,7 +56,7 @@ const fs = require('fs'); const path = require('path'); const { execFileSync } = require('child_process'); -const { almPath } = require('./alm-paths'); +const { almPath, planDataPath, planHtmlPath } = require('./alm-paths'); const PHASES = new Set([ 'setup-solution', @@ -1049,8 +1049,8 @@ function mtimeMs(filePath) { // marker schema the refresh can't parse); the reconcile still heals the other phases. function reconcile({ projectRoot, render, rendererPath }) { if (!projectRoot) throw new Error('--projectRoot is required'); - const dataPath = path.join(projectRoot, 'docs', '.alm-plan-data.json'); - const htmlPath = path.join(projectRoot, 'docs', 'alm-plan.html'); + const dataPath = planDataPath(projectRoot); + const htmlPath = planHtmlPath(projectRoot); // Respect the project-level ALM opt-out. nextStep is null on these early paths // (there's nothing to guide toward), kept on the return for a stable contract so @@ -1151,8 +1151,8 @@ function refresh({ projectRoot, phase, render, rendererPath, stageName }) { throw new Error('--phase must be one of: ' + [...PHASES].join(', ')); } - const dataPath = path.join(projectRoot, 'docs', '.alm-plan-data.json'); - const htmlPath = path.join(projectRoot, 'docs', 'alm-plan.html'); + const dataPath = planDataPath(projectRoot); + const htmlPath = planHtmlPath(projectRoot); if (!fs.existsSync(dataPath)) { return { diff --git a/plugins/power-pages/scripts/tests/alm-paths.test.js b/plugins/power-pages/scripts/tests/alm-paths.test.js index c6c4903ae..6aef5a4a3 100644 --- a/plugins/power-pages/scripts/tests/alm-paths.test.js +++ b/plugins/power-pages/scripts/tests/alm-paths.test.js @@ -6,7 +6,7 @@ const fs = require('fs'); const path = require('path'); const os = require('os'); -const { almDir, almPath, ensureAlmDir, FILE_NAMES, ALM_DIR } = require('../lib/alm-paths'); +const { almDir, almPath, ensureAlmDir, FILE_NAMES, ALM_DIR, planDataPath, planHtmlPath } = require('../lib/alm-paths'); function makeTmp(t) { const root = fs.mkdtempSync(path.join(os.tmpdir(), 'alm-paths-')); @@ -73,6 +73,23 @@ test('ensureAlmDir is idempotent when docs/alm/ already exists', (t) => { assert.equal(fs.readFileSync(path.join(root, 'docs', 'alm', 'sentinel.txt'), 'utf8'), 'pre-existing'); }); +test('planDataPath / planHtmlPath resolve to the docs/ ROOT, not docs/alm/', () => { + const root = path.join(path.sep, 'tmp', 'project'); + // The rendered plan + backing JSON intentionally live at docs/ (NOT docs/alm/). + assert.equal(planDataPath(root), path.join(root, 'docs', '.alm-plan-data.json')); + assert.equal(planHtmlPath(root), path.join(root, 'docs', 'alm-plan.html')); + // Guard the dotfile + non-alm-subdir invariant the helpers exist to centralize. + assert.ok(planDataPath(root).endsWith(path.join('docs', '.alm-plan-data.json'))); + assert.ok(!planDataPath(root).includes(path.join('docs', 'alm', '')), 'plan data must NOT be under docs/alm/'); +}); + +test('planDataPath / planHtmlPath throw when projectRoot is missing', () => { + assert.throws(() => planDataPath(undefined), /projectRoot is required/); + assert.throws(() => planDataPath(''), /projectRoot is required/); + assert.throws(() => planHtmlPath(undefined), /projectRoot is required/); + assert.throws(() => planHtmlPath(''), /projectRoot is required/); +}); + test('every FILE_NAMES entry resolves via almPath without error', () => { const root = path.join(path.sep, 'tmp', 'project'); for (const key of Object.keys(FILE_NAMES)) { diff --git a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js index ab23aecea..d538fff2c 100644 --- a/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js +++ b/plugins/power-pages/scripts/tests/refresh-alm-plan-data.test.js @@ -279,6 +279,74 @@ test('refresh finalize sets PLAN_STATUS to Completed', (t) => { assert.equal(planData.PLAN_STATUS, 'Completed'); }); +// evaluatePlanCompletion — the In Execution → Completed transition that makes the +// LAST execution skill terminate the plan automatically. These tests pin the +// status-gate edges (the defensive "Approved" fallback + the Draft / failed-step +// guards) so the lifecycle can't silently over- or under-complete. + +test('completion: the last execution phase flips a plan from Approved straight to Completed (defensive fallback)', (t) => { + // Normal flow promotes Approved → In Execution in check-alm-plan.js's Phase 0 + // gate. If that promotion was ever skipped (read-only caller, --no-heartbeat, + // a manual run) the plan can still be "Approved" when its final step lands. + // evaluatePlanCompletion accepts BOTH In Execution AND Approved so the plan + // still terminates instead of getting wedged one transition short of done. + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + PLAN_STATUS: 'Approved', + SITE_NAME: 'TestSite', + validationRuns: { Staging: null }, + steps: [ + { name: 'Deploy via pipeline to Staging', status: 'completed' }, + { name: 'Test site in Staging', status: 'pending' }, + ], + }); + writeJson(path.join(root, 'docs', 'alm', 'last-test-site.json'), { runOutcome: 'passed', runAt: '2026-06-16T00:00:00.000Z' }); + + refresh({ projectRoot: root, phase: 'test-site', render: false, stageName: 'Staging' }); + + const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); + assert.equal(planData.steps[1].status, 'completed', 'the final step flips to completed'); + assert.equal(planData.PLAN_STATUS, 'Completed', 'Approved → Completed (defensive fallback path)'); + assert.ok(planData.COMPLETED_AT, 'COMPLETED_AT is stamped on completion'); +}); + +test('completion: a Draft plan is never auto-completed even when every step is done', (t) => { + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + PLAN_STATUS: 'Draft', + SITE_NAME: 'TestSite', + validationRuns: { Staging: null }, + steps: [{ name: 'Test site in Staging', status: 'completed' }], + }); + writeJson(path.join(root, 'docs', 'alm', 'last-test-site.json'), { runOutcome: 'passed', runAt: '2026-06-16T00:00:00.000Z' }); + + refresh({ projectRoot: root, phase: 'test-site', render: false, stageName: 'Staging' }); + + const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); + assert.equal(planData.PLAN_STATUS, 'Draft', 'an unapproved plan must not complete itself'); + assert.equal(planData.COMPLETED_AT, undefined, 'no COMPLETED_AT stamp for a Draft plan'); +}); + +test('completion: a failed step blocks completion so a failed deploy can never look "done"', (t) => { + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + PLAN_STATUS: 'In Execution', + SITE_NAME: 'TestSite', + validationRuns: { Staging: null }, + steps: [ + { name: 'Deploy via pipeline to Staging', status: 'failed' }, + { name: 'Test site in Staging', status: 'pending' }, + ], + }); + writeJson(path.join(root, 'docs', 'alm', 'last-test-site.json'), { runOutcome: 'passed', runAt: '2026-06-16T00:00:00.000Z' }); + + refresh({ projectRoot: root, phase: 'test-site', render: false, stageName: 'Staging' }); + + const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); + assert.equal(planData.PLAN_STATUS, 'In Execution', 'a failed step keeps the plan in execution'); + assert.equal(planData.COMPLETED_AT, undefined, 'no COMPLETED_AT while a step is failed'); +}); + test('refresh setup-pipeline preserves prior pipelineMeta.reusedByWiring annotation', (t) => { const root = makeProject(t); writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { diff --git a/plugins/power-pages/scripts/tests/run-skill-posttool-validation.test.js b/plugins/power-pages/scripts/tests/run-skill-posttool-validation.test.js new file mode 100644 index 000000000..649f29a6f --- /dev/null +++ b/plugins/power-pages/scripts/tests/run-skill-posttool-validation.test.js @@ -0,0 +1,139 @@ +'use strict'; + +// Smoke tests for the centralized PostToolUse hook (hooks/run-skill-posttool-validation.js). +// +// Two contracts are covered here that no other test exercised: +// 1. The ALM-plan reconcile BACKSTOP actually spawns after an ALM skill when a +// docs/.alm-plan-data.json exists — and heals a skipped refresh (auto-heal). +// 2. EXIT-CODE NEUTRALITY: the reconcile is best-effort and must never change the +// hook's exit code — the validator's status stands. We prove this by running the +// SAME blocking-validator scenario with the reconcile branch reachable (plan +// present) and unreachable (no plan) and asserting the exit code is identical. +// +// The hook is exercised as a real child process (the way Claude Code invokes it), +// piping a tool_input JSON over stdin. + +const test = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('fs'); +const path = require('path'); +const os = require('os'); +const { spawnSync } = require('child_process'); + +const HOOK_PATH = path.join(__dirname, '..', '..', 'hooks', 'run-skill-posttool-validation.js'); + +function makeProject(t) { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'hook-posttool-')); + fs.mkdirSync(path.join(root, 'docs', 'alm'), { recursive: true }); + t.after(() => fs.rmSync(root, { recursive: true, force: true })); + return root; +} + +function writeJson(filePath, data) { + fs.mkdirSync(path.dirname(filePath), { recursive: true }); + fs.writeFileSync(filePath, JSON.stringify(data, null, 2), 'utf8'); +} + +function readJson(filePath) { + return JSON.parse(fs.readFileSync(filePath, 'utf8')); +} + +// Backdate the plan file so a just-written marker is unambiguously "newer". +function backdatePlan(root, secondsAgo = 60) { + const p = path.join(root, 'docs', '.alm-plan-data.json'); + const ts = (Date.now() - secondsAgo * 1000) / 1000; + fs.utimesSync(p, ts, ts); +} + +function runHook(root, skill) { + return spawnSync(process.execPath, [HOOK_PATH], { + input: JSON.stringify({ tool_input: { skill }, cwd: root }), + encoding: 'utf8', + cwd: root, + }); +} + +test('hook spawns the reconcile backstop and heals a skipped refresh after an ALM skill', (t) => { + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'T', + pipelineMeta: { lastDeploy: null }, + steps: [{ name: 'Deploy via pipeline to Staging', status: 'pending' }], + stages: [{ label: 'Staging', envUrl: 'https://stg.crm.dynamics.com/', type: 'target' }], + }); + // Marker written AFTER the plan (skill ran but its in-skill refresh step was skipped). + writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { + pipelineId: 'p1', + stageRunId: 'sr1', + solutionName: 'MySolution', + stageName: 'Staging', + status: 'Succeeded', + deployedAt: '2026-06-16T00:00:00.000Z', + componentCount: 118, + }); + backdatePlan(root); + + const res = runHook(root, 'deploy-pipeline'); + + // Validator approves (all required fields present, not Failed) → exit 0. + assert.equal(res.status, 0, `hook should exit 0; stderr=${res.stderr}`); + // The backstop fired and announced the auto-heal. + assert.match(res.stdout, /refreshed automatically/i, + `expected the reconcile notice in stdout; got: ${res.stdout}`); + // And it actually ingested the marker into the plan. + const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); + assert.equal(planData.pipelineMeta.lastDeploy.status, 'Succeeded'); + assert.equal(planData.pipelineMeta.lastDeploy.componentCount, 118); +}); + +test('hook does NOT reconcile for a non-ALM skill even when a plan + newer marker exist', (t) => { + const root = makeProject(t); + writeJson(path.join(root, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'T', + pipelineMeta: { lastDeploy: null }, + steps: [{ name: 'Deploy via pipeline to Staging', status: 'pending' }], + }); + writeJson(path.join(root, 'docs', 'alm', 'last-deploy.json'), { + pipelineId: 'p1', stageRunId: 'sr1', solutionName: 'S', status: 'Succeeded', + deployedAt: '2026-06-16T00:00:00.000Z', componentCount: 99, + }); + backdatePlan(root); + + // create-site is a tracked skill but NOT an ALM plan skill → isAlmPlanSkill === false. + const res = runHook(root, 'create-site'); + + assert.doesNotMatch(res.stdout, /refreshed automatically/i, + 'non-ALM skill must not trigger the reconcile backstop'); + // The plan must be untouched — the deploy marker was NOT ingested. + const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); + assert.equal(planData.pipelineMeta.lastDeploy, null, + 'non-ALM skill path must leave the plan unchanged'); +}); + +test('reconcile backstop is exit-code-neutral: a blocking validator status is unchanged whether or not the plan is present', (t) => { + // A Failed deploy marker makes validate-deploy-pipeline.js BLOCK (exit 2). The + // reconcile runs too (plan present) — and must NOT mask or alter that exit code. + const failedMarker = { + pipelineId: 'p1', stageRunId: 'sr1', solutionName: 'S', stageName: 'Staging', + status: 'Failed', deployedAt: '2026-06-16T00:00:00.000Z', + }; + + // (A) Reconcile branch REACHABLE — docs/.alm-plan-data.json exists. + const withPlan = makeProject(t); + writeJson(path.join(withPlan, 'docs', 'alm', 'last-deploy.json'), failedMarker); + writeJson(path.join(withPlan, 'docs', '.alm-plan-data.json'), { + SITE_NAME: 'T', steps: [{ name: 'Deploy via pipeline to Staging', status: 'pending' }], + }); + backdatePlan(withPlan); + const resWith = runHook(withPlan, 'deploy-pipeline'); + + // (B) Reconcile branch UNREACHABLE — no plan file at all. + const withoutPlan = makeProject(t); + writeJson(path.join(withoutPlan, 'docs', 'alm', 'last-deploy.json'), failedMarker); + const resWithout = runHook(withoutPlan, 'deploy-pipeline'); + + assert.equal(resWith.status, 2, 'blocking validator must surface exit 2 with the plan present'); + assert.equal(resWithout.status, 2, 'blocking validator must surface exit 2 with no plan'); + assert.equal(resWith.status, resWithout.status, + 'the reconcile backstop must not change the validator-determined exit code'); +}); From f27128fede032595851786a9f4b0655e25b4677c Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Mon, 22 Jun 2026 11:24:54 +0530 Subject: [PATCH 26/38] Address #194 Copilot review comments (3) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1. hook: forward reconcile failure detail to stderr. The empty JSON.parse catch swallowed spawn errors / timeouts / non-zero exits, and rec.stderr was never forwarded — so the "See stderr for details" pointer was empty. Now track spawnFailed (rec.error / non-zero status / signal) and a parsed flag, report on STDERR only on actual failure (clean runs stay quiet — the hook fires on every Skill use), and forward the child's stderr verbatim where refresh-alm-plan-data.js already writes its per-phase error detail. Stays non-blocking (validator's exit code unchanged). + test: malformed plan → broken reconcile is surfaced AND exit code is unchanged; + quiet-on-success assertion on the heal test. 2. render-alm-plan.js: COMPLETED_AT is optional, not required. Move it out of the required-keys list into its own "optional lifecycle key" note — it's present only once the plan reaches "Completed"; the renderer omits the footer line otherwise. 3. powerpages-hook-utils.js: isAlmPlanSkill JSDoc said @param {string} but the function (and its tests) accept non-strings (null/undefined → false). Widen to {*} and note the contract. Full suite: 1247 pass. legacy-compat + alm-lint: clean. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../hooks/run-skill-posttool-validation.js | 31 +++++++++++++++---- .../scripts/lib/powerpages-hook-utils.js | 4 ++- .../run-skill-posttool-validation.test.js | 22 +++++++++++++ .../plan-alm/scripts/render-alm-plan.js | 7 ++++- 4 files changed, 56 insertions(+), 8 deletions(-) diff --git a/plugins/power-pages/hooks/run-skill-posttool-validation.js b/plugins/power-pages/hooks/run-skill-posttool-validation.js index df36ea2d7..d77959395 100644 --- a/plugins/power-pages/hooks/run-skill-posttool-validation.js +++ b/plugins/power-pages/hooks/run-skill-posttool-validation.js @@ -71,26 +71,45 @@ process.stdin.on('end', () => { cwd, timeout: 20000, }); + // spawnSync surfaces a spawn/timeout failure on rec.error (e.g. ETIMEDOUT) + // and a non-zero / signalled exit on rec.status / rec.signal — none of which + // produce parseable stdout. Track those so a broken reconcile is reported + // rather than silently swallowed by the JSON.parse catch below. + const spawnFailed = !!rec.error || rec.status !== 0 || !!rec.signal; let reconciled = []; let failed = []; + let parsed = false; try { const out = JSON.parse((rec.stdout || '').trim()); reconciled = out.reconciled || []; failed = out.failed || []; - } catch {} + parsed = true; + } catch { /* parsed stays false — surfaced in the spawnFailed/!parsed branch */ } if (reconciled.length > 0) { process.stdout.write( `[power-pages] ALM plan was out of sync with ${reconciled.length} run marker(s) — refreshed automatically (${reconciled.join(', ')}).\n`, ); } + // Failure reporting goes to STDERR (only on an actual failure, never on the + // happy path — the hook fires on every Skill use, so clean runs must stay + // quiet). A swallowed reconcile failure is exactly what makes a stale plan + // impossible to diagnose, so we forward the child's stderr verbatim — that's + // where refresh-alm-plan-data.js already writes its per-phase error detail, + // which is what makes the summary line below actionable. if (failed.length > 0) { - // Non-blocking, but surfaced — a swallowed reconcile failure is exactly - // what makes a stale plan impossible to diagnose. - process.stdout.write( - `[power-pages] ALM plan reconcile could not heal ${failed.length} phase(s): ${failed.map((f) => f.phase).join(', ')}. See stderr for details.\n`, + process.stderr.write( + `[power-pages] ALM plan reconcile could not heal ${failed.length} phase(s): ${failed.map((f) => f.phase).join(', ')}. Details below.\n`, ); + if (rec.stderr) process.stderr.write(rec.stderr); + } else if (spawnFailed || !parsed) { + // The reconcile didn't even produce a parseable result (spawn error, + // timeout, non-zero exit, or garbled stdout). Non-blocking, but the user + // should still see why the auto-heal didn't run. + const why = rec.error ? rec.error.message : rec.signal ? `signal ${rec.signal}` : `exit ${rec.status}`; + process.stderr.write(`[power-pages] ALM plan reconcile did not complete (${why}). Details below.\n`); + if (rec.stderr) process.stderr.write(rec.stderr); } - debug(`[power-pages hook] reconcile reconciled=${JSON.stringify(reconciled)} failed=${JSON.stringify(failed)}\n`); + debug(`[power-pages hook] reconcile reconciled=${JSON.stringify(reconciled)} failed=${JSON.stringify(failed)} spawnFailed=${spawnFailed} parsed=${parsed}\n`); } catch (e) { // Best-effort — a reconcile failure must never break the skill or the hook. debug(`[power-pages hook] reconcile error (ignored): ${e.message}\n`); diff --git a/plugins/power-pages/scripts/lib/powerpages-hook-utils.js b/plugins/power-pages/scripts/lib/powerpages-hook-utils.js index 5d5f98211..af405f958 100644 --- a/plugins/power-pages/scripts/lib/powerpages-hook-utils.js +++ b/plugins/power-pages/scripts/lib/powerpages-hook-utils.js @@ -126,7 +126,9 @@ const ALM_PLAN_SKILLS = new Set([ * True when `value` (a raw skill name, `/skill`, or `power-pages:skill`) resolves * to an ALM plan skill. Normalizes via `detectTrackedSkill`, so it also confirms * the skill actually exists in this plugin. - * @param {string} value + * Accepts any value — non-strings (including null/undefined) resolve to false + * via detectTrackedSkill, so callers may pass an unvalidated skill name. + * @param {*} value * @returns {boolean} */ function isAlmPlanSkill(value) { diff --git a/plugins/power-pages/scripts/tests/run-skill-posttool-validation.test.js b/plugins/power-pages/scripts/tests/run-skill-posttool-validation.test.js index 649f29a6f..f15bb402c 100644 --- a/plugins/power-pages/scripts/tests/run-skill-posttool-validation.test.js +++ b/plugins/power-pages/scripts/tests/run-skill-posttool-validation.test.js @@ -84,6 +84,28 @@ test('hook spawns the reconcile backstop and heals a skipped refresh after an AL const planData = readJson(path.join(root, 'docs', '.alm-plan-data.json')); assert.equal(planData.pipelineMeta.lastDeploy.status, 'Succeeded'); assert.equal(planData.pipelineMeta.lastDeploy.componentCount, 118); + // A clean reconcile must stay quiet on stderr — the hook fires on every Skill + // use, so success must not produce failure noise. + assert.doesNotMatch(res.stderr, /reconcile/i, 'a successful reconcile must not write failure noise to stderr'); +}); + +test('hook forwards reconcile failure detail to stderr without changing the exit code', (t) => { + // A malformed plan file makes refresh-alm-plan-data.js --reconcile throw and exit + // non-zero with its reason on stderr. The hook must (a) surface that — the prior + // empty JSON.parse catch swallowed spawn errors / timeouts / non-zero exits — and + // (b) stay non-blocking (the reconcile is best-effort; the validator's status stands). + const root = makeProject(t); + fs.writeFileSync(path.join(root, 'docs', '.alm-plan-data.json'), 'not json {{{', 'utf8'); + // A newer marker guarantees the reconcile reaches the plan-parse (and would heal if it could). + writeJson(path.join(root, 'docs', 'alm', 'last-export.json'), { solutionUniqueName: 'S', exportedAt: '2026-06-16T00:00:00.000Z' }); + backdatePlan(root); + + // export-solution is an ALM skill; its validator gracefully approves (no zip) → exit 0. + const res = runHook(root, 'export-solution'); + + assert.equal(res.status, 0, 'a broken reconcile must not change the validator-determined exit code'); + assert.match(res.stderr, /reconcile did not complete/i, 'the hook must report the broken reconcile'); + assert.match(res.stderr, /Could not parse/i, 'the child reconcile stderr must be forwarded verbatim'); }); test('hook does NOT reconcile for a non-ALM skill even when a plan + newer marker exist', (t) => { diff --git a/plugins/power-pages/skills/plan-alm/scripts/render-alm-plan.js b/plugins/power-pages/skills/plan-alm/scripts/render-alm-plan.js index 5f66bc00f..b19487f77 100644 --- a/plugins/power-pages/skills/plan-alm/scripts/render-alm-plan.js +++ b/plugins/power-pages/skills/plan-alm/scripts/render-alm-plan.js @@ -6,9 +6,14 @@ * node render-alm-plan.js --output --data * * Required top-level keys in the JSON data file: - * SITE_NAME, GENERATED_AT, STRATEGY, PLAN_STATUS, APPROVED_BY, APPROVAL_DATE, COMPLETED_AT, + * SITE_NAME, GENERATED_AT, STRATEGY, PLAN_STATUS, APPROVED_BY, APPROVAL_DATE, * stages, steps, risks * + * Optional lifecycle key: + * COMPLETED_AT — present only once the plan reaches PLAN_STATUS "Completed"; + * the renderer emits the footer "Completed" line when it exists and omits it + * otherwise (an in-flight plan has no COMPLETED_AT). + * * Optional v2 keys (added for split-solutions support): * sizeAnalysis, assetAdvisory, proposedSolutions, appliedStrategies, * recommendations, envVars, breakdown, estimationMethod, estimationAccuracyPct From af087e767cac84d8938e4b86c2cd9186b3ef3848 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Mon, 22 Jun 2026 11:40:41 +0530 Subject: [PATCH 27/38] plan-alm: move PLAN_STATUS badge next to the "Generated" line The Draft/Approved/In Execution/Completed status badge sat top-right of the topbar (justify-content:space-between). Move it inline next to the "Generated " sub-line via a new .topbar-sub-row flexbox, so the status reads alongside the plan metadata rather than floating in the corner. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../skills/plan-alm/assets/alm-plan-template.html | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/plugins/power-pages/skills/plan-alm/assets/alm-plan-template.html b/plugins/power-pages/skills/plan-alm/assets/alm-plan-template.html index be5b53a80..b59753a82 100644 --- a/plugins/power-pages/skills/plan-alm/assets/alm-plan-template.html +++ b/plugins/power-pages/skills/plan-alm/assets/alm-plan-template.html @@ -30,6 +30,7 @@ .topbar-left{display:flex;align-items:center;gap:14px;} .logo{width:36px;height:36px;border-radius:var(--radius);background:linear-gradient(135deg,#0078d4,#5c2d91);display:flex;align-items:center;justify-content:center;font-size:13px;font-weight:800;color:#fff;font-family:var(--mono);} .topbar-title{font-size:16px;font-weight:700;color:var(--text-bright);} +.topbar-sub-row{display:flex;align-items:center;gap:10px;margin-top:1px;} .topbar-sub{font-size:11px;color:var(--text-dim);margin-top:1px;} .plan-status{font-size:10px;font-weight:700;padding:3px 10px;border-radius:10px;text-transform:uppercase;letter-spacing:0.6px;background:var(--accent-bg);color:var(--accent);border:1px solid var(--accent-border);} .plan-status.draft{background:var(--high-bg);color:var(--high);border-color:var(--high-border);} @@ -319,10 +320,12 @@
ALM Plan — __SITE_NAME__
-
Generated __GENERATED_AT__
+
+
Generated __GENERATED_AT__
+ __PLAN_STATUS__ +
- __PLAN_STATUS__
From 356f63fb5db706f033573b6c7450a28fcb1589e1 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Mon, 22 Jun 2026 14:08:51 +0530 Subject: [PATCH 28/38] plan-alm: deterministic Draft/Approved status write + consistency guard + in-place approve MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Draft/Approved tag was the only PLAN_STATUS transition with no helper behind it — plan-alm Phase 4 set it via hand-authored Edits to the HTML spans AND the JSON. Since the badge + approver are re-derived from docs/.alm-plan-data.json on every render, the manual HTML Edit was non-durable, and a partial write left the plan "approver recorded but PLAN_STATUS=Draft" — stuck forever (check-alm-plan only promotes from Approved; evaluatePlanCompletion ignores Draft). - NEW scripts/lib/set-plan-status.js: single deterministic owner of the creation-time Draft/Approved write. Writes PLAN_STATUS + PLAN_MODE + APPROVED_BY + APPROVAL_DATE together (atomic temp+rename) and optionally re-renders (reuses refresh-alm-plan-data.js findRendererPath/invokeRenderer, now exported). Invariants: only Draft/Approved settable here; Approved requires a non-empty approver; Draft clears the approver; a live (In Execution/Completed) plan is not re-drafted without --force. + 10 unit tests. - validate-plan-alm.js: consistency guard blocks the two half-written states (Draft+approver, Approved+no-approver) for plans created the old way or hand-edited. + 5 tests. - plan-alm Phase 4: both save options now call set-plan-status.js instead of hand-editing the HTML spans + JSON. - plan-alm Phase 1 step 0b: in-place Draft -> Approved fast-path (new gate plan-alm:1.approve-draft) — approve an existing draft without a full re-plan. - approval-gates.md: catalog the new gate. AGENTS.md: document the helper + lifecycle ownership. Version -> 2.6.0 (2.5.0 held by #194). Full suite: 1262 pass. alm-lint: 0 findings. legacy-compat: in sync. End-to-end: stuck state blocks (2) -> set-plan-status -> badge Approved -> validator approves (0). Co-Authored-By: Claude Opus 4.8 (1M context) --- marketplace.json | 2 +- plugins/power-pages/.plugin/plugin.json | 2 +- plugins/power-pages/AGENTS.md | 1 + .../power-pages/references/approval-gates.md | 1 + .../scripts/lib/refresh-alm-plan-data.js | 5 + .../scripts/lib/set-plan-status.js | 195 ++++++++++++++++++ .../scripts/tests/set-plan-status.test.js | 138 +++++++++++++ .../scripts/tests/validate-plan-alm.test.js | 80 +++++++ plugins/power-pages/skills/plan-alm/SKILL.md | 49 ++++- .../plan-alm/scripts/validate-plan-alm.js | 43 ++++ 10 files changed, 505 insertions(+), 11 deletions(-) create mode 100644 plugins/power-pages/scripts/lib/set-plan-status.js create mode 100644 plugins/power-pages/scripts/tests/set-plan-status.test.js diff --git a/marketplace.json b/marketplace.json index 207f7a55e..1028cf6e9 100644 --- a/marketplace.json +++ b/marketplace.json @@ -13,7 +13,7 @@ "source": "./plugins/power-pages", "description": "Power Pages development and management plugin for Claude Code and GitHub Copilot", "category": "development", - "version": "2.5.0", + "version": "2.6.0", "license": "MIT", "tags": [ "power platform", diff --git a/plugins/power-pages/.plugin/plugin.json b/plugins/power-pages/.plugin/plugin.json index f3fa3f687..5fa58e6c5 100644 --- a/plugins/power-pages/.plugin/plugin.json +++ b/plugins/power-pages/.plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "power-pages", - "version": "2.5.0", + "version": "2.6.0", "description": "Create and deploy Power Pages sites using modern development approaches. Supports code sites (SPAs) with React, Angular, Vue, or Astro. Includes ALM orchestration (plan-alm) with a solution-splitting decision tree, per-solution pipelines, Azure Blob asset advisory, manifest schema v2 for multi-solution deployments, and force-link remediation for cross-host pipeline migrations.", "author": { "name": "Microsoft", diff --git a/plugins/power-pages/AGENTS.md b/plugins/power-pages/AGENTS.md index 198394c06..5b3a32f3d 100644 --- a/plugins/power-pages/AGENTS.md +++ b/plugins/power-pages/AGENTS.md @@ -201,6 +201,7 @@ Shared lib modules live at `scripts/lib/` and are imported by other scripts via - `scripts/lib/detect-project-context.js`: Reads Power Pages project context from the project root. The `siteType` discriminator is the **build axis** — code/SPA vs declarative (design-studio) site — NOT the Dataverse data-model axis (a declarative site can be on the standard OR enhanced data model; both download to a `.powerpages-site/` tree). `siteType: "data-model"` is the (compat-named) declarative bucket; a future pass may rename it `"declarative"`. Resolves identity in order: (1) `powerpages.config.json` → `siteType: "code"` (code/SPA sites); (2) `.powerpages-site/` → `siteType: "data-model"` (declarative design-studio sites — standard or enhanced data model — which have **no** `powerpages.config.json`). The **authoritative declarative marker is the `.powerpages-site/.portalconfig/` directory** (only declarative sites have it); `website.yml` is the identity source (`id`→`websiteRecordId`, `name`→`siteName`) but is NOT a reliable declarative signal alone because **both** site types carry it. `environmentUrl: null` for declarative sites (no env URL in the local files — callers re-confirm via `pac env who`). Also reads `.solution-manifest.json` and `.datamodel-manifest.json`. Args: `--projectRoot` (opt). Output: `{ projectRoot, siteType, siteName, websiteRecordId, environmentUrl, solutionManifest, datamodelManifest }`. Exit 0 on success, exit 1 only if neither `powerpages.config.json` nor a `.powerpages-site/` (`.portalconfig/`/`website.yml`) marker is found. Note: `findProjectRoot` (in `validation-helpers.js`) likewise treats a `.powerpages-site/` directory as a project-root marker. - `scripts/lib/alm-paths.js`: Single source of truth for ALM artifact paths. Exports `ALM_DIR` (always `docs/alm`), `FILE_NAMES` (frozen object mapping logical key → filename for all 14 ALM artifacts), `almDir(projectRoot) → path`, `almPath(projectRoot, key) → path`, `ensureAlmDir(projectRoot) → path` (mkdir -p idempotent). Every ALM-only state file (5 plan/decision JSONs + 9 `last-*.json` skill-run markers including `last-export.json`) writes under `/docs/alm/`. **Always resolve through this helper** — never inline a raw `docs/alm/...` path in a script. Files intentionally NOT moved here (and not in `FILE_NAMES`): `.solution-manifest.json`, `.datamodel-manifest.json`, `.alm-config.json`, `.alm-deferred`, `deployment-settings.json`. Adding a new ALM marker means adding its key + filename to `FILE_NAMES` first; `almPath` throws on unknown keys to catch typos at call-site. - `scripts/lib/check-alm-plan.js`: Phase 0 gate helper used by every ALM skill to detect (a) whether an ALM plan exists for this project, (b) whether the user has explicitly deferred ALM via the `.alm-deferred` marker, and (c) whether an existing plan is stale (the source solution was modified after the plan was generated). Args: `--projectRoot`, `--envUrl` (opt — required for staleness check), `--token` (opt), `--solutionId` (opt — required for staleness check). Output: `{ exists, deferred, deferral, planPath, htmlPath, stale, staleness: { reason, detail }, generatedAt, planStatus, solution: {...} }`. Without env/solution context the helper does an existence-only check; with them it queries Dataverse for `solutions(solutionId)?$select=modifiedon` and compares against `planData.generatedAt`. Used by `setup-solution`, `setup-pipeline`, `deploy-pipeline`, `export-solution`, `import-solution`, `configure-env-variables`, `ensure-pipelines-host`, `force-link-environment` Phase 0 gates — the "fail closed when no plan" pattern. **PLAN_STATUS lifecycle — promotes `Approved` → `In Execution`:** plan-alm is plan-only and leaves the plan `Approved`; this helper performs the `Approved` → `In Execution` transition (and writes the first heartbeat) the **first time an execution skill's Phase 0 runs** — it is the only thing that sets `In Execution`, so without it the heartbeat/active-chain machinery (multi-hour-deploy `stale-heartbeat` reclassification) never engages. Gated on heartbeat-write: read-only callers pass `--no-heartbeat` (plan-alm's own deferral check, audits, tests) and are never promoted. The terminal `In Execution` → `Completed` transition is owned by `refresh-alm-plan-data.js` (completion evaluator). +- `scripts/lib/set-plan-status.js`: **The single deterministic owner of the creation-time `Draft` / `Approved` write** — the one PLAN_STATUS transition that used to be done by hand-authored `Edit`s in plan-alm Phase 4 (to the HTML spans *and* the JSON), with no helper. Because the badge + `approved-by` / `approval-date` spans are re-derived from `docs/.alm-plan-data.json` on every render, the old manual HTML Edit was non-durable (reverted on the next refresh) and a partial write left the plan "approver recorded but PLAN_STATUS=Draft" — stuck forever, since `check-alm-plan.js` only promotes from `Approved`. This helper writes `PLAN_STATUS` + `PLAN_MODE` + `APPROVED_BY` + `APPROVAL_DATE` **together** (atomic temp+rename) and optionally re-renders (reuses `refresh-alm-plan-data.js → findRendererPath`/`invokeRenderer`). Enforced invariants: only `Draft` / `Approved` are settable here (`In Execution` is owned by `check-alm-plan.js`, `Completed` by `refresh-alm-plan-data.js`); `Approved` **requires** a non-empty `--approver`; `Draft` **clears** the approver fields; a plan already `In Execution` / `Completed` is **not** re-drafted without `--force`. Args: `--projectRoot`, `--status Draft|Approved`, `--approver`, `--approvalDate` (opt — defaults to now), `--force`, `--render`, `--rendererPath` (opt). Output: `{ ok, previousStatus, status, mode, approver, approvalDate, rendered }`. Called by plan-alm Phase 4 (both save options) and the Phase 1 step-0b in-place Draft→Approved fast-path. The `validate-plan-alm.js` consistency guard blocks the two half-written states (`Draft`+approver, `Approved`+no-approver) for plans created the old way or hand-edited. - `scripts/lib/resolve-target-solution.js`: Resolves "which solution should this new Dataverse record land in?" Implements the strict 3-step order from the ALM-aware-by-default principle: (1) explicit `--solutionUniqueName` (or equivalent caller arg) wins; (2) `.solution-manifest.json` in the project root; (3) neither → throw `NoSolutionConfiguredError`. **The module NEVER auto-picks from Dataverse** — interactive prompt UX is the caller's responsibility (catch the error, present an `AskUserQuestion` list, re-invoke with `explicit` populated). Callers that need to confirm the solution still exists in Dataverse can pass `verifyExists: true`; the module then enriches the result with `{ solutionId, version, ismanaged }`. Component-creation scripts must require this helper and pass through `--solutionUniqueName` so records land in the user's solution instead of `Default`. #### Solution Splitting Decision Tree (v1.3.0+) diff --git a/plugins/power-pages/references/approval-gates.md b/plugins/power-pages/references/approval-gates.md index 5ada7883c..0a080a6f5 100644 --- a/plugins/power-pages/references/approval-gates.md +++ b/plugins/power-pages/references/approval-gates.md @@ -263,6 +263,7 @@ Each section lists every `AskUserQuestion` in that skill. Catalog rows are marke | ID | Kind | Category | Phase | Trigger / question | Cancel leaves | |---|---|---|---|---|---| | `plan-alm:1.deferral` | gate | progress | 1 | `.alm-deferred` marker present — *"Continue with deferral / remove and proceed / cancel"* | `deferral-marker` | +| `plan-alm:1.approve-draft` | gate | plan | 1 (0b) | Existing **Draft** plan found — *"Approve this draft now (no re-plan) / re-plan from scratch / cancel"*. Approve writes status via `set-plan-status.js` and exits | nothing | | `plan-alm:1.completeness` | gate | progress | 1 | Completeness check found gaps — *"Sync first / plan with gaps / cancel"* | nothing | | `plan-alm:2.q1-existing` | gate | plan | 2 (Q1) | `SOLUTION_DONE=true` — *"Use existing solution **{name}**?"* | nothing | | `plan-alm:2.q1-fresh` | gate | plan | 2 (Q1) | `SOLUTION_DONE=false` — *"Include solution setup in plan?"* | nothing | diff --git a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js index 27d912fe8..58d1b07f7 100644 --- a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js +++ b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js @@ -1212,4 +1212,9 @@ module.exports = { STEP_TO_SKILL, MARKER_TO_PHASE, PHASES, + // Exported so other plan-data writers (e.g. set-plan-status.js) reuse the SAME + // renderer-invocation instead of re-implementing the execFileSync call — keeps + // the "where is render-alm-plan.js / how is it invoked" knowledge in one place. + findRendererPath, + invokeRenderer, }; diff --git a/plugins/power-pages/scripts/lib/set-plan-status.js b/plugins/power-pages/scripts/lib/set-plan-status.js new file mode 100644 index 000000000..d3f5ead9f --- /dev/null +++ b/plugins/power-pages/scripts/lib/set-plan-status.js @@ -0,0 +1,195 @@ +#!/usr/bin/env node +'use strict'; + +// set-plan-status.js — the single deterministic owner of the CREATION-TIME ALM +// plan status write (`Draft` / `Approved`). +// +// Background / why this exists: +// The plan-status badge and the "Approved by" stamp in docs/alm-plan.html are +// BOTH re-derived from docs/.alm-plan-data.json every time the plan is rendered +// (render-alm-plan.js reads PLAN_STATUS / APPROVED_BY / APPROVAL_DATE). Every +// OTHER status transition is owned by a deterministic helper: +// - Approved -> In Execution : check-alm-plan.js (first execution skill) +// - In Execution -> Completed: refresh-alm-plan-data.js (evaluatePlanCompletion) +// ...but the Draft/Approved write was historically done by HAND-AUTHORED Edits +// in plan-alm Phase 4 — to two places (the HTML spans AND the JSON), with no +// helper. That produced two real bugs: +// 1. Editing the HTML span is non-durable — the next refresh re-derives the +// badge from plan-data and reverts it if plan-data wasn't also updated. +// 2. A partial write (APPROVED_BY set in plan-data but PLAN_STATUS left at +// "Draft") leaves the plan shown-as-approved but stuck on Draft forever, +// because check-alm-plan.js only promotes from "Approved". +// This helper makes plan-data the single source of truth and writes all four +// fields together (atomically), so neither bug can recur. Phase 4 (and the +// in-place Draft->Approved fast-path) call this instead of hand-editing. +// +// Usage: +// node set-plan-status.js --projectRoot --status Approved --approver "Jane Doe" [--render] +// node set-plan-status.js --projectRoot --status Draft [--render] +// node set-plan-status.js --projectRoot --status Draft --force (re-draft a running plan) +// +// Output (JSON to stdout): +// { "ok": true, "projectRoot": "...", "previousStatus": "Draft", "status": "Approved", +// "mode": "approved", "approver": "Jane Doe", "approvalDate": "2026-…Z", "rendered": true } +// +// Exit 0 on success, exit 1 on any validation error (missing plan, bad status, +// Approved-without-approver, or a refused regression of a live plan). + +const fs = require('fs'); +const { planDataPath, planHtmlPath } = require('./alm-paths'); +// Reuse the SAME renderer-invocation as the post-run refresh, rather than +// re-implementing the execFileSync call. Requiring this module is side-effect +// free (its CLI body is guarded by `require.main === module`). +const { findRendererPath, invokeRenderer } = require('./refresh-alm-plan-data'); + +// The two statuses this helper owns. In Execution / Completed are owned by +// check-alm-plan.js and refresh-alm-plan-data.js respectively and must NOT be +// settable here — that would let a caller fabricate lifecycle state. +const CREATION_STATUSES = new Set(['Draft', 'Approved']); +// A plan in one of these states is past the creation/approval stage; re-writing +// it back to Draft/Approved would erase live execution state, so it is refused +// unless --force is passed. +const LIVE_STATUSES = new Set(['In Execution', 'Completed']); + +/** + * Atomically set the creation-time plan status in docs/.alm-plan-data.json. + * + * @param {object} opts + * @param {string} opts.projectRoot + * @param {'Draft'|'Approved'} opts.status + * @param {string} [opts.approver] required (non-empty) when status === 'Approved' + * @param {string} [opts.approvalDate] ISO string; defaults to now when status === 'Approved' + * @param {boolean} [opts.force] allow overwriting an In Execution / Completed plan + * @param {boolean} [opts.render] re-render docs/alm-plan.html after writing + * @param {string} [opts.rendererPath] override the renderer path (tests) + * @param {() => string} [opts.makeNow] injectable clock (tests); returns an ISO string + * @returns {{ ok: true, projectRoot, previousStatus, status, mode, approver, approvalDate, rendered }} + */ +function setPlanStatus(opts) { + const { + projectRoot, + status, + approver, + approvalDate, + force = false, + render = false, + rendererPath = null, + makeNow = () => new Date().toISOString(), + } = opts || {}; + + if (!projectRoot) throw new Error('--projectRoot is required'); + if (!CREATION_STATUSES.has(status)) { + throw new Error( + `--status must be one of: ${[...CREATION_STATUSES].join(', ')} ` + + `(got ${JSON.stringify(status)}). "In Execution"/"Completed" are owned by ` + + 'check-alm-plan.js / refresh-alm-plan-data.js, not this helper.', + ); + } + + const dataPath = planDataPath(projectRoot); + if (!fs.existsSync(dataPath)) { + throw new Error(`No ALM plan found at ${dataPath}. Run /power-pages:plan-alm first.`); + } + + let planData; + try { + planData = JSON.parse(fs.readFileSync(dataPath, 'utf8')); + } catch (e) { + throw new Error(`Could not parse ${dataPath}: ${e.message}`); + } + + const previousStatus = planData.PLAN_STATUS || null; + + // Never silently erase live execution state. A plan that has started executing + // (In Execution) or finished (Completed) should not be quietly reset to a + // creation-time status — that would drop heartbeat/step state and confuse the + // downstream gates. Require an explicit --force to override. + if (LIVE_STATUSES.has(previousStatus) && !force) { + throw new Error( + `Refusing to set status to "${status}": the plan is already "${previousStatus}". ` + + 'Pass --force to override (this discards live execution state).', + ); + } + + const approverTrimmed = (approver || '').trim(); + let mode; + let finalApprover; + let finalApprovalDate; + + if (status === 'Approved') { + // Approved without an approver is exactly the half-written state the + // consistency guard flags — refuse to create it here. + if (!approverTrimmed) { + throw new Error('--approver is required (and must be non-empty) when --status is Approved.'); + } + mode = 'approved'; + finalApprover = approverTrimmed; + finalApprovalDate = (approvalDate && approvalDate.trim()) || makeNow(); + } else { + // Draft: per plan-alm Phase 4 option 2, a draft does NOT carry an approver. + // Clear any stale approver fields so we never leave "Draft + approver" behind. + mode = 'draft'; + finalApprover = ''; + finalApprovalDate = ''; + } + + planData.PLAN_STATUS = status; + planData.PLAN_MODE = mode; + planData.APPROVED_BY = finalApprover; + planData.APPROVAL_DATE = finalApprovalDate; + + // Atomic write: temp + rename, so a crash mid-write can't truncate the plan + // file that every downstream Phase 0 gate depends on. + const tmp = dataPath + '.tmp'; + fs.writeFileSync(tmp, JSON.stringify(planData, null, 2)); + fs.renameSync(tmp, dataPath); + + let rendered = false; + if (render) { + const htmlPath = planHtmlPath(projectRoot); + invokeRenderer(findRendererPath(rendererPath), dataPath, htmlPath); + rendered = true; + } + + return { + ok: true, + projectRoot, + previousStatus, + status, + mode, + approver: finalApprover, + approvalDate: finalApprovalDate, + rendered, + }; +} + +function parseArgs(argv) { + const args = argv.slice(2); + const out = { + projectRoot: null, status: null, approver: null, approvalDate: null, + force: false, render: false, rendererPath: null, + }; + for (let i = 0; i < args.length; i++) { + if (args[i] === '--projectRoot' && args[i + 1]) out.projectRoot = args[++i]; + else if (args[i] === '--status' && args[i + 1]) out.status = args[++i]; + else if (args[i] === '--approver' && args[i + 1]) out.approver = args[++i]; + else if (args[i] === '--approvalDate' && args[i + 1]) out.approvalDate = args[++i]; + else if (args[i] === '--force') out.force = true; + else if (args[i] === '--render') out.render = true; + else if (args[i] === '--rendererPath' && args[i + 1]) out.rendererPath = args[++i]; + } + return out; +} + +if (require.main === module) { + try { + const result = setPlanStatus(parseArgs(process.argv)); + process.stdout.write(JSON.stringify(result) + '\n'); + process.exit(0); + } catch (err) { + process.stderr.write(`set-plan-status: ${err.message}\n`); + process.exit(1); + } +} + +module.exports = { setPlanStatus, parseArgs, CREATION_STATUSES, LIVE_STATUSES }; diff --git a/plugins/power-pages/scripts/tests/set-plan-status.test.js b/plugins/power-pages/scripts/tests/set-plan-status.test.js new file mode 100644 index 000000000..df71a19fa --- /dev/null +++ b/plugins/power-pages/scripts/tests/set-plan-status.test.js @@ -0,0 +1,138 @@ +'use strict'; + +// Tests for set-plan-status.js — the deterministic Draft/Approved writer that +// replaces plan-alm Phase 4's hand-authored Edits. The key invariants: +// - Approved writes all four fields together (no partial "approver but Draft"). +// - Draft clears the approver (a draft has no approver). +// - Approved without an approver is refused (can't create the broken state). +// - A live plan (In Execution / Completed) is not silently re-drafted. +// - --render regenerates docs/alm-plan.html with the matching badge. + +const test = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('fs'); +const path = require('path'); +const os = require('os'); + +const { setPlanStatus } = require('../lib/set-plan-status'); + +function makeProject(t, planData) { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'set-plan-status-')); + t.after(() => fs.rmSync(root, { recursive: true, force: true })); + fs.mkdirSync(path.join(root, 'docs'), { recursive: true }); + if (planData !== undefined) { + fs.writeFileSync(path.join(root, 'docs', '.alm-plan-data.json'), JSON.stringify(planData, null, 2)); + } + return root; +} + +function readPlan(root) { + return JSON.parse(fs.readFileSync(path.join(root, 'docs', '.alm-plan-data.json'), 'utf8')); +} + +test('Approved writes PLAN_STATUS + PLAN_MODE + APPROVED_BY + APPROVAL_DATE atomically', (t) => { + const root = makeProject(t, { PLAN_STATUS: 'Draft', SITE_NAME: 'T' }); + const res = setPlanStatus({ + projectRoot: root, status: 'Approved', approver: 'Jane Doe', + makeNow: () => '2026-06-22T00:00:00.000Z', + }); + assert.equal(res.ok, true); + assert.equal(res.previousStatus, 'Draft'); + assert.equal(res.status, 'Approved'); + assert.equal(res.mode, 'approved'); + + const plan = readPlan(root); + assert.equal(plan.PLAN_STATUS, 'Approved'); + assert.equal(plan.PLAN_MODE, 'approved'); + assert.equal(plan.APPROVED_BY, 'Jane Doe'); + assert.equal(plan.APPROVAL_DATE, '2026-06-22T00:00:00.000Z'); + // The whole point: no half-written state — all four agree. +}); + +test('Approved trims the approver and defaults APPROVAL_DATE to now', (t) => { + const root = makeProject(t, { PLAN_STATUS: 'Draft' }); + const res = setPlanStatus({ + projectRoot: root, status: 'Approved', approver: ' Spaced Name ', + makeNow: () => '2026-01-02T03:04:05.000Z', + }); + assert.equal(res.approver, 'Spaced Name'); + assert.equal(res.approvalDate, '2026-01-02T03:04:05.000Z'); + assert.equal(readPlan(root).APPROVED_BY, 'Spaced Name'); +}); + +test('Draft clears a stale approver (a draft carries no approver)', (t) => { + // This is exactly the broken state to recover from: approver set but Draft. + const root = makeProject(t, { PLAN_STATUS: 'Draft', APPROVED_BY: 'Stale', APPROVAL_DATE: '2026-01-01T00:00:00Z' }); + const res = setPlanStatus({ projectRoot: root, status: 'Draft' }); + assert.equal(res.mode, 'draft'); + const plan = readPlan(root); + assert.equal(plan.PLAN_STATUS, 'Draft'); + assert.equal(plan.PLAN_MODE, 'draft'); + assert.equal(plan.APPROVED_BY, ''); + assert.equal(plan.APPROVAL_DATE, ''); +}); + +test('Approved without an approver is refused (cannot create the broken state)', (t) => { + const root = makeProject(t, { PLAN_STATUS: 'Draft' }); + assert.throws(() => setPlanStatus({ projectRoot: root, status: 'Approved' }), /--approver is required/); + assert.throws(() => setPlanStatus({ projectRoot: root, status: 'Approved', approver: ' ' }), /--approver is required/); + // Plan must be untouched after a refused write. + assert.equal(readPlan(root).PLAN_STATUS, 'Draft'); +}); + +test('rejects a status this helper does not own', (t) => { + const root = makeProject(t, { PLAN_STATUS: 'Draft' }); + assert.throws(() => setPlanStatus({ projectRoot: root, status: 'In Execution' }), /--status must be one of/); + assert.throws(() => setPlanStatus({ projectRoot: root, status: 'Completed' }), /--status must be one of/); +}); + +test('refuses to regress a live plan without --force, allows with --force', (t) => { + for (const live of ['In Execution', 'Completed']) { + const root = makeProject(t, { PLAN_STATUS: live, APPROVED_BY: 'X' }); + assert.throws( + () => setPlanStatus({ projectRoot: root, status: 'Draft' }), + new RegExp(`already "${live}"`), + `should refuse to re-draft a ${live} plan`, + ); + // With --force it proceeds. + const res = setPlanStatus({ projectRoot: root, status: 'Draft', force: true }); + assert.equal(res.status, 'Draft'); + assert.equal(readPlan(root).PLAN_STATUS, 'Draft'); + } +}); + +test('throws when there is no plan file', (t) => { + const root = makeProject(t); // no plan-data written + assert.throws(() => setPlanStatus({ projectRoot: root, status: 'Draft' }), /No ALM plan found/); +}); + +test('throws on unparseable plan file', (t) => { + const root = makeProject(t); + fs.writeFileSync(path.join(root, 'docs', '.alm-plan-data.json'), 'not json {{{'); + assert.throws(() => setPlanStatus({ projectRoot: root, status: 'Draft' }), /Could not parse/); +}); + +test('--render regenerates docs/alm-plan.html with the matching badge', (t) => { + const root = makeProject(t, { + PLAN_STATUS: 'Draft', SITE_NAME: 'DemoSite', GENERATED_AT: '2026-06-22', + STRATEGY: 'Pipelines', stages: [], steps: [], risks: [], + }); + const res = setPlanStatus({ + projectRoot: root, status: 'Approved', approver: 'Jane', render: true, + makeNow: () => '2026-06-22T00:00:00.000Z', + }); + assert.equal(res.rendered, true); + const html = fs.readFileSync(path.join(root, 'docs', 'alm-plan.html'), 'utf8'); + // Badge derived from plan-data — text "Approved" and the status class applied. + assert.match(html, /Approved<\/span>/); + // Approver surfaces in the Execution tab footer. + assert.match(html, /Jane/); +}); + +test('idempotent: re-writing the same status yields the same plan-data', (t) => { + const root = makeProject(t, { PLAN_STATUS: 'Draft', SITE_NAME: 'T' }); + setPlanStatus({ projectRoot: root, status: 'Approved', approver: 'A', makeNow: () => '2026-06-22T00:00:00.000Z' }); + const first = readPlan(root); + setPlanStatus({ projectRoot: root, status: 'Approved', approver: 'A', makeNow: () => '2026-06-22T00:00:00.000Z' }); + assert.deepEqual(readPlan(root), first); +}); diff --git a/plugins/power-pages/scripts/tests/validate-plan-alm.test.js b/plugins/power-pages/scripts/tests/validate-plan-alm.test.js index bff23dba6..09de558d5 100644 --- a/plugins/power-pages/scripts/tests/validate-plan-alm.test.js +++ b/plugins/power-pages/scripts/tests/validate-plan-alm.test.js @@ -108,3 +108,83 @@ test('validate-plan-alm: approves gracefully when stdin is missing or malformed' }); assert.equal(result.status, 0, 'Expected exit 0 on malformed stdin'); }); + +// --- consistency guard: PLAN_STATUS vs APPROVED_BY in docs/.alm-plan-data.json --- +// +// The badge + approver in the HTML are derived from plan-data, so the JSON is the +// source of truth. These cover the two half-written states the old hand-Edit Phase 4 +// could leave behind (and that set-plan-status.js now prevents at the source). + +// A valid rendered plan (> 500 bytes, has the plan-status marker) so the guard is +// reached, plus an optional .alm-plan-data.json with the given status fields. +function makeProjectWithPlan(planData) { + const dir = makeTempProject(); + const docsDir = path.join(dir, 'docs'); + fs.mkdirSync(docsDir); + fs.writeFileSync( + path.join(docsDir, 'alm-plan.html'), + 'X' + 'x'.repeat(500) + '', + ); + if (planData !== undefined) { + fs.writeFileSync(path.join(docsDir, '.alm-plan-data.json'), JSON.stringify(planData, null, 2)); + } + return dir; +} + +test('validate-plan-alm: blocks when APPROVED_BY is set but PLAN_STATUS is Draft (stuck state)', () => { + const dir = makeProjectWithPlan({ PLAN_STATUS: 'Draft', APPROVED_BY: 'Jane Doe' }); + try { + const { status, stderr } = runValidator(dir); + assert.equal(status, 2, 'Expected exit 2 for approver-set-but-Draft'); + assert.match(stderr, /inconsistent/i); + assert.match(stderr, /Jane Doe/); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } +}); + +test('validate-plan-alm: blocks when PLAN_STATUS is Approved but APPROVED_BY is empty', () => { + const dir = makeProjectWithPlan({ PLAN_STATUS: 'Approved', APPROVED_BY: '' }); + try { + const { status, stderr } = runValidator(dir); + assert.equal(status, 2, 'Expected exit 2 for Approved-without-approver'); + assert.match(stderr, /APPROVED_BY is empty/); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } +}); + +test('validate-plan-alm: approves consistent Approved (status + approver) and Draft (no approver)', () => { + for (const planData of [ + { PLAN_STATUS: 'Approved', APPROVED_BY: 'Jane' }, + { PLAN_STATUS: 'Draft', APPROVED_BY: '' }, + ]) { + const dir = makeProjectWithPlan(planData); + try { + assert.equal(runValidator(dir).status, 0, `Expected exit 0 for ${planData.PLAN_STATUS}`); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } + } +}); + +test('validate-plan-alm: approves a live plan (In Execution / Completed) regardless of approver', () => { + for (const PLAN_STATUS of ['In Execution', 'Completed']) { + const dir = makeProjectWithPlan({ PLAN_STATUS, APPROVED_BY: 'Jane' }); + try { + assert.equal(runValidator(dir).status, 0, `Expected exit 0 for ${PLAN_STATUS}`); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } + } +}); + +test('validate-plan-alm: approves (gracefully) when plan-data is malformed JSON', () => { + const dir = makeProjectWithPlan(); + try { + fs.writeFileSync(path.join(dir, 'docs', '.alm-plan-data.json'), 'not json {{{'); + assert.equal(runValidator(dir).status, 0, 'malformed plan-data is the renderer\'s concern, not this guard'); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } +}); diff --git a/plugins/power-pages/skills/plan-alm/SKILL.md b/plugins/power-pages/skills/plan-alm/SKILL.md index e3122ebb4..9768e1be1 100644 --- a/plugins/power-pages/skills/plan-alm/SKILL.md +++ b/plugins/power-pages/skills/plan-alm/SKILL.md @@ -63,7 +63,31 @@ Steps: - **Continue and keep marker** → set `DEFERRAL_PRESERVED = true` and `DEFERRAL_REASON = {reason}`. Proceed to step 1. Surface a one-line note in the Phase 1 step 9 user report (e.g. *"Note: `.alm-deferred` is preserved — other ALM skills will continue to skip plan-completeness checks for this project."*) so the user remembers the marker remains in effect after planning. - **Cancel** → exit cleanly (don't touch the marker). - If `deferred === false`, skip this step silently and proceed to step 1. + If `deferred === false`, skip this step silently and proceed to step 0b. + +0b. **Offer to approve an existing Draft in place (skip re-planning).** The same `check-alm-plan.js` output from step 0 also carries `exists` and `planStatus`. When `exists === true` **and** `planStatus === "Draft"`, the user already has a saved Draft plan — offer to approve it directly instead of regenerating the whole plan. (This is the only Draft→Approved path; without it, approving a draft means a full re-plan.) + + + > 🚦 **Gate (plan · plan-alm:1.approve-draft):** An existing **Draft** plan was found — approve it in place (no re-plan), re-plan from scratch, or cancel. Approving here writes the status via `set-plan-status.js` and exits without re-running discovery; no deployment is triggered. + + Ask via `AskUserQuestion`: + > "This site already has an ALM plan saved as **Draft** (`docs/alm-plan.html`). What would you like to do?" + + | Question | Header | Options | + |---|---|---| + | What would you like to do? | Existing draft plan | Approve this draft now — no re-plan (Recommended), Re-plan from scratch, Cancel | + + - **Approve this draft now (Recommended)** → capture the approver using the **Phase 4 approver-capture procedure** (the always-interactive prompt with git/OS-name prefill), then write the status atomically with the helper: + + ```bash + node "${PLUGIN_ROOT}/scripts/lib/set-plan-status.js" --projectRoot "." --status Approved --approver "{APPROVER}" --render + ``` + + Commit (`git add docs/alm-plan.html docs/.alm-plan-data.json && git commit -m "Approve ALM plan for {siteName}"`), run skill tracking (Phase 4 finalize), print the Phase 4 next-steps guidance, and **exit**. Do **not** continue to step 1 — there is nothing to re-plan. + - **Re-plan from scratch** → proceed to step 1 (the rest of Phase 1 regenerates the plan; Phase 4 saves the new version). + - **Cancel** → exit cleanly (leave the Draft as-is). + + If `exists === false`, or `planStatus` is anything other than `"Draft"` (`Approved` / `In Execution` / `Completed` / null), skip this step silently and proceed to step 1. 1. **Resolve the site identity from the local project.** `.powerpages-site/website.yml` is the source of truth for `websiteRecordId` and `siteName`, and it is present for **both** Power Pages site types: - **Code / SPA sites** — scaffolded by `/power-pages:create-site` and downloaded with `pac pages download-code-site`. These also have a `powerpages.config.json` and SPA source (`src/`, build output in `dist/`/`build/`). @@ -988,8 +1012,20 @@ Options: 3. **I want to change something** — go back to questions - **If option 3:** Re-run Phase 2 (ask which section to change, then re-gather those answers). Regenerate the plan (repeat Phase 3). Re-present for approval. -- **If option 1 (approved):** Capture the approver (see below). Stamp `` / `` in the HTML and set `` text to `Approved` via `Edit`. **Update `docs/.alm-plan-data.json`**: set `PLAN_STATUS: "Approved"` and `PLAN_MODE: "approved"`. Then run the finalize steps below (skill tracking + commit), print the next-steps guidance, mark task 2 `completed`, and **exit**. -- **If option 2 (draft):** Do **not** capture an approver. Set `` text to `Draft` via `Edit`. Update `docs/.alm-plan-data.json`: `PLAN_STATUS: "Draft"`, `PLAN_MODE: "draft"`. Run the finalize steps below (commit only — skip skill tracking or run it, your choice; commit message `"Add ALM plan for {siteName} (draft)"`), tell the user to re-run `/power-pages:plan-alm` when ready to approve, mark task 2 `completed`, and **exit**. +- **If option 1 (approved):** Capture the approver (see below), then write the status **and** approver atomically with the deterministic helper. **Do not hand-edit the HTML spans** — the badge, `approved-by`, and `approval-date` are all derived from `docs/.alm-plan-data.json` on render, so a manual span Edit is non-durable (the next execution-skill refresh re-derives it). The helper updates plan-data (`PLAN_STATUS`, `PLAN_MODE`, `APPROVED_BY`, `APPROVAL_DATE` — all four together) and re-renders `docs/alm-plan.html`: + + ```bash + node "${PLUGIN_ROOT}/scripts/lib/set-plan-status.js" --projectRoot "." --status Approved --approver "{APPROVER}" --render + ``` + + Then run the finalize steps below (skill tracking + commit), print the next-steps guidance, mark task 2 `completed`, and **exit**. +- **If option 2 (draft):** Do **not** capture an approver. Save as Draft with the same helper (writes plan-data + re-renders; clears any stale approver fields so the plan never ends up "Draft + approver"): + + ```bash + node "${PLUGIN_ROOT}/scripts/lib/set-plan-status.js" --projectRoot "." --status Draft --render + ``` + + Run the finalize steps below (commit only — skip skill tracking or run it, your choice; commit message `"Add ALM plan for {siteName} (draft)"`), tell the user to re-run `/power-pages:plan-alm` when ready to approve, mark task 2 `completed`, and **exit**. **Capturing the approver (option 1 only) — always interactive (#1):** @@ -1007,12 +1043,7 @@ Then **always** ask via `AskUserQuestion` (even when the suggestion is non-empty > > Options: 1. *{suggested name from git/OS, if any}* · 2. Other (enter name) -If the command returned an empty string, present only option 2 (free-text). Store the confirmed result as `APPROVER`, then use `Edit` to replace the spans in `docs/alm-plan.html`: - -- Find `` (or `` / `__APPROVED_BY__`) and replace its inner text with `APPROVER`. -- Find `` and replace its inner text with the current ISO timestamp. - -Both spans are guaranteed to exist in the template — there is exactly one of each in the "Execution Checklist" tab footer. +If the command returned an empty string, present only option 2 (free-text). Store the confirmed result as `APPROVER` and pass it to `set-plan-status.js` (option 1 above) — **do not** hand-edit the `approved-by` / `approval-date` spans. The helper writes `APPROVED_BY` + `APPROVAL_DATE` (current ISO timestamp) into `docs/.alm-plan-data.json` and re-renders, and the template fills both spans from plan-data. This keeps the audit trail and the status in lockstep — the half-written "approver recorded but status still Draft" state (which `validate-plan-alm.js` now blocks) cannot happen. **Finalize (both save options):** diff --git a/plugins/power-pages/skills/plan-alm/scripts/validate-plan-alm.js b/plugins/power-pages/skills/plan-alm/scripts/validate-plan-alm.js index ad92b5cc0..e7ac580c1 100644 --- a/plugins/power-pages/skills/plan-alm/scripts/validate-plan-alm.js +++ b/plugins/power-pages/skills/plan-alm/scripts/validate-plan-alm.js @@ -10,6 +10,7 @@ const path = require('path'); const fs = require('fs'); const { runValidation, findProjectRoot, block, approve, readDeferralMarker } = require('../../../scripts/lib/validation-helpers'); +const { planDataPath } = require('../../../scripts/lib/alm-paths'); runValidation((cwd) => { if (readDeferralMarker(findProjectRoot(cwd) || cwd)) return approve(); // ALM deferred — silent-approve. @@ -64,5 +65,47 @@ runValidation((cwd) => { return; } + // Consistency guard for the creation-time status fields. The badge + approver + // in the HTML are derived from docs/.alm-plan-data.json, so the JSON is the + // source of truth — catch the two half-written states that the old hand-Edit + // Phase 4 could produce (and that set-plan-status.js now prevents). Read-only; + // graceful-approve when there is no plan-data (not every session has one). + const dataPath = planDataPath(projectRoot); + if (fs.existsSync(dataPath)) { + let planData; + try { + planData = JSON.parse(fs.readFileSync(dataPath, 'utf8')); + } catch { + // A malformed plan-data file is the render path's concern, not this guard's + // — don't block the plan-alm session over it here. + planData = null; + } + if (planData) { + const status = planData.PLAN_STATUS || null; + const approver = (planData.APPROVED_BY || '').trim(); + // Draft must NOT carry an approver — "approver set + Draft" is the stuck + // state where the plan shows as approved but never advances (check-alm-plan + // only promotes from "Approved"). + if (status === 'Draft' && approver) { + block( + `validate-plan-alm: docs/.alm-plan-data.json is inconsistent — APPROVED_BY is "${approver}" ` + + `but PLAN_STATUS is "Draft". An approver was recorded but the plan was never moved to ` + + `"Approved", so downstream skills will treat it as unapproved. Re-run the approve step ` + + `(scripts/lib/set-plan-status.js --status Approved --approver "${approver}" --render) so the status matches.` + ); + return; + } + // Approved must HAVE an approver — an empty audit trail on an approved plan. + if (status === 'Approved' && !approver) { + block( + `validate-plan-alm: docs/.alm-plan-data.json is inconsistent — PLAN_STATUS is "Approved" ` + + `but APPROVED_BY is empty. An approved plan must record who approved it. Re-run the approve ` + + `step (scripts/lib/set-plan-status.js --status Approved --approver "" --render).` + ); + return; + } + } + } + approve(); }); From d2caebb4ec613bd385cfdefb05c396c9612ec0f3 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Mon, 22 Jun 2026 17:10:37 +0530 Subject: [PATCH 29/38] Sync legacy JSON mirrors to power-pages v2.6.0 (mirror drift after merging main) The merge resolution left .claude-plugin/marketplace.json and plugins/power-pages/.claude-plugin/plugin.json at 2.5.0 while the sources (marketplace.json + .plugin/plugin.json) are bumped to 2.6.0 for this PR. validate-legacy-compatibility requires each mirror to JSON-deep-equal its source (main #203), so bump both mirrors to 2.6.0. Co-Authored-By: Claude Opus 4.8 (1M context) --- .claude-plugin/marketplace.json | 2 +- plugins/power-pages/.claude-plugin/plugin.json | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 207f7a55e..1028cf6e9 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -13,7 +13,7 @@ "source": "./plugins/power-pages", "description": "Power Pages development and management plugin for Claude Code and GitHub Copilot", "category": "development", - "version": "2.5.0", + "version": "2.6.0", "license": "MIT", "tags": [ "power platform", diff --git a/plugins/power-pages/.claude-plugin/plugin.json b/plugins/power-pages/.claude-plugin/plugin.json index f3fa3f687..5fa58e6c5 100644 --- a/plugins/power-pages/.claude-plugin/plugin.json +++ b/plugins/power-pages/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "power-pages", - "version": "2.5.0", + "version": "2.6.0", "description": "Create and deploy Power Pages sites using modern development approaches. Supports code sites (SPAs) with React, Angular, Vue, or Astro. Includes ALM orchestration (plan-alm) with a solution-splitting decision tree, per-solution pipelines, Azure Blob asset advisory, manifest schema v2 for multi-solution deployments, and force-link remediation for cross-host pipeline migrations.", "author": { "name": "Microsoft", From 1df51b3e2b48c0bd4c135e852e622fa549e15a35 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Mon, 22 Jun 2026 17:24:20 +0530 Subject: [PATCH 30/38] =?UTF-8?q?plan-alm:=20address=20#202=20review=20?= =?UTF-8?q?=E2=80=94=20robust=20approver=20coercion=20+=20runnable=20remed?= =?UTF-8?q?iation=20cmds?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three Copilot findings on validate-plan-alm.js: 1. APPROVED_BY was assumed to be a string — a hand-edited plan-data with a truthy non-string (number/object) made `.trim()` throw, which escaped runValidation and SILENTLY APPROVED, bypassing the consistency guard. Coerce with String() first. Added a regression test (Draft + numeric APPROVED_BY must still block, exit 2). 2-3. The two block-message remediation hints showed a bare `scripts/lib/set-plan-status.js --status ...` with no `node` and no --projectRoot, so following them literally failed with "--projectRoot is required". Replace with a copy/pasteable command matching the skill docs: `node "${PLUGIN_ROOT}/scripts/lib/set-plan-status.js" --projectRoot "" --status Approved --approver "..." --render`. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../scripts/tests/validate-plan-alm.test.js | 16 ++++++++++++++++ .../skills/plan-alm/scripts/validate-plan-alm.js | 15 +++++++++++---- 2 files changed, 27 insertions(+), 4 deletions(-) diff --git a/plugins/power-pages/scripts/tests/validate-plan-alm.test.js b/plugins/power-pages/scripts/tests/validate-plan-alm.test.js index 09de558d5..817be6ee3 100644 --- a/plugins/power-pages/scripts/tests/validate-plan-alm.test.js +++ b/plugins/power-pages/scripts/tests/validate-plan-alm.test.js @@ -154,6 +154,22 @@ test('validate-plan-alm: blocks when PLAN_STATUS is Approved but APPROVED_BY is } }); +test('validate-plan-alm: still blocks the stuck state when APPROVED_BY is a non-string (hand-edited)', () => { + // Regression: a hand-edited plan-data could set APPROVED_BY to a truthy non-string + // (a number/object). Before the String() coercion, `.trim()` threw, runValidation + // swallowed the error and silently APPROVED — bypassing the guard. The coercion + // keeps the Draft+approver stuck state caught (exit 2) instead of leaking through. + const dir = makeProjectWithPlan({ PLAN_STATUS: 'Draft', APPROVED_BY: 123 }); + try { + const { status, stderr } = runValidator(dir); + assert.equal(status, 2, 'non-string approver must not bypass the guard via a thrown .trim()'); + assert.match(stderr, /inconsistent/i); + assert.match(stderr, /123/); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } +}); + test('validate-plan-alm: approves consistent Approved (status + approver) and Draft (no approver)', () => { for (const planData of [ { PLAN_STATUS: 'Approved', APPROVED_BY: 'Jane' }, diff --git a/plugins/power-pages/skills/plan-alm/scripts/validate-plan-alm.js b/plugins/power-pages/skills/plan-alm/scripts/validate-plan-alm.js index e7ac580c1..775a59e2b 100644 --- a/plugins/power-pages/skills/plan-alm/scripts/validate-plan-alm.js +++ b/plugins/power-pages/skills/plan-alm/scripts/validate-plan-alm.js @@ -82,7 +82,12 @@ runValidation((cwd) => { } if (planData) { const status = planData.PLAN_STATUS || null; - const approver = (planData.APPROVED_BY || '').trim(); + // Coerce before trimming: APPROVED_BY is normally a string, but a hand-edited + // plan-data could set it to a truthy non-string (number/object), and calling + // .trim() on that throws — which would escape runValidation and silently + // approve, bypassing this guard. String(...) keeps the guard robust to any + // malformed-but-parseable JSON. + const approver = String(planData.APPROVED_BY || '').trim(); // Draft must NOT carry an approver — "approver set + Draft" is the stuck // state where the plan shows as approved but never advances (check-alm-plan // only promotes from "Approved"). @@ -90,8 +95,9 @@ runValidation((cwd) => { block( `validate-plan-alm: docs/.alm-plan-data.json is inconsistent — APPROVED_BY is "${approver}" ` + `but PLAN_STATUS is "Draft". An approver was recorded but the plan was never moved to ` + - `"Approved", so downstream skills will treat it as unapproved. Re-run the approve step ` + - `(scripts/lib/set-plan-status.js --status Approved --approver "${approver}" --render) so the status matches.` + `"Approved", so downstream skills will treat it as unapproved. Re-run the approve step so ` + + `the status matches:\n` + + ` node "\${PLUGIN_ROOT}/scripts/lib/set-plan-status.js" --projectRoot "${projectRoot}" --status Approved --approver "${approver}" --render` ); return; } @@ -100,7 +106,8 @@ runValidation((cwd) => { block( `validate-plan-alm: docs/.alm-plan-data.json is inconsistent — PLAN_STATUS is "Approved" ` + `but APPROVED_BY is empty. An approved plan must record who approved it. Re-run the approve ` + - `step (scripts/lib/set-plan-status.js --status Approved --approver "" --render).` + `step with the approver's name:\n` + + ` node "\${PLUGIN_ROOT}/scripts/lib/set-plan-status.js" --projectRoot "${projectRoot}" --status Approved --approver "" --render` ); return; } From cb68f868b9ec42dc1f82f3e02322cd5abb6b3d9b Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Mon, 22 Jun 2026 18:16:40 +0530 Subject: [PATCH 31/38] plan-alm: fix 5 issues surfaced by an EDM-site end-to-end run MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All five confirmed against source (and PAC 2.8.1) during a data-model/EDM site run of plan-alm: 1. getEnvironmentUrl() grepped "Environment URL:" but `pac env who` on PAC 2.8.x prints the URL under "Org URL:" → returned null, so every caller relying on the pac-env-who fallback (verify-alm-prerequisites without --envUrl, the datamodel / solution / permissions validators, and the #204 declarative-site path) silently failed. Match either label. Extracted parseEnvironmentUrl() (pure) + tests. 2. `pac env list --output json` is INVALID on PAC 2.8.1 (env list accepts only --filter), so ENV_LIST pre-fill never worked. New list-environments.js parses the plain `pac env list` table into JSON {displayName, environmentId, environmentUrl, uniqueName, active}; verified against live PAC (237 rows). Updated all call sites (plan-alm, setup-pipeline, ensure-pipelines-host, cicd-pipeline-patterns.md) + AGENTS.md. Pure parseEnvList() + tests. 3. estimate-solution-size.js hardcoded siteType:'code-site', mislabeling every EDM/data-model site. Now resolves via new --siteType arg (plan-alm passes SITE_TYPE from Phase 1) with a local marker fallback, emitting canonical 'code' | 'data-model' | 'unknown'. resolveSiteType() + tests. 4. plan-alm risk rule #9 emitted a spurious "pipeline host resolution did not run" warning whenever a pipeline already exists (rawDiscovery.hostResolution is legitimately null then, per the Phase 1 Step 12 skip rule). Added a PIPELINE_DONE carve-out so the warning only fires for fresh-pipeline projects. 5. NEW env-match guard (plan-alm Phase 1 step 6b, gate plan-alm:1.env-match, warn+prompt): cross-checks `pac env who` against the project's recorded env URL (powerpages.config.json / .solution-manifest.json) and a websiteRecordId existence probe, so discovery can't silently run against the wrong environment. Catalogued in approval-gates.md §6.1; alm-lint 0 findings. 1279 tests pass (+10). Riding the existing 2.6.0 bump on this branch. Co-Authored-By: Claude Opus 4.8 (1M context) --- plugins/power-pages/AGENTS.md | 1 + .../power-pages/references/approval-gates.md | 3 +- .../references/cicd-pipeline-patterns.md | 6 +- .../scripts/lib/estimate-solution-size.js | 35 ++++++- .../scripts/lib/list-environments.js | 96 +++++++++++++++++++ .../scripts/lib/validation-helpers.js | 19 +++- .../tests/estimate-solution-size.test.js | 39 ++++++++ .../scripts/tests/list-environments.test.js | 54 +++++++++++ .../scripts/tests/validation-helpers.test.js | 42 ++++++++ .../skills/ensure-pipelines-host/SKILL.md | 2 +- plugins/power-pages/skills/plan-alm/SKILL.md | 38 ++++++-- .../skills/setup-pipeline/SKILL.md | 4 +- 12 files changed, 322 insertions(+), 17 deletions(-) create mode 100644 plugins/power-pages/scripts/lib/list-environments.js create mode 100644 plugins/power-pages/scripts/tests/list-environments.test.js diff --git a/plugins/power-pages/AGENTS.md b/plugins/power-pages/AGENTS.md index da8df303d..261b4bc40 100644 --- a/plugins/power-pages/AGENTS.md +++ b/plugins/power-pages/AGENTS.md @@ -233,6 +233,7 @@ Shared lib modules live at `scripts/lib/` and are imported by other scripts via #### PP Pipelines +- `scripts/lib/list-environments.js`: Enumerates the Dataverse environments the signed-in PAC user can access, as JSON, for `ENV_LIST` pre-fill (plan-alm Phase 1 Step 5, setup-pipeline, ensure-pipelines-host "Other (paste URL)" prompts). **Why it exists:** the skills used to run `pac env list --output json`, which is INVALID on current PAC CLI (verified 2.8.1 — `pac env list` accepts only `--filter` and errors on `--output`), so the JSON pre-fill silently never worked. This helper runs the plain `pac env list` and parses its table (anchored on the env GUID + https URL + unique-name tokens, so display names with spaces survive). `pac admin list --json` was rejected as the source — it's admin-only and tenant-wide, the wrong scope for a per-user pre-fill. Exports `parseEnvList(stdout)` (pure, tested) + `listEnvironments()`. CLI prints a JSON array of `{ displayName, environmentId, environmentUrl, uniqueName, active }`; prints `[]` and exits 0 on any failure (unauthenticated PAC, parse miss) so callers degrade to manual entry. Match envs by `environmentUrl` origin. - `scripts/lib/discover-pipelines-host.js`: Discovers the tenant-level default Power Platform Pipelines host environment URL by calling `RetrieveSetting('DefaultCustomPipelinesHostEnvForTenant')` on the dev/source environment. Args: `--envUrl`, `--token`, `--userId`. Output: `{ found, hostEnvUrl }`. Exit 0 (including when not found); exit 1 on error. - `scripts/lib/create-deployment-environment.js`: Creates a `deploymentenvironments` record in the Pipelines host environment using the **unprefixed** field schema (`name`, `environmentid`, `environmenttype`), then polls `validationstatus` until Succeeded (`200000001`) or Failed (`200000002`). Args: `--hostEnvUrl`, `--token`, `--name`, `--bapEnvId`, `--environmentType` (`200000000` Dev / `200000001` Target), `--environmentUrl` (opt, only echoed in output marker). Idempotent: if a record already exists for the same `environmentid`, returns it with `reused: true`. Output: `{ deploymentEnvironmentId, name, bapEnvId, environmentUrl, environmentType, validationStatus, reused }`. - `scripts/lib/create-deployment-pipeline.js`: Creates a `deploymentpipelines` record, associates the source environment via `$ref` (relative path + `@odata.context`), and creates `deploymentstages` records for each target environment. Args: `--hostEnvUrl`, `--token`, `--pipelineName`, `--description`, `--sourceDeploymentEnvironmentId`, `--stagesJson` (JSON array of `{ name, targetDeploymentEnvironmentId, order }`). Output: `{ pipelineId, pipelineName, stages[] }`. diff --git a/plugins/power-pages/references/approval-gates.md b/plugins/power-pages/references/approval-gates.md index 0a080a6f5..f60ea1933 100644 --- a/plugins/power-pages/references/approval-gates.md +++ b/plugins/power-pages/references/approval-gates.md @@ -256,7 +256,7 @@ Each section lists every `AskUserQuestion` in that skill. Catalog rows are marke --- -### 6.1 `plan-alm` (15 calls; planner) +### 6.1 `plan-alm` (16 calls; planner) > `plan-alm` is a **planner** — it produces an approved/draft HTML plan and never executes. The execution gates that used to live in Phases 5–8 (deploy-failure, post-deploy activation, manual export/import checkpoint) now belong to the individual ALM skills the user runs afterward; they are catalogued under those skills' sections, not here. @@ -265,6 +265,7 @@ Each section lists every `AskUserQuestion` in that skill. Catalog rows are marke | `plan-alm:1.deferral` | gate | progress | 1 | `.alm-deferred` marker present — *"Continue with deferral / remove and proceed / cancel"* | `deferral-marker` | | `plan-alm:1.approve-draft` | gate | plan | 1 (0b) | Existing **Draft** plan found — *"Approve this draft now (no re-plan) / re-plan from scratch / cancel"*. Approve writes status via `set-plan-status.js` and exits | nothing | | `plan-alm:1.completeness` | gate | progress | 1 | Completeness check found gaps — *"Sync first / plan with gaps / cancel"* | nothing | +| `plan-alm:1.env-match` | gate | progress | 1 (6b) | `pac env who` env ≠ project's (recorded-URL mismatch or `websiteRecordId` not found in connected env) — *"Switch PAC env & re-run / continue against connected env (degraded) / cancel"*. Only fires on a detected mismatch | nothing | | `plan-alm:2.q1-existing` | gate | plan | 2 (Q1) | `SOLUTION_DONE=true` — *"Use existing solution **{name}**?"* | nothing | | `plan-alm:2.q1-fresh` | gate | plan | 2 (Q1) | `SOLUTION_DONE=false` — *"Include solution setup in plan?"* | nothing | | `plan-alm:2.q1b-split` | gate | plan | 2 (Q1b) | `RECOMMEND_SPLIT=true` — *"Follow recommended {strategy} split?"* | nothing | diff --git a/plugins/power-pages/references/cicd-pipeline-patterns.md b/plugins/power-pages/references/cicd-pipeline-patterns.md index 8562857d7..6425efac6 100644 --- a/plugins/power-pages/references/cicd-pipeline-patterns.md +++ b/plugins/power-pages/references/cicd-pipeline-patterns.md @@ -513,13 +513,13 @@ Accept: application/json Returns `{ "SettingValue": "{BAP-environment-GUID}" }` or empty/null if no default is configured. -Cross-reference the GUID with `pac env list` output to find the host environment URL: +Cross-reference the GUID with the environment list to find the host environment URL: ```bash -pac env list --output json 2>/dev/null +node "${PLUGIN_ROOT}/scripts/lib/list-environments.js" ``` -Match on `EnvironmentId` field. If no match, probe each environment from `pac env list` with: +This emits a JSON array of `{ displayName, environmentId, environmentUrl, uniqueName, active }`. (It parses `pac env list`; the older `pac env list --output json` is invalid on current PAC CLI, which only accepts `--filter` on `env list`.) Match the GUID on the `environmentId` field. If no match, probe each environment from `pac env list` with: ``` GET {envUrl}/api/data/v9.1/deploymentpipelines?$top=0 diff --git a/plugins/power-pages/scripts/lib/estimate-solution-size.js b/plugins/power-pages/scripts/lib/estimate-solution-size.js index 4a29b7cfe..de9ec72d9 100644 --- a/plugins/power-pages/scripts/lib/estimate-solution-size.js +++ b/plugins/power-pages/scripts/lib/estimate-solution-size.js @@ -65,6 +65,7 @@ function parseArgs(argv) { datamodelManifest: null, solutionId: null, projectRoot: null, + siteType: null, }; for (let i = 0; i < args.length; i++) { if (args[i] === '--envUrl' && args[i + 1]) out.envUrl = args[++i]; @@ -75,10 +76,35 @@ function parseArgs(argv) { else if (args[i] === '--datamodelManifest' && args[i + 1]) out.datamodelManifest = args[++i]; else if (args[i] === '--solutionId' && args[i + 1]) out.solutionId = args[++i]; else if (args[i] === '--projectRoot' && args[i + 1]) out.projectRoot = args[++i]; + else if (args[i] === '--siteType' && args[i + 1]) out.siteType = args[++i]; } return out; } +// Resolve the build-axis site type for the estimator's diagnostic `siteType` +// output field. Prefer the caller-supplied value (plan-alm resolves this in +// Phase 1 via detect-project-context.js, the authoritative source), and fall +// back to a lightweight local probe of the same markers documented in CLAUDE.md: +// - `powerpages.config.json` → code / SPA site +// - `.powerpages-site/.portalconfig/` → declarative design-studio (data-model/EDM) site +// Returns the canonical values ('code' | 'data-model') to match +// detect-project-context.js — NOT the old hardcoded 'code-site', which mislabeled +// every EDM/data-model site as a code site. Returns 'unknown' when neither marker +// is present (e.g. running outside a project root). +function resolveSiteType(explicitSiteType, projectRoot) { + if (explicitSiteType) return explicitSiteType; + if (!projectRoot) return 'unknown'; + const fs = require('fs'); + const path = require('path'); + try { + if (fs.existsSync(path.join(projectRoot, 'powerpages.config.json'))) return 'code'; + if (fs.existsSync(path.join(projectRoot, '.powerpages-site', '.portalconfig'))) return 'data-model'; + } catch { + // Filesystem probe is best-effort — a diagnostic label must never be fatal. + } + return 'unknown'; +} + // Page size for paginated OData queries. Dataverse caps `Prefer: odata.maxpagesize` // at 5000 — requesting more is silently downgraded. Using the cap minimizes // roundtrips for large sites. @@ -738,7 +764,7 @@ async function countSolutionMembership(envUrl, solutionId, token, sitePpcIdSet = }; } -async function estimateSolutionSize({ envUrl, websiteRecordId, token, publisherPrefix, siteName, datamodelManifest, solutionId, projectRoot }) { +async function estimateSolutionSize({ envUrl, websiteRecordId, token, publisherPrefix, siteName, datamodelManifest, solutionId, projectRoot, siteType }) { if (!envUrl || !websiteRecordId) { throw new Error('--envUrl and --websiteRecordId are required'); } @@ -1122,7 +1148,10 @@ async function estimateSolutionSize({ envUrl, websiteRecordId, token, publisherP // scope so reviewers can spot the divergence. envVarCountTenantWide, mediaRatio: Math.round(webMeasure.mediaRatio * 100) / 100, - siteType: 'code-site', + // Build-axis label: 'code' | 'data-model' | 'unknown' (was hardcoded + // 'code-site', which mislabeled every declarative/EDM site). Prefers the + // caller-supplied --siteType (plan-alm Phase 1), falls back to a local marker probe. + siteType: resolveSiteType(siteType, projectRoot), tables: tables.map((t) => ({ logicalName: t.logicalName, attributeCount: t.attributeCount || 0 })), // Dependency edges among the scoped tables ([a,b], lowercased, a prints JSON array to stdout +// +// Output (JSON array; empty [] when PAC is unauthenticated / the command fails — +// the pre-fill is best-effort and callers degrade gracefully to manual entry): +// [ { "displayName": "...", "environmentId": "...", "environmentUrl": "https://…", +// "uniqueName": "...", "active": true|false }, ... ] +// +// Exit 0 always (callers parse stdout; [] means "no pre-fill available"). + +const { execSync } = require('child_process'); + +// Parse the plain `pac env list` table. Pure + exported for unit testing. +// Example real output (PAC 2.8.1) — note the header row, the "Connected as" banner +// line, and that the active env is flagged with `*` in the leading "Active" column: +// +// Connected as admin@contoso.onmicrosoft.com +// Active Display Name Environment ID Environment URL Unique Name +// * Contoso Dev d664a1f5-5c5b-efbf-9cc9-c1923c437109 https://contosodev.crm.dynamics.com/ unq78bd16d6e4baf01189f56045bd003 +// Contoso Prod e8ccb697-db78-e2d6-b721-ef23eedbc302 https://contosoprod.crm4.dynamics.com/ unqe4574a3ea1bff01195c56045bd03c +// +// Display names contain spaces and variable padding, so we anchor on the three +// unambiguous tokens that always appear in order — the 36-char environment GUID, +// the https URL, and the trailing unique name — and treat everything before the +// GUID as `[activeMarker] + displayName`. +function parseEnvList(stdout) { + if (!stdout || typeof stdout !== 'string') return []; + const rows = []; + for (const rawLine of stdout.split(/\r?\n/)) { + const line = rawLine.replace(/\s+$/, ''); + if (!line.trim()) continue; + // Skip the "Connected as ..." banner and the column header row. + if (/^Connected as\b/i.test(line.trim())) continue; + if (/^Active\s+Display Name\b/i.test(line.trim())) continue; + + // prefix = (optional `*` active marker) + display name; then GUID, URL, uniqueName. + const m = line.match( + /^(.*?)\s+([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})\s+(https:\/\/\S+)\s+(\S+)\s*$/i, + ); + if (!m) continue; + const prefix = m[1]; + // The active env is flagged with a leading `*` in the "Active" column. + const active = /^\s*\*/.test(prefix); + const displayName = prefix.replace(/^\s*\*?\s*/, '').trim(); + rows.push({ + displayName, + environmentId: m[2], + environmentUrl: m[3].replace(/\/+$/, ''), + uniqueName: m[4], + active, + }); + } + return rows; +} + +function listEnvironments() { + let stdout = ''; + try { + stdout = execSync('pac env list', { encoding: 'utf8', timeout: 20000 }); + } catch (e) { + // Best-effort: an unauthenticated / failing PAC CLI yields no pre-fill, not an + // error — callers (plan-alm Phase 1, setup-pipeline) fall back to manual entry. + // `pac` writes its table to stdout even on some non-zero exits, so try to parse + // whatever was captured before giving up. + stdout = (e && (e.stdout || '')) || ''; + } + return parseEnvList(stdout); +} + +if (require.main === module) { + // Never throw to the caller — emit [] on any failure so the consumer always + // receives parseable JSON. + let result = []; + try { result = listEnvironments(); } catch { result = []; } + process.stdout.write(JSON.stringify(result) + '\n'); + process.exit(0); +} + +module.exports = { parseEnvList, listEnvironments }; diff --git a/plugins/power-pages/scripts/lib/validation-helpers.js b/plugins/power-pages/scripts/lib/validation-helpers.js index b9a097ee5..6c1d7905c 100644 --- a/plugins/power-pages/scripts/lib/validation-helpers.js +++ b/plugins/power-pages/scripts/lib/validation-helpers.js @@ -160,11 +160,25 @@ function getAuthToken(resourceUrl) { * Gets the environment URL from `pac env who`. * @returns {string|null} Environment URL, or null */ +// Pure parser (exported for unit testing — getEnvironmentUrl() shells out, so the +// regex itself is tested here against raw banner text rather than through execSync). +// PAC CLI labels the environment URL differently across versions / commands: +// `pac env who` on 2.8.x prints it under "Org URL:" (inside "Organization +// Information"); older/other builds emit "Environment URL:". Match EITHER — with +// only the "Environment URL:" form this returned null on 2.8.x and every caller +// relying on the pac-env-who fallback (verify-alm-prerequisites when --envUrl is +// omitted, the datamodel / solution / permissions validators) silently failed. +// Example 2.8.1 line: ` Org URL: https://org4a2942d9.crm17.dynamics.com/` +function parseEnvironmentUrl(whoOutput) { + if (!whoOutput) return null; + const match = whoOutput.match(/(?:Org URL|Environment URL):\s*(https:\/\/[^\s]+)/i); + return match ? match[1].replace(/\/+$/, '') : null; +} + function getEnvironmentUrl() { try { const output = execSync('pac env who', { encoding: 'utf8', timeout: 15000 }); - const match = output.match(/Environment URL:\s*(https:\/\/[^\s]+)/i); - return match ? match[1].replace(/\/+$/, '') : null; + return parseEnvironmentUrl(output); } catch { return null; } @@ -326,6 +340,7 @@ module.exports = { odataGet, odataGetAll, getEnvironmentUrl, + parseEnvironmentUrl, getPacAuthInfo, CLOUD_TO_API, CLOUD_TO_SITE_DOMAIN, diff --git a/plugins/power-pages/scripts/tests/estimate-solution-size.test.js b/plugins/power-pages/scripts/tests/estimate-solution-size.test.js index 78055b750..c79be4718 100644 --- a/plugins/power-pages/scripts/tests/estimate-solution-size.test.js +++ b/plugins/power-pages/scripts/tests/estimate-solution-size.test.js @@ -789,3 +789,42 @@ test('estimateSolutionSize tableCountScope is "unavailable" with no local signal assert.equal(result.tableCountScope, 'unavailable'); assert.deepEqual(result.tableRelationships, []); }); + +// --- resolveSiteType: correct build-axis label (was hardcoded 'code-site') ----- + +test('resolveSiteType prefers the explicit caller value (plan-alm Phase 1 detection)', () => { + const { resolveSiteType } = require('../lib/estimate-solution-size'); + assert.equal(resolveSiteType('data-model', '/whatever'), 'data-model'); + assert.equal(resolveSiteType('code', null), 'code'); +}); + +test('resolveSiteType falls back to local markers: powerpages.config.json => code, .portalconfig => data-model', () => { + const fs = require('fs'); + const os = require('os'); + const path = require('path'); + const { resolveSiteType } = require('../lib/estimate-solution-size'); + + const codeRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'est-stype-code-')); + const edmRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'est-stype-edm-')); + const bareRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'est-stype-bare-')); + try { + fs.writeFileSync(path.join(codeRoot, 'powerpages.config.json'), '{}'); + assert.equal(resolveSiteType(null, codeRoot), 'code'); + + fs.mkdirSync(path.join(edmRoot, '.powerpages-site', '.portalconfig'), { recursive: true }); + assert.equal(resolveSiteType(null, edmRoot), 'data-model', 'EDM/declarative site must NOT be mislabeled code'); + + // No markers and no projectRoot → 'unknown', never a wrong guess. + assert.equal(resolveSiteType(null, bareRoot), 'unknown'); + assert.equal(resolveSiteType(null, null), 'unknown'); + } finally { + for (const d of [codeRoot, edmRoot, bareRoot]) fs.rmSync(d, { recursive: true, force: true }); + } +}); + +test('parseArgs captures --siteType', () => { + const { parseArgs } = require('../lib/estimate-solution-size'); + const a = parseArgs(['node', 'x', '--siteType', 'data-model', '--envUrl', 'https://x']); + assert.equal(a.siteType, 'data-model'); + assert.equal(parseArgs(['node', 'x']).siteType, null); +}); diff --git a/plugins/power-pages/scripts/tests/list-environments.test.js b/plugins/power-pages/scripts/tests/list-environments.test.js new file mode 100644 index 000000000..25ec12821 --- /dev/null +++ b/plugins/power-pages/scripts/tests/list-environments.test.js @@ -0,0 +1,54 @@ +'use strict'; + +const test = require('node:test'); +const assert = require('node:assert/strict'); + +const { parseEnvList } = require('../lib/list-environments'); + +// Real `pac env list` table shape on PAC 2.8.1 (the command that replaced the +// invalid `pac env list --output json`). Header row + "Connected as" banner must +// be skipped; the active env carries a leading `*`. +const SAMPLE = [ + 'Connected as admin@contoso.onmicrosoft.com', + 'Active Display Name Environment ID Environment URL Unique Name', + ' 1841 Community V2 Fresh d664a1f5-5c5b-efbf-9cc9-c1923c437109 https://1841communityv2fresh.crm.dynamics.com/ unq78bd16d6e4baf01189f56045bd003', + '* Contoso Dev e8ccb697-db78-e2d6-b721-ef23eedbc302 https://contosodev.crm4.dynamics.com/ unqe4574a3ea1bff01195c56045bd03c', + ' 281025 a33797f3-ca8b-e81b-97f9-01dec883d806 https://org9cf0ed45.crm.dynamics.com/ unqb66c441afcb3f01195c56045bd021', +].join('\n'); + +test('parseEnvList extracts each env row with display name, GUID, URL (trailing slash stripped), unique name', () => { + const rows = parseEnvList(SAMPLE); + assert.equal(rows.length, 3, 'header + banner lines must be skipped, 3 data rows kept'); + + assert.deepEqual(rows[0], { + displayName: '1841 Community V2 Fresh', + environmentId: 'd664a1f5-5c5b-efbf-9cc9-c1923c437109', + environmentUrl: 'https://1841communityv2fresh.crm.dynamics.com', // trailing / stripped + uniqueName: 'unq78bd16d6e4baf01189f56045bd003', + active: false, + }); + + // Display names with spaces are preserved; the active `*` marker is parsed, not + // leaked into the display name. + assert.equal(rows[1].displayName, 'Contoso Dev'); + assert.equal(rows[1].active, true); + assert.equal(rows[1].environmentUrl, 'https://contosodev.crm4.dynamics.com'); + + // Numeric-leading display names are fine (anchored on the GUID, not the name). + assert.equal(rows[2].displayName, '281025'); + assert.equal(rows[2].active, false); +}); + +test('parseEnvList returns [] for empty / banner-only / malformed input', () => { + assert.deepEqual(parseEnvList(''), []); + assert.deepEqual(parseEnvList(null), []); + assert.deepEqual(parseEnvList('Connected as x@y.com\nActive Display Name Environment ID Environment URL Unique Name'), []); + // A pac error banner with no table rows. + assert.deepEqual(parseEnvList('Error: An unknown argument --output was passed.'), []); +}); + +test('parseEnvList ignores rows without all three anchor tokens (GUID + URL + uniqueName)', () => { + // A wrapped/partial line missing the URL must not produce a half-populated row. + const partial = 'Connected as a@b.com\n Half Row d664a1f5-5c5b-efbf-9cc9-c1923c437109 unqonly'; + assert.deepEqual(parseEnvList(partial), []); +}); diff --git a/plugins/power-pages/scripts/tests/validation-helpers.test.js b/plugins/power-pages/scripts/tests/validation-helpers.test.js index 0e739fe9a..d8fcc1ba6 100644 --- a/plugins/power-pages/scripts/tests/validation-helpers.test.js +++ b/plugins/power-pages/scripts/tests/validation-helpers.test.js @@ -112,3 +112,45 @@ test('odataGet throws on transport error', async () => { await assert.rejects(() => odataGet('https://x/y', 'tok', fakeRequest), /OData request failed/); }); + +// --- parseEnvironmentUrl: PAC `pac env who` label compatibility (2.8.x "Org URL:") --- + +test('parseEnvironmentUrl extracts the URL from the 2.8.x "Org URL:" banner', () => { + const { parseEnvironmentUrl } = require(helpersPath); + // Real `pac env who` shape on PAC 2.8.1 — the URL is under "Org URL:", + // and there is an "Environment ID:" line but NO "Environment URL:" line. + const who = [ + 'Connected as admin@contoso.onmicrosoft.com', + 'Connected to... CitizenServicesDev', + 'Organization Information', + ' Org ID: 00e3facc-644f-f111-b31f-6045bd29e553', + ' Friendly Name: CitizenServicesDev', + ' Org URL: https://org4a2942d9.crm17.dynamics.com/', + ' Environment ID: d3b0c5e9-6fd9-e4f0-9bdc-eaf672fb6c5d', + ].join('\n'); + assert.equal(parseEnvironmentUrl(who), 'https://org4a2942d9.crm17.dynamics.com'); +}); + +test('parseEnvironmentUrl still extracts the URL from the legacy "Environment URL:" banner', () => { + const { parseEnvironmentUrl } = require(helpersPath); + const who = 'Environment URL: https://legacy.crm.dynamics.com/\nUser: x@y.com'; + assert.equal(parseEnvironmentUrl(who), 'https://legacy.crm.dynamics.com'); +}); + +test('parseEnvironmentUrl returns null when no URL label is present (and on empty input)', () => { + const { parseEnvironmentUrl } = require(helpersPath); + assert.equal(parseEnvironmentUrl('Connected as x@y.com\nNo URL here'), null); + assert.equal(parseEnvironmentUrl(''), null); + assert.equal(parseEnvironmentUrl(null), null); +}); + +test('getEnvironmentUrl parses the 2.8.x "Org URL:" output via mocked execSync', (t) => { + const originalExecSync = childProcess.execSync; + childProcess.execSync = () => ' Org URL: https://orgABC.crm.dynamics.com/\n'; + t.after(() => { childProcess.execSync = originalExecSync; }); + // Re-require fresh so the module binds the mocked execSync. + delete require.cache[require.resolve(helpersPath)]; + const { getEnvironmentUrl } = require(helpersPath); + assert.equal(getEnvironmentUrl(), 'https://orgABC.crm.dynamics.com'); + delete require.cache[require.resolve(helpersPath)]; +}); diff --git a/plugins/power-pages/skills/ensure-pipelines-host/SKILL.md b/plugins/power-pages/skills/ensure-pipelines-host/SKILL.md index f639b77ff..fcb0de0e6 100644 --- a/plugins/power-pages/skills/ensure-pipelines-host/SKILL.md +++ b/plugins/power-pages/skills/ensure-pipelines-host/SKILL.md @@ -524,7 +524,7 @@ Track the total eligible count separately — when `eligible.length > 5`, surfac - `eligible.length <= 5` → empty string (no suffix; all envs visible). - `eligible.length > 5` → ` Showing top 5 of {N}; the remaining {N-5} eligible env(s) can be reached via the "Other (paste URL)" entry.` (leading space). -When the user picks "Other (paste URL)", **pre-fill** the URL input with `pac env list --output json` results so they can paste-or-pick from the full tenant inventory rather than typing a URL by hand. +When the user picks "Other (paste URL)", **pre-fill** the URL input with the environment list from `node "${PLUGIN_ROOT}/scripts/lib/list-environments.js"` (parses `pac env list` into JSON `{ displayName, environmentId, environmentUrl, uniqueName, active }`; the old `pac env list --output json` is invalid on current PAC CLI) so they can paste-or-pick from the inventory rather than typing a URL by hand. **Test scenarios to verify when changing this prompt:** - 0 eligible → sub-option `a` dropped (sub-prompt shows only `b` / `c`). diff --git a/plugins/power-pages/skills/plan-alm/SKILL.md b/plugins/power-pages/skills/plan-alm/SKILL.md index 9768e1be1..b15f02601 100644 --- a/plugins/power-pages/skills/plan-alm/SKILL.md +++ b/plugins/power-pages/skills/plan-alm/SKILL.md @@ -124,9 +124,9 @@ Steps: 5. Run silently: ```bash - pac env list --output json 2>/dev/null + node "${PLUGIN_ROOT}/scripts/lib/list-environments.js" ``` - Store output as `ENV_LIST` for pre-filling environment URLs in Phase 2. + Store the JSON array as `ENV_LIST` for pre-filling environment URLs in Phase 2. (This helper parses `pac env list`; the old `pac env list --output json` is invalid on current PAC CLI — `pac env list` only accepts `--filter` — so the helper exists to produce the JSON the table form doesn't. It prints `[]` and exits 0 if PAC is unauthenticated, so pre-fill simply degrades to manual entry.) Each entry is `{ displayName, environmentId, environmentUrl, uniqueName, active }`. 6. Acquire dev environment token (silently): ```bash @@ -136,6 +136,32 @@ Steps: **Track plan quality.** Initialize a `PLAN_QUALITY` accumulator to `"complete"` at the start of Phase 1. If this token acquisition fails (auth error), set `DEV_TOKEN = null`, set `PLAN_QUALITY = "degraded"`, and record the cause (e.g. *"dev-environment auth failed — contents/size/host discovery skipped"*) — then continue. Contents discovery is skipped gracefully, but the resulting plan is built on partial inputs; Phase 3 surfaces this as a prominent risk so the user reviews before executing. (There is no execute path to block here — `plan-alm` only plans — but a degraded plan must be visibly flagged.) +6b. **Environment-match guard** — confirm `pac env who` points at the project's environment *before* running discovery. `DEV_ENV_URL` comes from whatever environment PAC happens to be connected to, which is **not** guaranteed to be the project's. If it isn't, every query in Steps 7–12 runs against the wrong environment and silently produces a degraded plan (zero/À-côté site settings, wrong size, wrong host) that *looks* valid. Cross-check both signals available: + + 1. **Recorded-URL comparison** (no token needed): collect any environment URL the project already records — `powerpages.config.json` → `environmentUrl` (code/SPA sites; absent for declarative/EDM sites) and `.solution-manifest.json` → its `environmentUrl`/`environmentUrl`-equivalent field if present. Normalize by **origin** (lowercase host, drop trailing slash + path/query). If any recorded URL exists and its origin **differs** from `DEV_ENV_URL`'s origin → **mismatch**. + 2. **Site-existence probe** (covers declarative/EDM sites that record no URL; only when `DEV_TOKEN` is available): verify the site's `websiteRecordId` actually exists in the connected env: + ``` + GET {DEV_ENV_URL}/api/data/v9.2/powerpagesites({websiteRecordId})?$select=powerpagesiteid + Authorization: Bearer {DEV_TOKEN} + ``` + A `404` (or empty result) means the connected environment does not contain this site → **mismatch**. (Skip this probe when `DEV_TOKEN = null` — Step 6 already degraded the plan; don't double-prompt.) + + If **neither** signal indicates a mismatch, continue silently to Step 7 — do not prompt. Only prompt on a detected mismatch: + + + > 🚦 **Gate (progress · plan-alm:1.env-match):** PAC CLI is connected to an environment that does not match the project's. Switch and re-run, or continue against the connected env (degraded plan). + + Ask via `AskUserQuestion`: + + | Question | Header | Options | + |---|---|---| + | PAC CLI is connected to **{DEV_ENV_NAME}** (`{DEV_ENV_URL}`), which does not match this project's configured environment ({recorded URL, or "this site was not found there"}). Discovery will run against the connected environment. How do you want to proceed? | Env Mismatch | Cancel — switch PAC env, then re-run (Recommended), Continue against {DEV_ENV_NAME} anyway, Cancel | + + - **Cancel — switch PAC env (Recommended)**: stop the skill. Tell the user to point PAC at the right environment (`pac auth select --name ` or `pac org select --environment `) and re-run `/power-pages:plan-alm`. Nothing has been written. + - **Continue anyway**: proceed to Step 7 against `DEV_ENV_URL`, but set `PLAN_QUALITY = "degraded"` and record the cause (*"discovery ran against {DEV_ENV_NAME}, which may not be the project's environment — verify the plan's site settings / size / host before executing"*) so Phase 3 surfaces it as a prominent risk. + + > **Why this exists**: a real EDM-site run produced a valid-looking plan after PAC had silently stayed connected to a different env than the project targeted. The site-existence probe + recorded-URL comparison catch that at the earliest gate, before any discovery runs. + 7. Discover and classify site settings (if `DEV_TOKEN` is available and `websiteRecordId` is known): Use Node.js `https` module to query. **Paginate via `@odata.nextLink`** — sites with > 500 settings would otherwise silently truncate, dropping tier classifications and underreporting `plannedEnvVarCount`. Send `Prefer: odata.maxpagesize=5000` so Dataverse emits the continuation link, then loop until exhausted: @@ -203,7 +229,7 @@ Steps: --envUrl "{DEV_ENV_URL}" --websiteRecordId "{websiteRecordId}" \ --publisherPrefix "{publisherPrefix}" --siteName "{siteName}" \ {if SOLUTION_DONE: --solutionId "{solutionManifest.solution.solutionId}"} \ - --projectRoot "." \ + --projectRoot "." --siteType "{SITE_TYPE}" \ --datamodelManifest "./.datamodel-manifest.json" > ./docs/alm/alm-size-estimate.json.tmp \ && mv ./docs/alm/alm-size-estimate.json.tmp ./docs/alm/alm-size-estimate.json ``` @@ -495,7 +521,7 @@ If option 4: accept free-text description (via "Other") and build a stage list f Store stages as `PP_STAGES` (array of `{ label, envUrl, envName, type }`). Dev is always the source. -For each stage, populate `envName` from `ENV_LIST` (gathered in Phase 1 Step 5 via `pac env list --output json`). Match by URL origin (lowercase, trailing slash stripped, path/query ignored) and copy the entry's `DisplayName` (or `displayName`) into `envName`. When no match is found — usually because the user pasted a custom URL via "Other" — leave `envName` unset; the renderer falls back to showing the URL alone in the stage card. The renderer puts `envName` between the stage label and the URL (e.g. *Staging / **Supplier Portal Staging** / https://orgd6a9894f.crm5.dynamics.com/*) so reviewers recognize the env at a glance and the URL stays available as a one-click jump-to-env. Set `type: "source"` for the dev/source stage and `type: "target"` for every downstream stage so the renderer applies the active-stage styling correctly. +For each stage, populate `envName` from `ENV_LIST` (gathered in Phase 1 Step 5 via `list-environments.js`). Match by URL origin (lowercase, trailing slash stripped, path/query ignored) against each entry's `environmentUrl` and copy the entry's `displayName` into `envName`. When no match is found — usually because the user pasted a custom URL via "Other" — leave `envName` unset; the renderer falls back to showing the URL alone in the stage card. The renderer puts `envName` between the stage label and the URL (e.g. *Staging / **Supplier Portal Staging** / https://orgd6a9894f.crm5.dynamics.com/*) so reviewers recognize the env at a glance and the URL stays available as a one-click jump-to-env. Set `type: "source"` for the dev/source stage and `type: "target"` for every downstream stage so the renderer applies the active-stage styling correctly. @@ -529,7 +555,7 @@ Store the resulting `HOST_ENV_URL` for use by the rest of plan-alm. The auxiliar 2. **Fill remaining slots up to 5** from the rest of the eligible list, in the order returned by `list-tenant-envs.js` (name-hint pattern `pipeline|deploy|host|alm|cicd|govern` → admin-perms → recency). 3. **Append "Other (paste URL)"** as the last per-env entry inside option 1's nested list — escape hatch for envs that didn't make the cap. 4. When `eligible.length > 5`, suffix option 1's headline with: ` Showing top 5 of {N}; the remaining {N-5} eligible env(s) can be reached via the "Other (paste URL)" entry.` When `eligible.length <= 5`, no suffix (all envs visible inline). -5. When the user picks "Other (paste URL)", pre-fill the URL input with `ENV_LIST` (the `pac env list --output json` output gathered in Phase 1) so they can paste-or-pick from the full inventory rather than typing a URL by hand. +5. When the user picks "Other (paste URL)", pre-fill the URL input with `ENV_LIST` (the `list-environments.js` output gathered in Phase 1 — each entry's `environmentUrl`) so they can paste-or-pick from the full inventory rather than typing a URL by hand. The same cap policy applies to `ensure-pipelines-host` Phase 3.C — see that skill's Step 3a for the same rules. Keep the two implementations consistent so users see the same prompt shape regardless of whether they enter via plan-alm or directly via setup-pipeline → ensure-pipelines-host. @@ -885,7 +911,7 @@ Populate `risks` based on gathered data: - If `HOST_RESOLUTION.status === "PlatformHostExistsUnbound"`: `{ type: "info", message: "Tenant has a Platform Host. Reusing it is the lowest-friction option; creating a Custom Host instead provides better governance for separate-tenant or governed scenarios." }` - If `HOST_RESOLUTION.status === "CannotRedirect"`: `{ type: "warning", message: "CannotRedirect: source env ProjectHostEnvironmentId points at PE but tenant default custom host is set elsewhere. Resolution requires Power Platform admin." }` (Note: Phase 2 Q4 normally blocks plan generation in this state; this is a defensive entry in case the plan is somehow generated.) - **Manual path** (always, when `STRATEGY = "manual"`): `{ type: "info", message: "Recommended sequence: run /power-pages:export-solution, review the produced zip, then run /power-pages:import-solution for each target. plan-alm does not perform the export/import itself." }` -- **Raw-discovery gaps (#9)** — for each of `rawDiscovery.estimate`, `rawDiscovery.splitPlan`, and (PP path) `rawDiscovery.hostResolution` that is `null` at planData-build time: `{ type: "warning", message: "Discovery for {X} did not run; the related size/split/host decisions in this plan are unverified." }` (substitute `{X}` = "solution size estimate" / "split analysis" / "pipeline host resolution"). +- **Raw-discovery gaps (#9)** — for each of `rawDiscovery.estimate`, `rawDiscovery.splitPlan`, and (PP path) `rawDiscovery.hostResolution` that is `null` at planData-build time: `{ type: "warning", message: "Discovery for {X} did not run; the related size/split/host decisions in this plan are unverified." }` (substitute `{X}` = "solution size estimate" / "split analysis" / "pipeline host resolution"). **Carve-out for `rawDiscovery.hostResolution`:** when `PIPELINE_DONE = true`, host resolution is *intentionally* skipped (Phase 1 Step 12's skip rule — the host comes from `docs/alm/last-pipeline.json`, not a fresh probe), so a `null` `hostResolution` is expected, not a gap. **Do NOT emit the "pipeline host resolution" warning when `PIPELINE_DONE = true`** — it would be a spurious "host resolution did not run" on every project that already has a pipeline. The `estimate` and `splitPlan` gap warnings still apply regardless of `PIPELINE_DONE`. **Plan completeness check (#10).** Before writing planData, verify the plan rests on real discovery: - `sizeAnalysis.totalSizeMB` is non-null, diff --git a/plugins/power-pages/skills/setup-pipeline/SKILL.md b/plugins/power-pages/skills/setup-pipeline/SKILL.md index 26a2559d6..99a010db1 100644 --- a/plugins/power-pages/skills/setup-pipeline/SKILL.md +++ b/plugins/power-pages/skills/setup-pipeline/SKILL.md @@ -144,9 +144,9 @@ Steps: 3. Run silently: ```bash - pac env list --output json 2>/dev/null + node "${PLUGIN_ROOT}/scripts/lib/list-environments.js" ``` - Store output as `ENV_LIST`. + Store the JSON array as `ENV_LIST` (entries: `{ displayName, environmentId, environmentUrl, uniqueName, active }`). This helper parses `pac env list` — the old `pac env list --output json` is invalid on current PAC CLI (`pac env list` only accepts `--filter`). It prints `[]` and exits 0 when PAC is unauthenticated, so this step degrades gracefully. 4. **Resolve the Pipelines host via `ensure-pipelines-host-detect.js`** (the same flow `/power-pages:ensure-pipelines-host` runs internally — it reads any cached `docs/alm/last-host-check.json`, then walks the resolution order: org-setting binding → BAP env GET → tenant default custom host → tenant-wide enumeration. Read-only; never prompts the user): From 6c3a04804f86d12b32a1f14f55821b3c92f4ec40 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Mon, 22 Jun 2026 18:37:48 +0530 Subject: [PATCH 32/38] =?UTF-8?q?plan-alm:=20rename=20siteType=20data-mode?= =?UTF-8?q?l=20=E2=86=92=20declarative=20+=20final-review=20fixes?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rename (canonical, build-axis label only — nothing branches on the value, and it is NOT rendered in the plan HTML; it lives in docs/.alm-plan-data.json + the estimator's diagnostic field + agent prose): - detect-project-context.js now emits siteType 'declarative' (was 'data-model'); 'data-model' documented as the legacy alias (older plan-data stays equivalent). - estimate-solution-size.js resolveSiteType returns 'declarative' and NORMALIZES a legacy 'data-model' arg → 'declarative'; non-canonical values (e.g. an unsubstituted "{SITE_TYPE}" literal) are ignored in favor of the marker probe. - plan-alm SKILL.md SITE_TYPE prose + planData comment; AGENTS.md contract; validation-helpers / check-activation-status comments. Left the unrelated "Dataverse data model / EDM" prose in pipeline skills untouched. Tests updated. - Corrected the SKILL.md claim that siteType is "surfaced in the plan" — it isn't. Final-review fixes (from the 8-angle review pass): - plan-alm Step 4: stop telling the agent to grep "Environment URL" from `pac env who` — PAC 2.8.x prints "Org URL:"; read either label (matches the getEnvironmentUrl fix), and fall back to verify-alm-prerequisites' resolved URL. - Removed mojibake ("À-côté" → "or wrong") in the Step 6b env-match prose. - approval-gates.md §6.1 header count 16 → 17 (matches the 17 catalog rows). - list-environments.js docstring: corrected the `pac admin list` rationale (admin-scoped + different shape, not "tenant-wide admin-only"). 1281 tests pass (+2). alm-lint 0, legacy-compat in sync, version-check pass. Co-Authored-By: Claude Opus 4.8 (1M context) --- plugins/power-pages/AGENTS.md | 2 +- .../power-pages/references/approval-gates.md | 2 +- .../scripts/check-activation-status.js | 2 +- .../scripts/lib/detect-project-context.js | 18 ++++++----- .../scripts/lib/estimate-solution-size.js | 24 ++++++++++---- .../scripts/lib/list-environments.js | 8 +++-- .../scripts/lib/validation-helpers.js | 2 +- .../tests/detect-project-context.test.js | 6 ++-- .../tests/estimate-solution-size.test.js | 32 ++++++++++++++++--- plugins/power-pages/skills/plan-alm/SKILL.md | 12 +++---- 10 files changed, 72 insertions(+), 36 deletions(-) diff --git a/plugins/power-pages/AGENTS.md b/plugins/power-pages/AGENTS.md index 261b4bc40..f7fbd76d4 100644 --- a/plugins/power-pages/AGENTS.md +++ b/plugins/power-pages/AGENTS.md @@ -198,7 +198,7 @@ Shared lib modules live at `scripts/lib/` and are imported by other scripts via #### ALM Prerequisites & Context - `scripts/lib/verify-alm-prerequisites.js`: Verifies all prerequisites for ALM skills — PAC CLI installed + authenticated (`pac env who`), Azure CLI installed + logged in, Dataverse API reachable (`WhoAmI`). Args: `--envUrl` (opt, overrides env from PAC CLI), `--require-manifest` (fails if `.solution-manifest.json` not found). Output: `{ envUrl, token, userId, organizationId, tenantId }`. Exit 0 on success, exit 1 on any failure. Used by `setup-solution`, `export-solution`, `import-solution`, `setup-pipeline`, `deploy-pipeline`, `plan-alm`. -- `scripts/lib/detect-project-context.js`: Reads Power Pages project context from the project root. The `siteType` discriminator is the **build axis** — code/SPA vs declarative (design-studio) site — NOT the Dataverse data-model axis (a declarative site can be on the standard OR enhanced data model; both download to a `.powerpages-site/` tree). `siteType: "data-model"` is the (compat-named) declarative bucket; a future pass may rename it `"declarative"`. Resolves identity in order: (1) `powerpages.config.json` → `siteType: "code"` (code/SPA sites); (2) `.powerpages-site/` → `siteType: "data-model"` (declarative design-studio sites — standard or enhanced data model — which have **no** `powerpages.config.json`). The **authoritative declarative marker is the `.powerpages-site/.portalconfig/` directory** (only declarative sites have it); `website.yml` is the identity source (`id`→`websiteRecordId`, `name`→`siteName`) but is NOT a reliable declarative signal alone because **both** site types carry it. `environmentUrl: null` for declarative sites (no env URL in the local files — callers re-confirm via `pac env who`). Also reads `.solution-manifest.json` and `.datamodel-manifest.json`. Args: `--projectRoot` (opt). Output: `{ projectRoot, siteType, siteName, websiteRecordId, environmentUrl, solutionManifest, datamodelManifest }`. Exit 0 on success, exit 1 only if neither `powerpages.config.json` nor a `.powerpages-site/` (`.portalconfig/`/`website.yml`) marker is found. Note: `findProjectRoot` (in `validation-helpers.js`) likewise treats a `.powerpages-site/` directory as a project-root marker. +- `scripts/lib/detect-project-context.js`: Reads Power Pages project context from the project root. The `siteType` discriminator is the **build axis** — code/SPA vs declarative (design-studio) site — NOT the Dataverse data-model axis (a declarative site can be on the standard OR enhanced data model; both download to a `.powerpages-site/` tree). `siteType: "declarative"` is the declarative bucket (it was historically labeled `"data-model"`; that value is now the legacy alias — nothing branches on the literal, so older plan-data carrying `"data-model"` stays equivalent). Resolves identity in order: (1) `powerpages.config.json` → `siteType: "code"` (code/SPA sites); (2) `.powerpages-site/` → `siteType: "declarative"` (declarative design-studio sites — standard or enhanced data model — which have **no** `powerpages.config.json`). The **authoritative declarative marker is the `.powerpages-site/.portalconfig/` directory** (only declarative sites have it); `website.yml` is the identity source (`id`→`websiteRecordId`, `name`→`siteName`) but is NOT a reliable declarative signal alone because **both** site types carry it. `environmentUrl: null` for declarative sites (no env URL in the local files — callers re-confirm via `pac env who`). Also reads `.solution-manifest.json` and `.datamodel-manifest.json`. Args: `--projectRoot` (opt). Output: `{ projectRoot, siteType, siteName, websiteRecordId, environmentUrl, solutionManifest, datamodelManifest }`. Exit 0 on success, exit 1 only if neither `powerpages.config.json` nor a `.powerpages-site/` (`.portalconfig/`/`website.yml`) marker is found. Note: `findProjectRoot` (in `validation-helpers.js`) likewise treats a `.powerpages-site/` directory as a project-root marker. - `scripts/lib/alm-paths.js`: Single source of truth for ALM artifact paths. Exports `ALM_DIR` (always `docs/alm`), `FILE_NAMES` (frozen object mapping logical key → filename for all 14 ALM artifacts), `almDir(projectRoot) → path`, `almPath(projectRoot, key) → path`, `ensureAlmDir(projectRoot) → path` (mkdir -p idempotent). Every ALM-only state file (5 plan/decision JSONs + 9 `last-*.json` skill-run markers including `last-export.json`) writes under `/docs/alm/`. **Always resolve through this helper** — never inline a raw `docs/alm/...` path in a script. Files intentionally NOT moved here (and not in `FILE_NAMES`): `.solution-manifest.json`, `.datamodel-manifest.json`, `.alm-config.json`, `.alm-deferred`, `deployment-settings.json`. Adding a new ALM marker means adding its key + filename to `FILE_NAMES` first; `almPath` throws on unknown keys to catch typos at call-site. - `scripts/lib/check-alm-plan.js`: Phase 0 gate helper used by every ALM skill to detect (a) whether an ALM plan exists for this project, (b) whether the user has explicitly deferred ALM via the `.alm-deferred` marker, and (c) whether an existing plan is stale (the source solution was modified after the plan was generated). Args: `--projectRoot`, `--envUrl` (opt — required for staleness check), `--token` (opt), `--solutionId` (opt — required for staleness check). Output: `{ exists, deferred, deferral, planPath, htmlPath, stale, staleness: { reason, detail }, generatedAt, planStatus, solution: {...} }`. Without env/solution context the helper does an existence-only check; with them it queries Dataverse for `solutions(solutionId)?$select=modifiedon` and compares against `planData.generatedAt`. Used by `setup-solution`, `setup-pipeline`, `deploy-pipeline`, `export-solution`, `import-solution`, `configure-env-variables`, `ensure-pipelines-host`, `force-link-environment` Phase 0 gates — the "fail closed when no plan" pattern. **PLAN_STATUS lifecycle — promotes `Approved` → `In Execution`:** plan-alm is plan-only and leaves the plan `Approved`; this helper performs the `Approved` → `In Execution` transition (and writes the first heartbeat) the **first time an execution skill's Phase 0 runs** — it is the only thing that sets `In Execution`, so without it the heartbeat/active-chain machinery (multi-hour-deploy `stale-heartbeat` reclassification) never engages. Gated on heartbeat-write: read-only callers pass `--no-heartbeat` (plan-alm's own deferral check, audits, tests) and are never promoted. The terminal `In Execution` → `Completed` transition is owned by `refresh-alm-plan-data.js` (completion evaluator). - `scripts/lib/set-plan-status.js`: **The single deterministic owner of the creation-time `Draft` / `Approved` write** — the one PLAN_STATUS transition that used to be done by hand-authored `Edit`s in plan-alm Phase 4 (to the HTML spans *and* the JSON), with no helper. Because the badge + `approved-by` / `approval-date` spans are re-derived from `docs/.alm-plan-data.json` on every render, the old manual HTML Edit was non-durable (reverted on the next refresh) and a partial write left the plan "approver recorded but PLAN_STATUS=Draft" — stuck forever, since `check-alm-plan.js` only promotes from `Approved`. This helper writes `PLAN_STATUS` + `PLAN_MODE` + `APPROVED_BY` + `APPROVAL_DATE` **together** (atomic temp+rename) and optionally re-renders (reuses `refresh-alm-plan-data.js → findRendererPath`/`invokeRenderer`). Enforced invariants: only `Draft` / `Approved` are settable here (`In Execution` is owned by `check-alm-plan.js`, `Completed` by `refresh-alm-plan-data.js`); `Approved` **requires** a non-empty `--approver`; `Draft` **clears** the approver fields; a plan already `In Execution` / `Completed` is **not** re-drafted without `--force`. Args: `--projectRoot`, `--status Draft|Approved`, `--approver`, `--approvalDate` (opt — defaults to now), `--force`, `--render`, `--rendererPath` (opt). Output: `{ ok, previousStatus, status, mode, approver, approvalDate, rendered }`. Called by plan-alm Phase 4 (both save options) and the Phase 1 step-0b in-place Draft→Approved fast-path. The `validate-plan-alm.js` consistency guard blocks the two half-written states (`Draft`+approver, `Approved`+no-approver) for plans created the old way or hand-edited. diff --git a/plugins/power-pages/references/approval-gates.md b/plugins/power-pages/references/approval-gates.md index f60ea1933..b63488cfa 100644 --- a/plugins/power-pages/references/approval-gates.md +++ b/plugins/power-pages/references/approval-gates.md @@ -256,7 +256,7 @@ Each section lists every `AskUserQuestion` in that skill. Catalog rows are marke --- -### 6.1 `plan-alm` (16 calls; planner) +### 6.1 `plan-alm` (17 calls; planner) > `plan-alm` is a **planner** — it produces an approved/draft HTML plan and never executes. The execution gates that used to live in Phases 5–8 (deploy-failure, post-deploy activation, manual export/import checkpoint) now belong to the individual ALM skills the user runs afterward; they are catalogued under those skills' sections, not here. diff --git a/plugins/power-pages/scripts/check-activation-status.js b/plugins/power-pages/scripts/check-activation-status.js index 1b3b2962c..f37a74434 100644 --- a/plugins/power-pages/scripts/check-activation-status.js +++ b/plugins/power-pages/scripts/check-activation-status.js @@ -27,7 +27,7 @@ function output(obj) { // // Resolution order: // 1. powerpages.config.json (code/SPA sites) — siteName + (optional) websiteRecordId. -// 2. .powerpages-site/website.yml (declarative "data-model" sites — standard or +// 2. .powerpages-site/website.yml (declarative sites — standard or // enhanced data model — which have no powerpages.config.json) — `name` -> siteName, // `id` -> websiteRecordId. // 3. `pac pages list` — ONLY when the GUID is still unknown (e.g. a code site whose diff --git a/plugins/power-pages/scripts/lib/detect-project-context.js b/plugins/power-pages/scripts/lib/detect-project-context.js index 6c4174f41..dfca7cf40 100644 --- a/plugins/power-pages/scripts/lib/detect-project-context.js +++ b/plugins/power-pages/scripts/lib/detect-project-context.js @@ -2,20 +2,22 @@ // Reads Power Pages project context files from the project root. // Locates powerpages.config.json (code/SPA sites) OR a .powerpages-site/ config tree -// (declarative "data-model" sites — Power Pages design-studio sites), plus +// (declarative sites — Power Pages design-studio sites), plus // .solution-manifest.json and .datamodel-manifest.json. // // NOTE on terminology: the discriminator here is the BUILD axis — code/SPA site vs // declarative (design-studio) site — NOT the Dataverse data-model axis. A declarative // site can be on the standard OR the enhanced data model ("EDM"); both download to a -// .powerpages-site/ tree via `pac pages download`. siteType "data-model" names that -// declarative bucket (kept for compatibility with plan-alm); a future pass may rename -// it to "declarative". +// .powerpages-site/ tree via `pac pages download`. siteType "declarative" names that +// bucket. (It was historically labeled "data-model"; that value is now the legacy +// alias. Nothing branches on the literal — it is a diagnostic label the agent reads +// and the estimator echoes — so the rename is safe, and any plan-data written before +// the rename that still carries "data-model" remains equivalent.) // // Site identity resolution order (first match wins): // 1. powerpages.config.json -> siteType "code" (code/SPA sites; has siteName, // websiteRecordId, environmentUrl) -// 2. .powerpages-site/ (.portalconfig/ + website.yml) -> siteType "data-model" +// 2. .powerpages-site/ (.portalconfig/ + website.yml) -> siteType "declarative" // (declarative design-studio sites; standard or // enhanced data model. website.yml carries `id` and // `name` but no environment URL — callers re-confirm @@ -31,7 +33,7 @@ // Output (JSON to stdout): // { // "projectRoot": "...", -// "siteType": "code" | "data-model", +// "siteType": "code" | "declarative", // "declarative" was formerly "data-model" // "siteName": "...", // "websiteRecordId": "...", // "environmentUrl": "..." | null, @@ -146,7 +148,7 @@ function detectProjectContext(options = {}) { }; } - // 2. Declarative ("data-model") site — a Power Pages design-studio site + // 2. Declarative site (siteType "declarative", formerly "data-model") — a Power Pages design-studio site // (`pac pages download`; standard or enhanced data model), as opposed to a // code/SPA site. The authoritative positive marker is the // `.powerpages-site/.portalconfig/` directory (only declarative sites have it). @@ -167,7 +169,7 @@ function detectProjectContext(options = {}) { } return { projectRoot, - siteType: 'data-model', + siteType: 'declarative', siteName: site ? (site.name || null) : null, websiteRecordId: site ? (site.id || null) : null, environmentUrl: null, diff --git a/plugins/power-pages/scripts/lib/estimate-solution-size.js b/plugins/power-pages/scripts/lib/estimate-solution-size.js index de9ec72d9..bbb5d1a69 100644 --- a/plugins/power-pages/scripts/lib/estimate-solution-size.js +++ b/plugins/power-pages/scripts/lib/estimate-solution-size.js @@ -86,19 +86,29 @@ function parseArgs(argv) { // Phase 1 via detect-project-context.js, the authoritative source), and fall // back to a lightweight local probe of the same markers documented in CLAUDE.md: // - `powerpages.config.json` → code / SPA site -// - `.powerpages-site/.portalconfig/` → declarative design-studio (data-model/EDM) site -// Returns the canonical values ('code' | 'data-model') to match +// - `.powerpages-site/.portalconfig/` → declarative design-studio (EDM/standard) site +// Returns the canonical values ('code' | 'declarative') to match // detect-project-context.js — NOT the old hardcoded 'code-site', which mislabeled -// every EDM/data-model site as a code site. Returns 'unknown' when neither marker -// is present (e.g. running outside a project root). +// every declarative/EDM site as a code site. ('declarative' was formerly labeled +// 'data-model'; the value is diagnostic-only, so a caller still passing 'data-model' +// is echoed unchanged and remains equivalent.) Returns 'unknown' when neither +// marker is present (e.g. running outside a project root). function resolveSiteType(explicitSiteType, projectRoot) { - if (explicitSiteType) return explicitSiteType; + // Normalize the caller-supplied label. 'data-model' is the legacy alias for + // 'declarative' (back-compat). Only canonical labels are trusted verbatim; + // anything else — notably an unsubstituted "{SITE_TYPE}" template literal an + // agent forwarded without resolving it — is IGNORED in favor of the local + // marker probe, so garbage never lands in the diagnostic output. + if (explicitSiteType === 'data-model') return 'declarative'; + if (explicitSiteType === 'code' || explicitSiteType === 'declarative' || explicitSiteType === 'unknown') { + return explicitSiteType; + } if (!projectRoot) return 'unknown'; const fs = require('fs'); const path = require('path'); try { if (fs.existsSync(path.join(projectRoot, 'powerpages.config.json'))) return 'code'; - if (fs.existsSync(path.join(projectRoot, '.powerpages-site', '.portalconfig'))) return 'data-model'; + if (fs.existsSync(path.join(projectRoot, '.powerpages-site', '.portalconfig'))) return 'declarative'; } catch { // Filesystem probe is best-effort — a diagnostic label must never be fatal. } @@ -1148,7 +1158,7 @@ async function estimateSolutionSize({ envUrl, websiteRecordId, token, publisherP // scope so reviewers can spot the divergence. envVarCountTenantWide, mediaRatio: Math.round(webMeasure.mediaRatio * 100) / 100, - // Build-axis label: 'code' | 'data-model' | 'unknown' (was hardcoded + // Build-axis label: 'code' | 'declarative' | 'unknown' (was hardcoded // 'code-site', which mislabeled every declarative/EDM site). Prefers the // caller-supplied --siteType (plan-alm Phase 1), falls back to a local marker probe. siteType: resolveSiteType(siteType, projectRoot), diff --git a/plugins/power-pages/scripts/lib/list-environments.js b/plugins/power-pages/scripts/lib/list-environments.js index 8608c85f3..a46e2d9f7 100644 --- a/plugins/power-pages/scripts/lib/list-environments.js +++ b/plugins/power-pages/scripts/lib/list-environments.js @@ -11,9 +11,11 @@ // and errors with "An unknown argument --output was passed", so the JSON // pre-fill silently never worked. `pac env list` DOES emit a plain table with // an "Environment URL" column, so this helper runs the plain command and parses -// that table into JSON. (`pac admin list --json` also yields JSON but is -// admin-only and enumerates the WHOLE tenant — wrong scope for a per-user -// pre-fill — so we deliberately parse `pac env list` instead.) +// that table into JSON. (`pac admin list --json` — used by pac-bap-shim.js — +// also yields JSON, but it scopes to environments the signed-in user ADMINISTERS, +// not the maker-accessible set `pac env list` shows, and returns a different +// BAP-shaped object; `pac env list` is the right scope + shape for a maker +// pre-fill, so we parse it instead.) // // Usage: // node list-environments.js -> prints JSON array to stdout diff --git a/plugins/power-pages/scripts/lib/validation-helpers.js b/plugins/power-pages/scripts/lib/validation-helpers.js index 6c1d7905c..059b6b8b8 100644 --- a/plugins/power-pages/scripts/lib/validation-helpers.js +++ b/plugins/power-pages/scripts/lib/validation-helpers.js @@ -93,7 +93,7 @@ function findPath(dir, target) { * * A project root is marked by EITHER: * - `powerpages.config.json` — code/SPA sites (`pac pages download-code-site`), OR - * - a `.powerpages-site/` directory — declarative ("data-model") design-studio sites + * - a `.powerpages-site/` directory — declarative design-studio sites * (`pac pages download`; standard or enhanced data model). These have NO * `powerpages.config.json`. * diff --git a/plugins/power-pages/scripts/tests/detect-project-context.test.js b/plugins/power-pages/scripts/tests/detect-project-context.test.js index 2a682cb36..e7b3ce132 100644 --- a/plugins/power-pages/scripts/tests/detect-project-context.test.js +++ b/plugins/power-pages/scripts/tests/detect-project-context.test.js @@ -46,10 +46,10 @@ test('detectProjectContext: declarative (data-model) site resolves identity from ); const result = detectProjectContext({ projectRoot }); - assert.equal(result.siteType, 'data-model'); + assert.equal(result.siteType, 'declarative'); assert.equal(result.websiteRecordId, '2ecc32f6-8665-f111-a826-000d3a5a7777'); assert.equal(result.siteName, 'Application processing EDM site - permitapplication-elyyn'); - // Data-model sites carry no environment URL locally — callers re-confirm via `pac env who`. + // Declarative sites carry no environment URL locally — callers re-confirm via `pac env who`. assert.equal(result.environmentUrl, null); }); @@ -60,7 +60,7 @@ test('detectProjectContext: .powerpages-site/.portalconfig/ is the positive decl writeProjectFile(projectRoot, '.powerpages-site/.portalconfig/manifest.yml', 'foo: bar\n'); const result = detectProjectContext({ projectRoot }); - assert.equal(result.siteType, 'data-model', '.portalconfig/ marks a declarative site'); + assert.equal(result.siteType, 'declarative', '.portalconfig/ marks a declarative site'); assert.equal(result.siteName, null); assert.equal(result.websiteRecordId, null); assert.equal(result.environmentUrl, null); diff --git a/plugins/power-pages/scripts/tests/estimate-solution-size.test.js b/plugins/power-pages/scripts/tests/estimate-solution-size.test.js index c79be4718..ff022d985 100644 --- a/plugins/power-pages/scripts/tests/estimate-solution-size.test.js +++ b/plugins/power-pages/scripts/tests/estimate-solution-size.test.js @@ -794,11 +794,33 @@ test('estimateSolutionSize tableCountScope is "unavailable" with no local signal test('resolveSiteType prefers the explicit caller value (plan-alm Phase 1 detection)', () => { const { resolveSiteType } = require('../lib/estimate-solution-size'); - assert.equal(resolveSiteType('data-model', '/whatever'), 'data-model'); + assert.equal(resolveSiteType('declarative', '/whatever'), 'declarative'); assert.equal(resolveSiteType('code', null), 'code'); }); -test('resolveSiteType falls back to local markers: powerpages.config.json => code, .portalconfig => data-model', () => { +test('resolveSiteType normalizes the legacy "data-model" alias to "declarative"', () => { + const { resolveSiteType } = require('../lib/estimate-solution-size'); + assert.equal(resolveSiteType('data-model', '/whatever'), 'declarative'); +}); + +test('resolveSiteType ignores a non-canonical value (e.g. unsubstituted "{SITE_TYPE}") and probes instead', () => { + const fs = require('fs'); + const os = require('os'); + const path = require('path'); + const { resolveSiteType } = require('../lib/estimate-solution-size'); + const edmRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'est-stype-lit-')); + try { + fs.mkdirSync(path.join(edmRoot, '.powerpages-site', '.portalconfig'), { recursive: true }); + // Garbage label must NOT pass through; the marker probe wins. + assert.equal(resolveSiteType('{SITE_TYPE}', edmRoot), 'declarative'); + // No markers + garbage label → 'unknown', never the garbage. + assert.equal(resolveSiteType('{SITE_TYPE}', null), 'unknown'); + } finally { + fs.rmSync(edmRoot, { recursive: true, force: true }); + } +}); + +test('resolveSiteType falls back to local markers: powerpages.config.json => code, .portalconfig => declarative', () => { const fs = require('fs'); const os = require('os'); const path = require('path'); @@ -812,7 +834,7 @@ test('resolveSiteType falls back to local markers: powerpages.config.json => cod assert.equal(resolveSiteType(null, codeRoot), 'code'); fs.mkdirSync(path.join(edmRoot, '.powerpages-site', '.portalconfig'), { recursive: true }); - assert.equal(resolveSiteType(null, edmRoot), 'data-model', 'EDM/declarative site must NOT be mislabeled code'); + assert.equal(resolveSiteType(null, edmRoot), 'declarative', 'EDM/declarative site must NOT be mislabeled code'); // No markers and no projectRoot → 'unknown', never a wrong guess. assert.equal(resolveSiteType(null, bareRoot), 'unknown'); @@ -824,7 +846,7 @@ test('resolveSiteType falls back to local markers: powerpages.config.json => cod test('parseArgs captures --siteType', () => { const { parseArgs } = require('../lib/estimate-solution-size'); - const a = parseArgs(['node', 'x', '--siteType', 'data-model', '--envUrl', 'https://x']); - assert.equal(a.siteType, 'data-model'); + const a = parseArgs(['node', 'x', '--siteType', 'declarative', '--envUrl', 'https://x']); + assert.equal(a.siteType, 'declarative'); assert.equal(parseArgs(['node', 'x']).siteType, null); }); diff --git a/plugins/power-pages/skills/plan-alm/SKILL.md b/plugins/power-pages/skills/plan-alm/SKILL.md index b15f02601..32fac29d5 100644 --- a/plugins/power-pages/skills/plan-alm/SKILL.md +++ b/plugins/power-pages/skills/plan-alm/SKILL.md @@ -99,8 +99,8 @@ Steps: - `name` field → `siteName` (the file uses short keys; it is `name:`, not `adx_name:`) 2. **`powerpages.config.json`** (fallback — code/SPA sites only; used during plugin development from this repo root or for sites scaffolded but not yet deployed) — read `siteName` and `websiteRecordId`. - **Determine `SITE_TYPE`** (recorded in planData as `siteType`, surfaced in the plan, and used to skip SPA-only assumptions below): - - `data-model` when `.powerpages-site/.portalconfig/` exists, **or** `.powerpages-site/website.yml` resolved while no `powerpages.config.json` is present. + **Determine `SITE_TYPE`** (recorded in planData as `siteType` and used to skip SPA-only assumptions below; it is a data field in `docs/.alm-plan-data.json`, not rendered in the HTML): + - `declarative` when `.powerpages-site/.portalconfig/` exists, **or** `.powerpages-site/website.yml` resolved while no `powerpages.config.json` is present. (This value was formerly `data-model`; plans written before the rename may still carry `data-model`, which is equivalent.) - `code` when `powerpages.config.json` is present. If neither marker is found, stop with: @@ -120,7 +120,7 @@ Steps: ```bash pac env who ``` - Capture the `Environment URL` and display name. Store as `DEV_ENV_URL` and `DEV_ENV_NAME`. + Capture the environment URL and display name. Store as `DEV_ENV_URL` and `DEV_ENV_NAME`. **The URL label varies by PAC version**: current PAC (2.8.x) prints it under `Org URL:`; older builds used `Environment URL:` — read whichever is present (there is no `Environment URL:` line on 2.8.x, so do not look only for that label). The display name is the `Friendly Name:` / `Connected to...` value. If you can't parse it reliably, leave `DEV_ENV_URL` empty — Step 6's `verify-alm-prerequisites.js` resolves the authoritative URL from `pac env who` via the shared `getEnvironmentUrl()` helper (which matches both labels) and returns it as `.envUrl`. 5. Run silently: ```bash @@ -136,7 +136,7 @@ Steps: **Track plan quality.** Initialize a `PLAN_QUALITY` accumulator to `"complete"` at the start of Phase 1. If this token acquisition fails (auth error), set `DEV_TOKEN = null`, set `PLAN_QUALITY = "degraded"`, and record the cause (e.g. *"dev-environment auth failed — contents/size/host discovery skipped"*) — then continue. Contents discovery is skipped gracefully, but the resulting plan is built on partial inputs; Phase 3 surfaces this as a prominent risk so the user reviews before executing. (There is no execute path to block here — `plan-alm` only plans — but a degraded plan must be visibly flagged.) -6b. **Environment-match guard** — confirm `pac env who` points at the project's environment *before* running discovery. `DEV_ENV_URL` comes from whatever environment PAC happens to be connected to, which is **not** guaranteed to be the project's. If it isn't, every query in Steps 7–12 runs against the wrong environment and silently produces a degraded plan (zero/À-côté site settings, wrong size, wrong host) that *looks* valid. Cross-check both signals available: +6b. **Environment-match guard** — confirm `pac env who` points at the project's environment *before* running discovery. `DEV_ENV_URL` comes from whatever environment PAC happens to be connected to, which is **not** guaranteed to be the project's. If it isn't, every query in Steps 7–12 runs against the wrong environment and silently produces a degraded plan (zero or wrong site settings, wrong size, wrong host) that *looks* valid. Cross-check both signals available: 1. **Recorded-URL comparison** (no token needed): collect any environment URL the project already records — `powerpages.config.json` → `environmentUrl` (code/SPA sites; absent for declarative/EDM sites) and `.solution-manifest.json` → its `environmentUrl`/`environmentUrl`-equivalent field if present. Normalize by **origin** (lowercase host, drop trailing slash + path/query). If any recorded URL exists and its origin **differs** from `DEV_ENV_URL`'s origin → **mismatch**. 2. **Site-existence probe** (covers declarative/EDM sites that record no URL; only when `DEV_TOKEN` is available): verify the site's `websiteRecordId` actually exists in the connected env: @@ -235,7 +235,7 @@ Steps: ``` When `SOLUTION_DONE = false`, omit `--solutionId`; the estimator's output will include `envVarCountScope: "publisher-prefix"` to signal the wider scope, and the renderer surfaces this caveat in the Env Variables tab so reviewers know the number reflects the tenant view, not a specific solution. `--projectRoot "."` enables the disk cross-check — the estimator walks the local build output (`dist/`, `public-output/`, `build/`, `.output/`) and surfaces `webFilesDiskMeasuredMB`. When that number is much larger than the Dataverse-measured `webFilesAggregateMB`, the estimator flips `truncationSuspected: true` with a warning — file-typed columns whose bytes aren't returned by `$select=content` are the usual cause and the plan should trust the disk number. -> **`SITE_TYPE = "data-model"` (EDM/standard) sites have no build output**, so the disk cross-check finds no `dist/`/`build/` directory and `webFilesDiskMeasuredMB` stays `null` — this is expected, not a problem. Web files for data-model sites live as records under `.powerpages-site/web-files/` and are measured via the Dataverse query, so the size estimate is still valid; there's simply no SPA bundle on disk to cross-check against. Pass `--projectRoot "."` regardless — it's a harmless no-op for these sites. +> **`SITE_TYPE = "declarative"` (EDM/standard data-model) sites have no build output**, so the disk cross-check finds no `dist/`/`build/` directory and `webFilesDiskMeasuredMB` stays `null` — this is expected, not a problem. Web files for declarative sites live as records under `.powerpages-site/web-files/` and are measured via the Dataverse query, so the size estimate is still valid; there's simply no SPA bundle on disk to cross-check against. Pass `--projectRoot "."` regardless — it's a harmless no-op for these sites. Then run the decision tree (same tmp-file pattern): ```bash node "${PLUGIN_ROOT}/scripts/lib/compute-split-plan.js" \ @@ -651,7 +651,7 @@ Build a `planData` object with all gathered strategy inputs: ```json { "SITE_NAME": "{siteName}", - "siteType": "code | data-model", // from Phase 1 Step 1 — "data-model" for enhanced/standard data-model (EDM) sites (no SPA build output), "code" for SPA sites + "siteType": "code | declarative", // from Phase 1 Step 1 — "declarative" (formerly "data-model") for enhanced/standard data-model (EDM) design-studio sites (no SPA build output), "code" for SPA sites "GENERATED_AT": "{ISO timestamp}", "STRATEGY": "pp-pipelines | manual", "EXPORT_TYPE": "managed | unmanaged", // PP Pipelines path: always "managed" From 6367f1de858a21983de7a7f625f9b905c73791d1 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Mon, 22 Jun 2026 18:44:08 +0530 Subject: [PATCH 33/38] set-plan-status: make --render atomic across plan-data + HTML MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses #202 review: previously the new plan-data was renamed into place BEFORE the renderer ran, so a renderer failure left docs/.alm-plan-data.json updated but docs/alm-plan.html stale while the CLI exited non-zero — a caller committing docs/ would ship a new JSON beside a stale HTML. Now stage plan-data to a temp file, render FROM the temp into a temp HTML, and only swap both into place after a clean render. A render failure discards both staged files, leaving plan-data AND the HTML byte-for-byte unchanged so the caller can retry cleanly. Added a regression test (failing renderer → both files untouched, no leftover temps). Co-Authored-By: Claude Opus 4.8 (1M context) --- .../scripts/lib/set-plan-status.js | 30 +++++++++++++++---- .../scripts/tests/set-plan-status.test.js | 27 +++++++++++++++++ 2 files changed, 51 insertions(+), 6 deletions(-) diff --git a/plugins/power-pages/scripts/lib/set-plan-status.js b/plugins/power-pages/scripts/lib/set-plan-status.js index d3f5ead9f..0471ba6d9 100644 --- a/plugins/power-pages/scripts/lib/set-plan-status.js +++ b/plugins/power-pages/scripts/lib/set-plan-status.js @@ -138,18 +138,36 @@ function setPlanStatus(opts) { planData.APPROVED_BY = finalApprover; planData.APPROVAL_DATE = finalApprovalDate; - // Atomic write: temp + rename, so a crash mid-write can't truncate the plan - // file that every downstream Phase 0 gate depends on. - const tmp = dataPath + '.tmp'; - fs.writeFileSync(tmp, JSON.stringify(planData, null, 2)); - fs.renameSync(tmp, dataPath); + // Stage the new plan-data to a temp file (don't commit it yet). Atomicity + // matters two ways: (1) a crash mid-write can't truncate the plan file every + // downstream Phase 0 gate depends on; (2) when --render is requested, a renderer + // failure must leave BOTH docs/.alm-plan-data.json AND docs/alm-plan.html + // unchanged — otherwise the status write lands, the HTML stays stale, the CLI + // exits non-zero, and a caller that commits docs/ ships a new JSON beside a + // stale HTML. So we render FROM the staged temp into a temp HTML first, and only + // swap both into place after a clean render. Without --render, just commit the JSON. + const dataTmp = dataPath + '.tmp'; + fs.writeFileSync(dataTmp, JSON.stringify(planData, null, 2)); let rendered = false; if (render) { const htmlPath = planHtmlPath(projectRoot); - invokeRenderer(findRendererPath(rendererPath), dataPath, htmlPath); + const htmlTmp = htmlPath + '.tmp'; + try { + invokeRenderer(findRendererPath(rendererPath), dataTmp, htmlTmp); + } catch (e) { + // Render failed — discard both staged files so nothing changed on disk. + try { fs.unlinkSync(dataTmp); } catch {} + try { fs.unlinkSync(htmlTmp); } catch {} + throw e; + } + // Both products are ready: commit the HTML then the JSON. (Same-dir renames in + // one process; a failure between them is vanishingly unlikely and would at worst + // reproduce the pre-existing "JSON behind HTML" state, never a torn JSON file.) + fs.renameSync(htmlTmp, htmlPath); rendered = true; } + fs.renameSync(dataTmp, dataPath); return { ok: true, diff --git a/plugins/power-pages/scripts/tests/set-plan-status.test.js b/plugins/power-pages/scripts/tests/set-plan-status.test.js index df71a19fa..9baf7bec2 100644 --- a/plugins/power-pages/scripts/tests/set-plan-status.test.js +++ b/plugins/power-pages/scripts/tests/set-plan-status.test.js @@ -129,6 +129,33 @@ test('--render regenerates docs/alm-plan.html with the matching badge', (t) => { assert.match(html, /Jane/); }); +test('--render failure leaves BOTH plan-data and alm-plan.html unchanged (atomic)', (t) => { + const root = makeProject(t, { + PLAN_STATUS: 'Draft', SITE_NAME: 'DemoSite', GENERATED_AT: '2026-06-22', + }); + const dataPath = path.join(root, 'docs', '.alm-plan-data.json'); + const htmlPath = path.join(root, 'docs', 'alm-plan.html'); + const before = fs.readFileSync(dataPath, 'utf8'); + + // A renderer that always fails — simulates a missing/broken render script or an + // unexpected render error. setPlanStatus must NOT leave a half-applied state. + const badRenderer = path.join(root, 'bad-renderer.js'); + fs.writeFileSync(badRenderer, 'process.stderr.write("boom\\n"); process.exit(1);\n'); + + assert.throws(() => setPlanStatus({ + projectRoot: root, status: 'Approved', approver: 'Jane', render: true, + rendererPath: badRenderer, makeNow: () => '2026-06-22T00:00:00.000Z', + })); + + // plan-data must be byte-for-byte unchanged (still Draft, no approver) — the + // status write must not "land" while the HTML stays stale. + assert.equal(fs.readFileSync(dataPath, 'utf8'), before, 'plan-data must be untouched on render failure'); + // No HTML written, and no leftover temp files. + assert.equal(fs.existsSync(htmlPath), false, 'no alm-plan.html on render failure'); + assert.equal(fs.existsSync(dataPath + '.tmp'), false, 'no stale .alm-plan-data.json.tmp'); + assert.equal(fs.existsSync(htmlPath + '.tmp'), false, 'no stale alm-plan.html.tmp'); +}); + test('idempotent: re-writing the same status yields the same plan-data', (t) => { const root = makeProject(t, { PLAN_STATUS: 'Draft', SITE_NAME: 'T' }); setPlanStatus({ projectRoot: root, status: 'Approved', approver: 'A', makeNow: () => '2026-06-22T00:00:00.000Z' }); From aa2cfa6a8965abb56c56a8fbad1ab51b1472807e Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Mon, 22 Jun 2026 18:54:41 +0530 Subject: [PATCH 34/38] estimate-solution-size: fix resolveSiteType docstring (legacy alias is normalized, not echoed) Addresses #202 review: the comment said a caller passing the legacy 'data-model' value is "echoed unchanged", but the code normalizes 'data-model' -> 'declarative'. Corrected the docstring to say it is normalized so the output is always canonical. Co-Authored-By: Claude Opus 4.8 (1M context) --- plugins/power-pages/scripts/lib/estimate-solution-size.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/plugins/power-pages/scripts/lib/estimate-solution-size.js b/plugins/power-pages/scripts/lib/estimate-solution-size.js index bbb5d1a69..fddbf20cc 100644 --- a/plugins/power-pages/scripts/lib/estimate-solution-size.js +++ b/plugins/power-pages/scripts/lib/estimate-solution-size.js @@ -90,8 +90,8 @@ function parseArgs(argv) { // Returns the canonical values ('code' | 'declarative') to match // detect-project-context.js — NOT the old hardcoded 'code-site', which mislabeled // every declarative/EDM site as a code site. ('declarative' was formerly labeled -// 'data-model'; the value is diagnostic-only, so a caller still passing 'data-model' -// is echoed unchanged and remains equivalent.) Returns 'unknown' when neither +// 'data-model'; a caller still passing the legacy 'data-model' is NORMALIZED to +// 'declarative' so the output is always canonical.) Returns 'unknown' when neither // marker is present (e.g. running outside a project root). function resolveSiteType(explicitSiteType, projectRoot) { // Normalize the caller-supplied label. 'data-model' is the legacy alias for From 6dcf7c33f4e6c546d740cc43e7e353838003611d Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Mon, 22 Jun 2026 19:14:48 +0530 Subject: [PATCH 35/38] deploy-pipeline: pin --envUrl from project config + re-refresh plan after activation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two gaps surfaced in EDM-site deploy testing (both SKILL.md-only — the helpers already support these paths): Fix #5 (Phase 1 Step 1): verify-alm-prerequisites.js defaulted to PAC's org context (`pac env who`), which isn't guaranteed to match the project — and with a stale/ambiguous context (duplicate active pac auth profiles) it failed with a misleading "PAC CLI is not authenticated". Now resolve the project's recorded env URL first (.solution-manifest.json top-level `environmentUrl`, else powerpages.config.json `environmentUrl`) and pass it as --envUrl so the gate is deterministic; fall back to the pac-context default only when neither file records one. (verify-alm-prerequisites already skips getEnvironmentUrl() when --envUrl is supplied — no code change.) Progress-tracking row updated to match. Fix #7 (Phase 7.7): the Phase 7.5b plan refresh runs BEFORE activation is resolved, so the plan's "Activate site in {stage}" step stayed pending even after last-deploy.json recorded activationStatus. Added a second refresh-alm-plan-data.js --phase deploy-pipeline call right after the marker is patched; its existing auto-complete logic flips the activate step to completed when activationStatus === "Activated" (a deferred "Pending" correctly leaves it pending). Soft no-op when no plan exists. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../skills/deploy-pipeline/SKILL.md | 23 +++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/plugins/power-pages/skills/deploy-pipeline/SKILL.md b/plugins/power-pages/skills/deploy-pipeline/SKILL.md index ba80f76a8..baa055333 100644 --- a/plugins/power-pages/skills/deploy-pipeline/SKILL.md +++ b/plugins/power-pages/skills/deploy-pipeline/SKILL.md @@ -121,11 +121,15 @@ Tasks to create: Steps: -1. Run `verify-alm-prerequisites.js` to confirm PAC CLI auth, acquire a token, and verify API access: +1. **Resolve the project's configured environment URL first, then verify prerequisites against it.** `verify-alm-prerequisites.js` defaults to whatever environment PAC's *org context* is connected to (`pac env who`), which is **not** guaranteed to match the project — and when that context is stale/ambiguous (e.g. duplicate active `pac auth` profiles) it fails with a misleading *"PAC CLI is not authenticated"*. Pin the env explicitly so the gate is deterministic. + + Read the project's recorded env URL (first match wins): `.solution-manifest.json` → top-level `environmentUrl`, else `powerpages.config.json` → `environmentUrl`. Store as `CONFIGURED_ENV_URL`. (Both fields are top-level `environmentUrl` strings; declarative/EDM sites have no `powerpages.config.json`, so the manifest is the source there.) + ```bash - node "${PLUGIN_ROOT}/scripts/lib/verify-alm-prerequisites.js" --require-manifest + node "${PLUGIN_ROOT}/scripts/lib/verify-alm-prerequisites.js" --require-manifest --envUrl "{CONFIGURED_ENV_URL}" ``` - Capture output as JSON; extract `.envUrl` (store as `devEnvUrl`) and `.token` (store as `DEV_TOKEN`). If the script exits non-zero, stop and surface the error — it will indicate whether `az login`, `pac auth`, or WhoAmI failed. + + (If neither file records an env URL, omit `--envUrl` and fall back to the pac-context default — `verify-alm-prerequisites.js` then resolves it via `pac env who`.) Capture output as JSON; extract `.envUrl` (store as `devEnvUrl`) and `.token` (store as `DEV_TOKEN`). If the script exits non-zero, stop and surface the error — it will indicate whether `az login`, `pac auth`, or WhoAmI failed. 2. Run `detect-project-context.js` to read project config and solution manifest: ```bash @@ -1189,6 +1193,17 @@ Evaluate the result and take action based on the outcome. In all cases, **after - `"activationStatus": "{ACTIVATION_OUTCOME.status}"` (or keep `null` if `ACTIVATION_OUTCOME` is null) - `"siteUrl": "{ACTIVATION_OUTCOME.siteUrl}"` (or keep `null`) +**Re-refresh the ALM plan so the activation outcome reaches it.** The Phase 7.5b refresh ran *before* activation was resolved, so the plan's "Activate site in {stage}" step is still pending even though `last-deploy.json` now records `activationStatus`. Re-run the refresh now that the marker is patched — the `deploy-pipeline` phase auto-completes the activate step when `activationStatus` is `"Activated"` (a deferred `"Pending"` correctly leaves it pending): + +```bash +node "${PLUGIN_ROOT}/scripts/lib/refresh-alm-plan-data.js" \ + --projectRoot "." \ + --phase deploy-pipeline \ + --render +``` + +(Soft no-op when no ALM plan exists — `refresh-alm-plan-data.js` returns `ok:false` when `docs/.alm-plan-data.json` is absent.) + Then update the deploy history HTML file (in-place `Edit`) — replace `__ACTIVATION_SECTION__` with the appropriate HTML: - **`status: "Activated"`**: @@ -1284,7 +1299,7 @@ Authorization: Bearer {HOST_TOKEN} | Task subject | activeForm | Description | |---|---|---| -| Verify prerequisites | Verifying prerequisites | Run verify-alm-prerequisites.js (--require-manifest) for PAC/az/WhoAmI; run detect-project-context.js for solutionManifest/siteName; read docs/alm/last-pipeline.json for pipelineId/stages; acquire host env token | +| Verify prerequisites | Verifying prerequisites | Run verify-alm-prerequisites.js (--require-manifest --envUrl pinned from project config) for PAC/az/WhoAmI; run detect-project-context.js for solutionManifest/siteName; read docs/alm/last-pipeline.json for pipelineId/stages; acquire host env token | | Select target stage | Selecting target stage | Show available stages from docs/alm/last-pipeline.json; ask user to select target; warn if last deploy to this stage failed | | Resolve pipeline info | Resolving pipeline info | Call RetrieveDeploymentPipelineInfo (v9.1) to get SourceDeploymentEnvironmentId and DeployableArtifacts; match solution | | Validate package | Validating package | **`MULTI_RUN_MODE`**: run Phase 3.6 once (parallel batch) — `validate-stage-runs-batch.js` fans out create-stage-run + ValidatePackageAsync + poll-validation-status for all non-skipped solutions concurrently; halts the deploy on any failure or pending-approval batch; persists per-solution stageRunIds for the serial deploy loop to reuse. **Single-solution / legacy v2**: Phase 4 inline — POST deploymentstageruns (→ 201 or 204+header); POST ValidatePackageAsync top-level action (204); poll stagerunstatus until not 200000006; JSON.parse validationresults twice; fetch aigenerateddeploymentnotes; PATCH artifactversion + deploymentnotes + deploymentsettingsjson (from deployment-settings.json) | From b814c62f66e5cc6d7809d1d2ffe461c3b48b2609 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Mon, 22 Jun 2026 19:49:22 +0530 Subject: [PATCH 36/38] ALM: env-drift guard, 400-vs-404 deploy path, stage reconciliation, env-var verify match MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four gaps from a live code-site Dev→Staging run (CitizenServices portal): Gap C (priority) — env drift: ALM skills trusted the ambient PAC env when --envUrl was omitted, with no cross-check against the project's env. The Org-URL parser fix (cb68f868) made this worse — a drifted PAC context now resolves and proceeds silently instead of failing loudly. Added an opt-in `--expectedEnvUrl` to verify-alm-prerequisites.js that compares the resolved env (origin-only) and HARD-STOPS on mismatch with a "run pac env select" message. Wired deploy-pipeline Phase 1 to assert against the project's configured env URL (.solution-manifest.json / powerpages.config.json) — chosen over pinning --envUrl because the assertion also protects later PAC-CLI ops (pac pipeline deploy), not just the Dataverse calls. Documented as the recommended guard in AGENTS.md. Tests added. Gap B (doc) — deploy-pipeline Phase 3 only documented the 404 fallback for RetrieveDeploymentPipelineInfo; a 400 (observed live) was wrongly routing to the PAC-CLI path even though ValidatePackageAsync still works. Added an inline branch: non-404 errors use the marker's sourceDeploymentEnvironmentId and CONTINUE the normal ValidatePackageAsync flow (do NOT set VALIDATE_PACKAGE_UNAVAILABLE). Gap D (enhancement) — plan-alm Phase 2 Q3 now emits a soft Risks warning when a chosen target stage has no matching stage on an existing pipeline (docs/alm/last-pipeline.json), e.g. plan says "Dev→Production" but the live pipeline only has "Deploy to Staging". Informational only; no new prompt. Gap E — verify-env-var-values.js returned total:0 for a String override: NOT a type filter (it counts all EnvironmentVariables[]), but a stage-label mismatch — deploy-pipeline passes "Deploy to Staging" while the settings file is keyed "Staging", and the match was exact. Now normalized via the shared normalizeStageLabel (exported from refresh-alm-plan-data.js — the documented single source for this), so the missed code path matches like every other consumer. Test added. 1286 tests pass (+4). alm-lint 0, legacy-compat in sync, version-check pass. Co-Authored-By: Claude Opus 4.8 (1M context) --- plugins/power-pages/AGENTS.md | 2 +- .../scripts/lib/refresh-alm-plan-data.js | 4 ++ .../scripts/lib/verify-alm-prerequisites.js | 47 ++++++++++++++++--- .../scripts/lib/verify-env-var-values.js | 18 +++++-- .../tests/verify-alm-prerequisites.test.js | 42 +++++++++++++++++ .../tests/verify-env-var-values.test.js | 25 ++++++++++ .../skills/deploy-pipeline/SKILL.md | 10 ++-- plugins/power-pages/skills/plan-alm/SKILL.md | 3 ++ 8 files changed, 136 insertions(+), 15 deletions(-) diff --git a/plugins/power-pages/AGENTS.md b/plugins/power-pages/AGENTS.md index f7fbd76d4..64776b302 100644 --- a/plugins/power-pages/AGENTS.md +++ b/plugins/power-pages/AGENTS.md @@ -197,7 +197,7 @@ Shared lib modules live at `scripts/lib/` and are imported by other scripts via #### ALM Prerequisites & Context -- `scripts/lib/verify-alm-prerequisites.js`: Verifies all prerequisites for ALM skills — PAC CLI installed + authenticated (`pac env who`), Azure CLI installed + logged in, Dataverse API reachable (`WhoAmI`). Args: `--envUrl` (opt, overrides env from PAC CLI), `--require-manifest` (fails if `.solution-manifest.json` not found). Output: `{ envUrl, token, userId, organizationId, tenantId }`. Exit 0 on success, exit 1 on any failure. Used by `setup-solution`, `export-solution`, `import-solution`, `setup-pipeline`, `deploy-pipeline`, `plan-alm`. +- `scripts/lib/verify-alm-prerequisites.js`: Verifies all prerequisites for ALM skills — PAC CLI installed + authenticated (`pac env who`), Azure CLI installed + logged in, Dataverse API reachable (`WhoAmI`). Args: `--envUrl` (opt, overrides env from PAC CLI), `--require-manifest` (fails if `.solution-manifest.json` not found), `--expectedEnvUrl` (opt — **env-drift guard**: assert the resolved env matches this origin and HARD-STOP on mismatch). Output: `{ envUrl, token, userId, organizationId, tenantId }`. Exit 0 on success, exit 1 on any failure. **`--expectedEnvUrl` is the recommended guard for any ALM skill that runs against the project's source/dev env**: since `getEnvironmentUrl()` now parses PAC 2.8.x's `Org URL:` successfully, a drifted PAC context resolves silently instead of failing loudly (the old parse-miss had been an accidental safety net), so an ALM op could target the wrong environment (e.g. PROD). Skills pass the project's env URL (from `.solution-manifest.json` top-level `environmentUrl` / `powerpages.config.json` `environmentUrl` / the approved plan's source env) so a mismatch stops the run before any token/write. Prefer this over pinning `--envUrl`, which only redirects the Dataverse-API calls while later PAC-CLI ops (`pac pipeline deploy`, `pac env select`) still follow the ambient context. Used by `setup-solution`, `export-solution`, `import-solution`, `setup-pipeline`, `deploy-pipeline`, `plan-alm`. - `scripts/lib/detect-project-context.js`: Reads Power Pages project context from the project root. The `siteType` discriminator is the **build axis** — code/SPA vs declarative (design-studio) site — NOT the Dataverse data-model axis (a declarative site can be on the standard OR enhanced data model; both download to a `.powerpages-site/` tree). `siteType: "declarative"` is the declarative bucket (it was historically labeled `"data-model"`; that value is now the legacy alias — nothing branches on the literal, so older plan-data carrying `"data-model"` stays equivalent). Resolves identity in order: (1) `powerpages.config.json` → `siteType: "code"` (code/SPA sites); (2) `.powerpages-site/` → `siteType: "declarative"` (declarative design-studio sites — standard or enhanced data model — which have **no** `powerpages.config.json`). The **authoritative declarative marker is the `.powerpages-site/.portalconfig/` directory** (only declarative sites have it); `website.yml` is the identity source (`id`→`websiteRecordId`, `name`→`siteName`) but is NOT a reliable declarative signal alone because **both** site types carry it. `environmentUrl: null` for declarative sites (no env URL in the local files — callers re-confirm via `pac env who`). Also reads `.solution-manifest.json` and `.datamodel-manifest.json`. Args: `--projectRoot` (opt). Output: `{ projectRoot, siteType, siteName, websiteRecordId, environmentUrl, solutionManifest, datamodelManifest }`. Exit 0 on success, exit 1 only if neither `powerpages.config.json` nor a `.powerpages-site/` (`.portalconfig/`/`website.yml`) marker is found. Note: `findProjectRoot` (in `validation-helpers.js`) likewise treats a `.powerpages-site/` directory as a project-root marker. - `scripts/lib/alm-paths.js`: Single source of truth for ALM artifact paths. Exports `ALM_DIR` (always `docs/alm`), `FILE_NAMES` (frozen object mapping logical key → filename for all 14 ALM artifacts), `almDir(projectRoot) → path`, `almPath(projectRoot, key) → path`, `ensureAlmDir(projectRoot) → path` (mkdir -p idempotent). Every ALM-only state file (5 plan/decision JSONs + 9 `last-*.json` skill-run markers including `last-export.json`) writes under `/docs/alm/`. **Always resolve through this helper** — never inline a raw `docs/alm/...` path in a script. Files intentionally NOT moved here (and not in `FILE_NAMES`): `.solution-manifest.json`, `.datamodel-manifest.json`, `.alm-config.json`, `.alm-deferred`, `deployment-settings.json`. Adding a new ALM marker means adding its key + filename to `FILE_NAMES` first; `almPath` throws on unknown keys to catch typos at call-site. - `scripts/lib/check-alm-plan.js`: Phase 0 gate helper used by every ALM skill to detect (a) whether an ALM plan exists for this project, (b) whether the user has explicitly deferred ALM via the `.alm-deferred` marker, and (c) whether an existing plan is stale (the source solution was modified after the plan was generated). Args: `--projectRoot`, `--envUrl` (opt — required for staleness check), `--token` (opt), `--solutionId` (opt — required for staleness check). Output: `{ exists, deferred, deferral, planPath, htmlPath, stale, staleness: { reason, detail }, generatedAt, planStatus, solution: {...} }`. Without env/solution context the helper does an existence-only check; with them it queries Dataverse for `solutions(solutionId)?$select=modifiedon` and compares against `planData.generatedAt`. Used by `setup-solution`, `setup-pipeline`, `deploy-pipeline`, `export-solution`, `import-solution`, `configure-env-variables`, `ensure-pipelines-host`, `force-link-environment` Phase 0 gates — the "fail closed when no plan" pattern. **PLAN_STATUS lifecycle — promotes `Approved` → `In Execution`:** plan-alm is plan-only and leaves the plan `Approved`; this helper performs the `Approved` → `In Execution` transition (and writes the first heartbeat) the **first time an execution skill's Phase 0 runs** — it is the only thing that sets `In Execution`, so without it the heartbeat/active-chain machinery (multi-hour-deploy `stale-heartbeat` reclassification) never engages. Gated on heartbeat-write: read-only callers pass `--no-heartbeat` (plan-alm's own deferral check, audits, tests) and are never promoted. The terminal `In Execution` → `Completed` transition is owned by `refresh-alm-plan-data.js` (completion evaluator). diff --git a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js index 58d1b07f7..49fdf36b0 100644 --- a/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js +++ b/plugins/power-pages/scripts/lib/refresh-alm-plan-data.js @@ -1217,4 +1217,8 @@ module.exports = { // the "where is render-alm-plan.js / how is it invoked" knowledge in one place. findRendererPath, invokeRenderer, + // Single source of the "Deploy to {label}" → "{label}" normalization so every + // stage consumer (verify-env-var-values.js included) matches stage labels the + // same way — prevents the mismatch recurring in one un-normalized code path. + normalizeStageLabel, }; diff --git a/plugins/power-pages/scripts/lib/verify-alm-prerequisites.js b/plugins/power-pages/scripts/lib/verify-alm-prerequisites.js index 60a60cd1f..43e650669 100644 --- a/plugins/power-pages/scripts/lib/verify-alm-prerequisites.js +++ b/plugins/power-pages/scripts/lib/verify-alm-prerequisites.js @@ -5,11 +5,14 @@ // 2. Azure CLI is installed and logged in (az account get-access-token) // 3. Dataverse API is reachable (WhoAmI) // -// Usage: node verify-alm-prerequisites.js [--envUrl ] [--require-manifest] +// Usage: node verify-alm-prerequisites.js [--envUrl ] [--require-manifest] [--expectedEnvUrl ] // // Options: // --envUrl Override environment URL (default: read from pac env who) // --require-manifest Fail if .solution-manifest.json is not found in project root +// --expectedEnvUrl Assert the resolved env matches this origin; HARD-STOP on +// mismatch (guards against an ambient PAC context drifting to +// the wrong environment). Compared origin-only. No-op if unset. // // Output (JSON to stdout): // { "envUrl": "...", "token": "...", "userId": "...", "organizationId": "...", "tenantId": "..." } @@ -28,16 +31,29 @@ function parseArgs(argv) { const args = argv.slice(2); let envUrl = null; let requireManifest = false; + let expectedEnvUrl = null; for (let i = 0; i < args.length; i++) { if (args[i] === '--envUrl' && args[i + 1]) envUrl = args[++i]; else if (args[i] === '--require-manifest') requireManifest = true; + else if (args[i] === '--expectedEnvUrl' && args[i + 1]) expectedEnvUrl = args[++i]; } - return { envUrl, requireManifest }; + return { envUrl, requireManifest, expectedEnvUrl }; } -async function verifyAlmPrerequisites({ envUrl, requireManifest } = {}) { +// Compare two Dataverse env URLs by origin only (scheme+host), case-insensitive, +// path/query/trailing-slash ignored — so `https://Org.crm.dynamics.com/` and +// `https://org.crm.dynamics.com/api/data/v9.2` count as the same environment. +function sameEnvOrigin(a, b) { + const origin = (u) => { + try { return new URL(u).origin.toLowerCase(); } + catch { return String(u || '').replace(/\/+$/, '').toLowerCase(); } + }; + return origin(a) === origin(b); +} + +async function verifyAlmPrerequisites({ envUrl, requireManifest, expectedEnvUrl } = {}) { // Step 1: PAC CLI check let resolvedEnvUrl = envUrl; if (!resolvedEnvUrl) { @@ -50,6 +66,25 @@ async function verifyAlmPrerequisites({ envUrl, requireManifest } = {}) { } resolvedEnvUrl = resolvedEnvUrl.replace(/\/+$/, ''); + // Step 1b: Environment-match assertion (opt-in via --expectedEnvUrl). When the + // env is resolved from the ambient PAC context (no explicit --envUrl), it is NOT + // guaranteed to be the project's environment — and since getEnvironmentUrl() now + // parses PAC 2.8.x's "Org URL:" successfully, a DRIFTED PAC context resolves and + // proceeds SILENTLY instead of failing loudly the way the old parse-miss did + // (which had been an accidental safety net). A caller that knows the project's + // env (from .solution-manifest.json / powerpages.config.json / the approved plan) + // passes it here; a mismatch HARD-STOPS before any token acquisition or write, so + // an ALM operation can never silently target the wrong environment (e.g. PROD). + if (expectedEnvUrl && !sameEnvOrigin(resolvedEnvUrl, expectedEnvUrl)) { + throw new Error( + `Environment mismatch: PAC CLI is connected to ${resolvedEnvUrl} but this project targets ` + + `${expectedEnvUrl.replace(/\/+$/, '')}. Run \`pac env select --environment ${expectedEnvUrl.replace(/\/+$/, '')}\` ` + + '(or `pac auth select` to the right profile) and retry. If `pac env who` keeps reverting to a ' + + 'different environment, an external process is changing the active env — resolve that before ' + + 'running ALM skills, or pass --envUrl to pin this run explicitly.' + ); + } + // Step 2: Azure CLI token const token = helpers.getAuthToken(resolvedEnvUrl); if (!token) { @@ -109,9 +144,9 @@ async function verifyAlmPrerequisites({ envUrl, requireManifest } = {}) { // CLI entry point if (require.main === module) { - const { envUrl, requireManifest } = parseArgs(process.argv); + const { envUrl, requireManifest, expectedEnvUrl } = parseArgs(process.argv); - verifyAlmPrerequisites({ envUrl, requireManifest }) + verifyAlmPrerequisites({ envUrl, requireManifest, expectedEnvUrl }) .then((result) => { console.log(JSON.stringify(result)); process.exit(0); @@ -122,4 +157,4 @@ if (require.main === module) { }); } -module.exports = { verifyAlmPrerequisites }; +module.exports = { verifyAlmPrerequisites, parseArgs, sameEnvOrigin }; diff --git a/plugins/power-pages/scripts/lib/verify-env-var-values.js b/plugins/power-pages/scripts/lib/verify-env-var-values.js index d597ff684..7d1e8bf06 100644 --- a/plugins/power-pages/scripts/lib/verify-env-var-values.js +++ b/plugins/power-pages/scripts/lib/verify-env-var-values.js @@ -64,6 +64,14 @@ const fs = require('fs'); const helpers = require('./validation-helpers'); const { getAuthToken } = helpers; +// Reuse the single "Deploy to {label}" → "{label}" normalizer so this helper +// matches the caller's --stageLabel against deployment-settings.json stage keys +// the SAME way the rest of the ALM stage consumers do. Without it, deploy-pipeline +// passing `--stageLabel "Deploy to Staging"` (the pipeline stage name) failed to +// match a settings file keyed by "Staging" → readSchemaNamesFromSettings returned +// [] → the whole verify reported total:0 (a silent no-op that gave false +// reassurance the overrides had landed). +const { normalizeStageLabel } = require('./refresh-alm-plan-data'); function parseArgs(argv) { const args = argv.slice(2); @@ -125,7 +133,9 @@ function readSettingsFile(filePath, stageLabel, options = {}) { throw new Error(`--settingsFile ${filePath} is not valid JSON: ${err.message}`); } - const lowerLabel = stageLabel ? stageLabel.toLowerCase() : null; + // Normalize the requested label ("Deploy to Staging" → "staging") so it lines up + // with however the settings file names its stages — see the require comment above. + const lowerLabel = stageLabel ? normalizeStageLabel(stageLabel).toLowerCase() : null; // Shape 2: per-stage array (`Stages: []`) if (Array.isArray(parsed.Stages)) { @@ -145,7 +155,7 @@ function readSettingsFile(filePath, stageLabel, options = {}) { return preserveAllStages ? all : dedupeBySchemaName(all); } const stage = parsed.Stages.find( - (s) => (s.Name || '').toLowerCase() === lowerLabel + (s) => normalizeStageLabel(s.Name || '').toLowerCase() === lowerLabel ); if (!stage || !Array.isArray(stage.EnvironmentVariables)) return []; return stage.EnvironmentVariables.map((ev) => ({ @@ -184,9 +194,9 @@ function readSettingsFile(filePath, stageLabel, options = {}) { } return preserveAllStages ? all : dedupeBySchemaName(all); } - // Case-insensitive key match + // Case-insensitive key match (normalized so "Deploy to Staging" ↔ "Staging") const matchKey = Object.keys(stagesObj).find( - (k) => k.toLowerCase() === lowerLabel + (k) => normalizeStageLabel(k).toLowerCase() === lowerLabel ); if (!matchKey) return []; const stage = stagesObj[matchKey]; diff --git a/plugins/power-pages/scripts/tests/verify-alm-prerequisites.test.js b/plugins/power-pages/scripts/tests/verify-alm-prerequisites.test.js index ab88dced4..fad296a0d 100644 --- a/plugins/power-pages/scripts/tests/verify-alm-prerequisites.test.js +++ b/plugins/power-pages/scripts/tests/verify-alm-prerequisites.test.js @@ -79,3 +79,45 @@ test('verifyAlmPrerequisites returns envUrl, userId, organizationId on success', assert.equal(result.organizationId, 'org-1'); assert.ok(result.token); }); + +// --- --expectedEnvUrl: hard-stop on ambient PAC env drift (Gap C) --- + +test('verifyAlmPrerequisites HARD-STOPS when resolved PAC env != expectedEnvUrl', async (t) => { + const helpers = require('../lib/validation-helpers'); + const origEnv = helpers.getEnvironmentUrl; + // PAC drifted to prod; the project targets dev. + helpers.getEnvironmentUrl = () => 'https://org-prod.crm.dynamics.com'; + t.after(() => { helpers.getEnvironmentUrl = origEnv; }); + + await assert.rejects( + () => verifyAlmPrerequisites({ expectedEnvUrl: 'https://org-dev.crm.dynamics.com' }), + /Environment mismatch.*org-prod.*targets.*org-dev/s, + ); +}); + +test('verifyAlmPrerequisites passes the env assertion when origins match (slash/case/path ignored)', async (t) => { + const helpers = require('../lib/validation-helpers'); + const origEnv = helpers.getEnvironmentUrl; + const origToken = helpers.getAuthToken; + const origReq = helpers.makeRequest; + helpers.getEnvironmentUrl = () => 'https://Org-Dev.crm.dynamics.com'; // mixed case, no trailing slash + helpers.getAuthToken = () => 'tok'; + helpers.makeRequest = async () => ({ statusCode: 200, body: JSON.stringify({ UserId: 'u', OrganizationId: 'o' }) }); + t.after(() => { + helpers.getEnvironmentUrl = origEnv; + helpers.getAuthToken = origToken; + helpers.makeRequest = origReq; + }); + + // Expected URL differs only by case + trailing slash → same origin → no throw. + const res = await verifyAlmPrerequisites({ expectedEnvUrl: 'https://org-dev.crm.dynamics.com/' }); + assert.equal(res.envUrl, 'https://Org-Dev.crm.dynamics.com'); +}); + +test('parseArgs captures --expectedEnvUrl', () => { + const { parseArgs } = require('../lib/verify-alm-prerequisites'); + const a = parseArgs(['node', 'x', '--expectedEnvUrl', 'https://dev.crm.dynamics.com', '--require-manifest']); + assert.equal(a.expectedEnvUrl, 'https://dev.crm.dynamics.com'); + assert.equal(a.requireManifest, true); + assert.equal(parseArgs(['node', 'x']).expectedEnvUrl, null); +}); diff --git a/plugins/power-pages/scripts/tests/verify-env-var-values.test.js b/plugins/power-pages/scripts/tests/verify-env-var-values.test.js index a4330473a..00ef64c3d 100644 --- a/plugins/power-pages/scripts/tests/verify-env-var-values.test.js +++ b/plugins/power-pages/scripts/tests/verify-env-var-values.test.js @@ -133,6 +133,31 @@ test('readSettingsFile filters Stages[] by stageLabel (case-insensitive)', async assert.equal(prodEntries[1].value, 'prod-b'); }); +test('readSettingsFile matches a "Deploy to {label}" pipeline stage name against a "{label}"-keyed file (Gap E: total:0 bug)', async (t) => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'verify-env-norm-')); + t.after(() => fs.rmSync(dir, { recursive: true, force: true })); + + // Stages[] array shape, keyed "Staging"; deploy-pipeline passes the pipeline + // stage NAME "Deploy to Staging" — must still resolve (was returning [] → total:0, + // a silent no-op that falsely reassured the override had landed). + const arrFile = path.join(dir, 'arr.json'); + fs.writeFileSync(arrFile, JSON.stringify({ + Stages: [{ Name: 'Staging', EnvironmentVariables: [{ SchemaName: 'c311_feature_label', Value: 'staging-val' }] }], + })); + assert.deepEqual(readSettingsFile(arrFile, 'Deploy to Staging'), [ + { schemaName: 'c311_feature_label', value: 'staging-val', stageLabel: 'Staging' }, + ]); + + // Keyed-object shape (configure-env-variables / what Pipelines accepts) — same fix. + const objFile = path.join(dir, 'obj.json'); + fs.writeFileSync(objFile, JSON.stringify({ + stages: { Staging: { EnvironmentVariables: [{ SchemaName: 'c311_feature_label', Value: 'staging-val' }] } }, + })); + const entries = readSettingsFile(objFile, 'Deploy to Staging'); + assert.equal(entries.length, 1, 'String-type override must be counted, not dropped to total:0'); + assert.equal(entries[0].schemaName, 'c311_feature_label'); +}); + test('readSettingsFile with no stageLabel flattens Stages[]', async (t) => { const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'verify-env-')); t.after(() => fs.rmSync(dir, { recursive: true, force: true })); diff --git a/plugins/power-pages/skills/deploy-pipeline/SKILL.md b/plugins/power-pages/skills/deploy-pipeline/SKILL.md index baa055333..3fdb0e165 100644 --- a/plugins/power-pages/skills/deploy-pipeline/SKILL.md +++ b/plugins/power-pages/skills/deploy-pipeline/SKILL.md @@ -121,15 +121,15 @@ Tasks to create: Steps: -1. **Resolve the project's configured environment URL first, then verify prerequisites against it.** `verify-alm-prerequisites.js` defaults to whatever environment PAC's *org context* is connected to (`pac env who`), which is **not** guaranteed to match the project — and when that context is stale/ambiguous (e.g. duplicate active `pac auth` profiles) it fails with a misleading *"PAC CLI is not authenticated"*. Pin the env explicitly so the gate is deterministic. +1. **Resolve the project's configured environment URL first, then assert PAC is actually connected to it.** `verify-alm-prerequisites.js` resolves the env from PAC's *ambient org context* (`pac env who`), which is **not** guaranteed to match the project. If PAC has drifted to another environment (duplicate active `pac auth` profiles, or an external process flipping the active env), the skill would silently run discovery — and later `pac pipeline deploy` — against the **wrong** environment (potentially PROD). Assert the match and **hard-stop** on mismatch rather than pinning `--envUrl`: pinning would correct only the Dataverse-API calls while later PAC-CLI operations still follow the drifted context, so asserting that PAC itself is on the right env is the safer gate. Read the project's recorded env URL (first match wins): `.solution-manifest.json` → top-level `environmentUrl`, else `powerpages.config.json` → `environmentUrl`. Store as `CONFIGURED_ENV_URL`. (Both fields are top-level `environmentUrl` strings; declarative/EDM sites have no `powerpages.config.json`, so the manifest is the source there.) ```bash - node "${PLUGIN_ROOT}/scripts/lib/verify-alm-prerequisites.js" --require-manifest --envUrl "{CONFIGURED_ENV_URL}" + node "${PLUGIN_ROOT}/scripts/lib/verify-alm-prerequisites.js" --require-manifest --expectedEnvUrl "{CONFIGURED_ENV_URL}" ``` - (If neither file records an env URL, omit `--envUrl` and fall back to the pac-context default — `verify-alm-prerequisites.js` then resolves it via `pac env who`.) Capture output as JSON; extract `.envUrl` (store as `devEnvUrl`) and `.token` (store as `DEV_TOKEN`). If the script exits non-zero, stop and surface the error — it will indicate whether `az login`, `pac auth`, or WhoAmI failed. + `--expectedEnvUrl` makes the helper compare PAC's resolved env (origin-only) against `CONFIGURED_ENV_URL` and exit non-zero with an *"Environment mismatch: PAC CLI is connected to {X} but this project targets {Y} — run `pac env select …`"* error on mismatch. (If neither file records an env URL, omit `--expectedEnvUrl`; the helper falls back to the pac-context default with no assertion.) Capture output as JSON; extract `.envUrl` (store as `devEnvUrl`) and `.token` (store as `DEV_TOKEN`). If the script exits non-zero, stop and surface the error — it indicates an env mismatch, or that `az login` / `pac auth` / WhoAmI failed. 2. Run `detect-project-context.js` to read project config and solution manifest: ```bash @@ -292,6 +292,8 @@ Use `solutionId` from `.solution-manifest.json` as `ARTIFACT_SOLUTION_ID` and `u > GET {hostEnvUrl}/api/data/v9.1/deploymentpipelines({pipelineId})/deploymentpipeline_deploymentenvironment?$select=deploymentenvironmentid,name,environmenttype > ``` > Filter for `environmenttype = 200000000` to get the source record. Use `deploymentenvironmentid` as the `sourceDeploymentEnvironmentId`. For the artifact/solution list, use `sourceDeploymentEnvironmentId` from `docs/alm/last-pipeline.json` and `solutionName` from `.solution-manifest.json` as fallbacks. Set a flag `VALIDATE_PACKAGE_UNAVAILABLE = true` to skip Phase 4.2–4.3 and use the PAC CLI path in Phase 6. +> +> **If `RetrieveDeploymentPipelineInfo` returns a NON-404 error (e.g. 400/4xx/5xx)** — observed: some Pipelines packages return **400** for this call even though `ValidatePackageAsync` works fine — do **NOT** set `VALIDATE_PACKAGE_UNAVAILABLE`. The 404 branch above is specifically for older packages that lack the OData validation API; a 400 is just this metadata call failing, not the validation API being absent. Instead, fall back to `sourceDeploymentEnvironmentId` from `docs/alm/last-pipeline.json` (and `solutionName` from `.solution-manifest.json`) and **continue the normal `ValidatePackageAsync` flow** (Phase 4 onward). Only a genuine 404 — or a later `ValidatePackageAsync` 404 (Phase 4.2) — routes to the PAC-CLI path. ### Phase 3.5 — Pre-deploy Completeness Check @@ -1299,7 +1301,7 @@ Authorization: Bearer {HOST_TOKEN} | Task subject | activeForm | Description | |---|---|---| -| Verify prerequisites | Verifying prerequisites | Run verify-alm-prerequisites.js (--require-manifest --envUrl pinned from project config) for PAC/az/WhoAmI; run detect-project-context.js for solutionManifest/siteName; read docs/alm/last-pipeline.json for pipelineId/stages; acquire host env token | +| Verify prerequisites | Verifying prerequisites | Run verify-alm-prerequisites.js (--require-manifest --expectedEnvUrl from project config; hard-stops on PAC env drift) for PAC/az/WhoAmI; run detect-project-context.js for solutionManifest/siteName; read docs/alm/last-pipeline.json for pipelineId/stages; acquire host env token | | Select target stage | Selecting target stage | Show available stages from docs/alm/last-pipeline.json; ask user to select target; warn if last deploy to this stage failed | | Resolve pipeline info | Resolving pipeline info | Call RetrieveDeploymentPipelineInfo (v9.1) to get SourceDeploymentEnvironmentId and DeployableArtifacts; match solution | | Validate package | Validating package | **`MULTI_RUN_MODE`**: run Phase 3.6 once (parallel batch) — `validate-stage-runs-batch.js` fans out create-stage-run + ValidatePackageAsync + poll-validation-status for all non-skipped solutions concurrently; halts the deploy on any failure or pending-approval batch; persists per-solution stageRunIds for the serial deploy loop to reuse. **Single-solution / legacy v2**: Phase 4 inline — POST deploymentstageruns (→ 201 or 204+header); POST ValidatePackageAsync top-level action (204); poll stagerunstatus until not 200000006; JSON.parse validationresults twice; fetch aigenerateddeploymentnotes; PATCH artifactversion + deploymentnotes + deploymentsettingsjson (from deployment-settings.json) | diff --git a/plugins/power-pages/skills/plan-alm/SKILL.md b/plugins/power-pages/skills/plan-alm/SKILL.md index 32fac29d5..a9de24786 100644 --- a/plugins/power-pages/skills/plan-alm/SKILL.md +++ b/plugins/power-pages/skills/plan-alm/SKILL.md @@ -523,6 +523,9 @@ Store stages as `PP_STAGES` (array of `{ label, envUrl, envName, type }`). Dev i For each stage, populate `envName` from `ENV_LIST` (gathered in Phase 1 Step 5 via `list-environments.js`). Match by URL origin (lowercase, trailing slash stripped, path/query ignored) against each entry's `environmentUrl` and copy the entry's `displayName` into `envName`. When no match is found — usually because the user pasted a custom URL via "Other" — leave `envName` unset; the renderer falls back to showing the URL alone in the stage card. The renderer puts `envName` between the stage label and the URL (e.g. *Staging / **Supplier Portal Staging** / https://orgd6a9894f.crm5.dynamics.com/*) so reviewers recognize the env at a glance and the URL stays available as a one-click jump-to-env. Set `type: "source"` for the dev/source stage and `type: "target"` for every downstream stage so the renderer applies the active-stage styling correctly. + +**Reconcile the chosen stages against an existing pipeline (soft warning, PP path).** When `PIPELINE_DONE = true` (a `docs/alm/last-pipeline.json` exists), compare each chosen **target** stage in `PP_STAGES` against that file's `stages[]` (match by `targetEnvironmentUrl` origin, then by stage `name`). For every chosen target that has **no** matching stage on the existing pipeline, record a `{ type: "warning" }` entry for the plan's Risks section: *"Chosen target '{label}' ({envUrl}) has no matching stage on the existing pipeline '{pipelineName}' — setup-pipeline will need to add it."* This catches the real mismatch where a saved plan says *Dev → Production directly* but the live pipeline only has a single *Deploy to Staging* stage. It is **informational only** — do not block or re-prompt; the user can still approve the plan, and `setup-pipeline` reconciles the actual stages at execution time. + > 🚦 **Gate (plan · plan-alm:2.q4-host):** Host environment selection — branches on `HOST_RESOLUTION.status` and surfaces the right menu (use-detected / pick from list / NoHost host-type / Sandbox confirm / CannotRedirect block / manual paste). Drives `HOST_ENV_URL` and `WILL_PROVISION_*` flags for the rest of plan-alm and ensure-pipelines-host. Uses `AskUserQuestion` per branch. From 25a35b6cbe9a67c634bf56210074ee89e6f67089 Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Mon, 22 Jun 2026 19:51:35 +0530 Subject: [PATCH 37/38] Address #202 review: fix two inaccurate code comments - set-plan-status.js: a crash between the HTML-then-JSON renames leaves "HTML ahead of JSON" (new HTML, old JSON), not "JSON behind HTML". Reworded to the correct direction (it's the inverse of the original pre-atomic bug; benign + self-healing). - estimate-solution-size.js: the marker probe was said to be "documented in CLAUDE.md"; point at the canonical source (detect-project-context.js, also in AGENTS.md) instead. Comment-only; no behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../power-pages/scripts/lib/estimate-solution-size.js | 3 ++- plugins/power-pages/scripts/lib/set-plan-status.js | 9 ++++++--- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/plugins/power-pages/scripts/lib/estimate-solution-size.js b/plugins/power-pages/scripts/lib/estimate-solution-size.js index fddbf20cc..74e45e8ee 100644 --- a/plugins/power-pages/scripts/lib/estimate-solution-size.js +++ b/plugins/power-pages/scripts/lib/estimate-solution-size.js @@ -84,7 +84,8 @@ function parseArgs(argv) { // Resolve the build-axis site type for the estimator's diagnostic `siteType` // output field. Prefer the caller-supplied value (plan-alm resolves this in // Phase 1 via detect-project-context.js, the authoritative source), and fall -// back to a lightweight local probe of the same markers documented in CLAUDE.md: +// back to a lightweight local probe of the same markers detect-project-context.js +// resolves on (also described in the plugin's AGENTS.md "detect-project-context.js" entry): // - `powerpages.config.json` → code / SPA site // - `.powerpages-site/.portalconfig/` → declarative design-studio (EDM/standard) site // Returns the canonical values ('code' | 'declarative') to match diff --git a/plugins/power-pages/scripts/lib/set-plan-status.js b/plugins/power-pages/scripts/lib/set-plan-status.js index 0471ba6d9..7c2660917 100644 --- a/plugins/power-pages/scripts/lib/set-plan-status.js +++ b/plugins/power-pages/scripts/lib/set-plan-status.js @@ -161,9 +161,12 @@ function setPlanStatus(opts) { try { fs.unlinkSync(htmlTmp); } catch {} throw e; } - // Both products are ready: commit the HTML then the JSON. (Same-dir renames in - // one process; a failure between them is vanishingly unlikely and would at worst - // reproduce the pre-existing "JSON behind HTML" state, never a torn JSON file.) + // Both products are ready: commit the HTML then the JSON. A crash BETWEEN these + // two same-dir renames (vanishingly unlikely in one process) would leave the new + // HTML in place with the JSON still old — "HTML ahead of JSON". That's benign and + // self-healing: the next render re-derives the HTML from whatever the JSON says, + // and no file is ever torn (each rename is atomic). It is the inverse of the + // original pre-atomic bug (new JSON + stale HTML), and harmless in the same way. fs.renameSync(htmlTmp, htmlPath); rendered = true; } From df4265b4e31eff65614f6b1b0aa8a7c985f4dcac Mon Sep 17 00:00:00 2001 From: Nidhi Tyagi Date: Mon, 22 Jun 2026 20:05:21 +0530 Subject: [PATCH 38/38] Final-review fixes: harden env-match assertion + SKILL.md coherence MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From a 4-angle final review of #202: - verify-alm-prerequisites.js: sameEnvOrigin/envOrigin hardened so a bare host (no scheme) matches a scheme-prefixed URL, and an empty / unsubstituted "{CONFIGURED_ENV_URL}" / junk value returns null (indeterminate) instead of a bogus origin. The --expectedEnvUrl assertion now hard-stops ONLY on a definite mismatch (=== false), so a misrendered or schemeless value can no longer FALSE- hard-stop a legitimate deploy. Tests added (bare host, placeholder, null). - deploy-pipeline Phase 1: show both invocation forms (with / without --expectedEnvUrl) so the agent omits the flag when no env URL is recorded instead of passing an empty/placeholder value. - deploy-pipeline Phase 3: the 400-branch now documents the nav-property fallback for the rare missing-sourceDeploymentEnvironmentId case (still without setting VALIDATE_PACKAGE_UNAVAILABLE). - plan-alm env-match gate: replace the two confusing "Cancel"-prefixed options with two distinct ones ("Switch PAC env & re-run" / "Continue anyway"), both documented. - plan-alm step 6b: fix the self-referential "environmentUrl/environmentUrl- equivalent" typo → top-level `environmentUrl`. - set-plan-status.js: unlink the staged temp if the final JSON rename fails (no orphaned .alm-plan-data.json.tmp). - compute-split-plan.test.js: stale siteType fixture 'code-site' → 'code'. Pre-existing issues NOT in this PR's scope (noted for follow-up): tableCountScope 'manifest-only' mislabel when an empty table-permissions dir + no manifest; verifyOne String() value-mismatch vs missing-value categorization; parseEnvList forward-compat edges on best-effort pre-fill. 1288 tests pass (+2). alm-lint 0, legacy-compat in sync, version-check pass. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../scripts/lib/set-plan-status.js | 10 +++- .../scripts/lib/verify-alm-prerequisites.js | 50 +++++++++++++++---- .../scripts/tests/compute-split-plan.test.js | 2 +- .../tests/verify-alm-prerequisites.test.js | 33 ++++++++++++ .../skills/deploy-pipeline/SKILL.md | 10 +++- plugins/power-pages/skills/plan-alm/SKILL.md | 9 ++-- 6 files changed, 96 insertions(+), 18 deletions(-) diff --git a/plugins/power-pages/scripts/lib/set-plan-status.js b/plugins/power-pages/scripts/lib/set-plan-status.js index 7c2660917..8096dd9e9 100644 --- a/plugins/power-pages/scripts/lib/set-plan-status.js +++ b/plugins/power-pages/scripts/lib/set-plan-status.js @@ -170,7 +170,15 @@ function setPlanStatus(opts) { fs.renameSync(htmlTmp, htmlPath); rendered = true; } - fs.renameSync(dataTmp, dataPath); + // Commit the JSON. If this final rename ever fails (e.g. a transient lock on the + // plan file), unlink the staged temp so we don't leave an orphaned + // `.alm-plan-data.json.tmp` behind, then rethrow so the caller sees the failure. + try { + fs.renameSync(dataTmp, dataPath); + } catch (e) { + try { fs.unlinkSync(dataTmp); } catch {} + throw e; + } return { ok: true, diff --git a/plugins/power-pages/scripts/lib/verify-alm-prerequisites.js b/plugins/power-pages/scripts/lib/verify-alm-prerequisites.js index 43e650669..bbdfb82a5 100644 --- a/plugins/power-pages/scripts/lib/verify-alm-prerequisites.js +++ b/plugins/power-pages/scripts/lib/verify-alm-prerequisites.js @@ -42,15 +42,40 @@ function parseArgs(argv) { return { envUrl, requireManifest, expectedEnvUrl }; } -// Compare two Dataverse env URLs by origin only (scheme+host), case-insensitive, -// path/query/trailing-slash ignored — so `https://Org.crm.dynamics.com/` and -// `https://org.crm.dynamics.com/api/data/v9.2` count as the same environment. +// Normalize a Dataverse env reference to its origin (scheme+host), lowercased. +// Tolerates a missing scheme (`org.crm.dynamics.com` → `https://org.crm.dynamics.com`) +// since a hand-authored manifest may omit it. Returns null when the value is empty +// or not a parseable host — so the caller can treat "can't compare" distinctly from +// "definitely different" and avoid a false mismatch on garbage input. +function envOrigin(u) { + const s = String(u || '').trim(); + if (!s) return null; + // Try as-is, then with an https:// prefix (covers a bare host like + // `org.crm.dynamics.com`). The WHATWG URL parser is lenient and will happily + // accept `https://{CONFIGURED_ENV_URL}` as a "host", so after parsing we ALSO + // require a plausible DNS hostname (dot-separated alnum/hyphen labels). That + // rejects an unsubstituted `{PLACEHOLDER}`, `null`, or other junk (→ null) so it + // can never be compared as if it were a real environment. + for (const candidate of [s, 'https://' + s.replace(/^\/+/, '')]) { + let parsed; + try { parsed = new URL(candidate); } catch { continue; } + const host = parsed.hostname.toLowerCase(); + if (!/^[a-z0-9-]+(\.[a-z0-9-]+)+$/.test(host)) continue; + return parsed.origin.toLowerCase(); + } + return null; +} + +// Compare two env references by origin only — path/query/trailing-slash/case ignored. +// Returns true (same), false (definitely different), or null (indeterminate: one side +// isn't a parseable env URL). Callers must hard-stop ONLY on an explicit `false`, never +// on null, so a missing/placeholder/garbage value disables the assertion instead of +// blocking a legitimate run. function sameEnvOrigin(a, b) { - const origin = (u) => { - try { return new URL(u).origin.toLowerCase(); } - catch { return String(u || '').replace(/\/+$/, '').toLowerCase(); } - }; - return origin(a) === origin(b); + const oa = envOrigin(a); + const ob = envOrigin(b); + if (oa === null || ob === null) return null; + return oa === ob; } async function verifyAlmPrerequisites({ envUrl, requireManifest, expectedEnvUrl } = {}) { @@ -75,7 +100,12 @@ async function verifyAlmPrerequisites({ envUrl, requireManifest, expectedEnvUrl // env (from .solution-manifest.json / powerpages.config.json / the approved plan) // passes it here; a mismatch HARD-STOPS before any token acquisition or write, so // an ALM operation can never silently target the wrong environment (e.g. PROD). - if (expectedEnvUrl && !sameEnvOrigin(resolvedEnvUrl, expectedEnvUrl)) { + // HARD-STOP only on a DEFINITE mismatch (sameEnvOrigin === false). A null result + // means expectedEnvUrl wasn't a parseable env URL (empty, an unsubstituted + // `{PLACEHOLDER}`, junk) — in that case skip the assertion rather than block a + // legitimate run on bad input; the SKILL.md guidance is to omit the flag entirely + // when no env URL is recorded. + if (expectedEnvUrl && sameEnvOrigin(resolvedEnvUrl, expectedEnvUrl) === false) { throw new Error( `Environment mismatch: PAC CLI is connected to ${resolvedEnvUrl} but this project targets ` + `${expectedEnvUrl.replace(/\/+$/, '')}. Run \`pac env select --environment ${expectedEnvUrl.replace(/\/+$/, '')}\` ` + @@ -157,4 +187,4 @@ if (require.main === module) { }); } -module.exports = { verifyAlmPrerequisites, parseArgs, sameEnvOrigin }; +module.exports = { verifyAlmPrerequisites, parseArgs, sameEnvOrigin, envOrigin }; diff --git a/plugins/power-pages/scripts/tests/compute-split-plan.test.js b/plugins/power-pages/scripts/tests/compute-split-plan.test.js index 9000e4486..0ebfefdfa 100644 --- a/plugins/power-pages/scripts/tests/compute-split-plan.test.js +++ b/plugins/power-pages/scripts/tests/compute-split-plan.test.js @@ -39,7 +39,7 @@ function baseEstimate(overrides = {}) { botCount: 0, envVarCount: 5, mediaRatio: 0.3, - siteType: 'code-site', + siteType: 'code', tables: [], ...overrides, }; diff --git a/plugins/power-pages/scripts/tests/verify-alm-prerequisites.test.js b/plugins/power-pages/scripts/tests/verify-alm-prerequisites.test.js index fad296a0d..93c406c87 100644 --- a/plugins/power-pages/scripts/tests/verify-alm-prerequisites.test.js +++ b/plugins/power-pages/scripts/tests/verify-alm-prerequisites.test.js @@ -121,3 +121,36 @@ test('parseArgs captures --expectedEnvUrl', () => { assert.equal(a.requireManifest, true); assert.equal(parseArgs(['node', 'x']).expectedEnvUrl, null); }); + +// --- sameEnvOrigin / envOrigin robustness (no false hard-stop on garbage/bare host) --- + +test('sameEnvOrigin: true/false for parseable URLs; null (indeterminate) for unparseable', () => { + const { sameEnvOrigin } = require('../lib/verify-alm-prerequisites'); + assert.equal(sameEnvOrigin('https://a.crm.dynamics.com', 'https://A.crm.dynamics.com/api/data/v9.2'), true); + assert.equal(sameEnvOrigin('https://a.crm.dynamics.com', 'https://b.crm.dynamics.com'), false); + // Bare host (no scheme) on the expected side must still match a scheme-prefixed resolved URL. + assert.equal(sameEnvOrigin('https://dev.crm.dynamics.com', 'dev.crm.dynamics.com'), true); + // Unparseable / empty / unsubstituted placeholder → null (caller must NOT hard-stop). + assert.equal(sameEnvOrigin('https://dev.crm.dynamics.com', ''), null); + assert.equal(sameEnvOrigin('https://dev.crm.dynamics.com', '{CONFIGURED_ENV_URL}'), null); + assert.equal(sameEnvOrigin('https://dev.crm.dynamics.com', null), null); +}); + +test('verifyAlmPrerequisites does NOT hard-stop when expectedEnvUrl is an unsubstituted placeholder', async (t) => { + const helpers = require('../lib/validation-helpers'); + const origEnv = helpers.getEnvironmentUrl; + const origToken = helpers.getAuthToken; + const origReq = helpers.makeRequest; + helpers.getEnvironmentUrl = () => 'https://dev.crm.dynamics.com'; + helpers.getAuthToken = () => 'tok'; + helpers.makeRequest = async () => ({ statusCode: 200, body: JSON.stringify({ UserId: 'u', OrganizationId: 'o' }) }); + t.after(() => { + helpers.getEnvironmentUrl = origEnv; + helpers.getAuthToken = origToken; + helpers.makeRequest = origReq; + }); + // A SKILL that fails to resolve {CONFIGURED_ENV_URL} would pass the literal — must + // NOT block the run (the assertion is skipped on an unparseable expected value). + const res = await verifyAlmPrerequisites({ expectedEnvUrl: '{CONFIGURED_ENV_URL}' }); + assert.equal(res.envUrl, 'https://dev.crm.dynamics.com'); +}); diff --git a/plugins/power-pages/skills/deploy-pipeline/SKILL.md b/plugins/power-pages/skills/deploy-pipeline/SKILL.md index 3fdb0e165..dd7603e43 100644 --- a/plugins/power-pages/skills/deploy-pipeline/SKILL.md +++ b/plugins/power-pages/skills/deploy-pipeline/SKILL.md @@ -125,11 +125,17 @@ Steps: Read the project's recorded env URL (first match wins): `.solution-manifest.json` → top-level `environmentUrl`, else `powerpages.config.json` → `environmentUrl`. Store as `CONFIGURED_ENV_URL`. (Both fields are top-level `environmentUrl` strings; declarative/EDM sites have no `powerpages.config.json`, so the manifest is the source there.) + **Pass `--expectedEnvUrl` only when `CONFIGURED_ENV_URL` actually resolved to a URL.** Use the first form when a recorded env URL exists, the second when neither file records one — do **not** pass an empty or unresolved `--expectedEnvUrl "{CONFIGURED_ENV_URL}"`: + ```bash + # CONFIGURED_ENV_URL resolved (recorded in manifest/config) — assert PAC is on it: node "${PLUGIN_ROOT}/scripts/lib/verify-alm-prerequisites.js" --require-manifest --expectedEnvUrl "{CONFIGURED_ENV_URL}" + + # Neither file records an env URL — omit the flag, fall back to the pac-context default: + node "${PLUGIN_ROOT}/scripts/lib/verify-alm-prerequisites.js" --require-manifest ``` - `--expectedEnvUrl` makes the helper compare PAC's resolved env (origin-only) against `CONFIGURED_ENV_URL` and exit non-zero with an *"Environment mismatch: PAC CLI is connected to {X} but this project targets {Y} — run `pac env select …`"* error on mismatch. (If neither file records an env URL, omit `--expectedEnvUrl`; the helper falls back to the pac-context default with no assertion.) Capture output as JSON; extract `.envUrl` (store as `devEnvUrl`) and `.token` (store as `DEV_TOKEN`). If the script exits non-zero, stop and surface the error — it indicates an env mismatch, or that `az login` / `pac auth` / WhoAmI failed. + `--expectedEnvUrl` makes the helper compare PAC's resolved env (origin-only) against `CONFIGURED_ENV_URL` and exit non-zero with an *"Environment mismatch: PAC CLI is connected to {X} but this project targets {Y} — run `pac env select …`"* error on mismatch. (As a safety net the helper skips the assertion if the value isn't a parseable env URL — an empty string or an unsubstituted placeholder won't hard-stop — but prefer omitting the flag outright when there's no recorded URL.) Capture output as JSON; extract `.envUrl` (store as `devEnvUrl`) and `.token` (store as `DEV_TOKEN`). If the script exits non-zero, stop and surface the error — it indicates an env mismatch, or that `az login` / `pac auth` / WhoAmI failed. 2. Run `detect-project-context.js` to read project config and solution manifest: ```bash @@ -293,7 +299,7 @@ Use `solutionId` from `.solution-manifest.json` as `ARTIFACT_SOLUTION_ID` and `u > ``` > Filter for `environmenttype = 200000000` to get the source record. Use `deploymentenvironmentid` as the `sourceDeploymentEnvironmentId`. For the artifact/solution list, use `sourceDeploymentEnvironmentId` from `docs/alm/last-pipeline.json` and `solutionName` from `.solution-manifest.json` as fallbacks. Set a flag `VALIDATE_PACKAGE_UNAVAILABLE = true` to skip Phase 4.2–4.3 and use the PAC CLI path in Phase 6. > -> **If `RetrieveDeploymentPipelineInfo` returns a NON-404 error (e.g. 400/4xx/5xx)** — observed: some Pipelines packages return **400** for this call even though `ValidatePackageAsync` works fine — do **NOT** set `VALIDATE_PACKAGE_UNAVAILABLE`. The 404 branch above is specifically for older packages that lack the OData validation API; a 400 is just this metadata call failing, not the validation API being absent. Instead, fall back to `sourceDeploymentEnvironmentId` from `docs/alm/last-pipeline.json` (and `solutionName` from `.solution-manifest.json`) and **continue the normal `ValidatePackageAsync` flow** (Phase 4 onward). Only a genuine 404 — or a later `ValidatePackageAsync` 404 (Phase 4.2) — routes to the PAC-CLI path. +> **If `RetrieveDeploymentPipelineInfo` returns a NON-404 error (e.g. 400/4xx/5xx)** — observed: some Pipelines packages return **400** for this call even though `ValidatePackageAsync` works fine — do **NOT** set `VALIDATE_PACKAGE_UNAVAILABLE`. The 404 branch above is specifically for older packages that lack the OData validation API; a 400 is just this metadata call failing, not the validation API being absent. Instead, fall back to `sourceDeploymentEnvironmentId` from `docs/alm/last-pipeline.json` (and `solutionName` from `.solution-manifest.json`) and **continue the normal `ValidatePackageAsync` flow** (Phase 4 onward). If `docs/alm/last-pipeline.json` is somehow missing `sourceDeploymentEnvironmentId`, use the same `deploymentpipeline_deploymentenvironment` navigation-property GET shown in the 404 branch above to recover it (still **without** setting `VALIDATE_PACKAGE_UNAVAILABLE`). Only a genuine 404 — or a later `ValidatePackageAsync` 404 (Phase 4.2) — routes to the PAC-CLI path. ### Phase 3.5 — Pre-deploy Completeness Check diff --git a/plugins/power-pages/skills/plan-alm/SKILL.md b/plugins/power-pages/skills/plan-alm/SKILL.md index a9de24786..507186afc 100644 --- a/plugins/power-pages/skills/plan-alm/SKILL.md +++ b/plugins/power-pages/skills/plan-alm/SKILL.md @@ -138,7 +138,7 @@ Steps: 6b. **Environment-match guard** — confirm `pac env who` points at the project's environment *before* running discovery. `DEV_ENV_URL` comes from whatever environment PAC happens to be connected to, which is **not** guaranteed to be the project's. If it isn't, every query in Steps 7–12 runs against the wrong environment and silently produces a degraded plan (zero or wrong site settings, wrong size, wrong host) that *looks* valid. Cross-check both signals available: - 1. **Recorded-URL comparison** (no token needed): collect any environment URL the project already records — `powerpages.config.json` → `environmentUrl` (code/SPA sites; absent for declarative/EDM sites) and `.solution-manifest.json` → its `environmentUrl`/`environmentUrl`-equivalent field if present. Normalize by **origin** (lowercase host, drop trailing slash + path/query). If any recorded URL exists and its origin **differs** from `DEV_ENV_URL`'s origin → **mismatch**. + 1. **Recorded-URL comparison** (no token needed): collect any environment URL the project already records — `powerpages.config.json` → top-level `environmentUrl` (code/SPA sites; absent for declarative/EDM sites) and `.solution-manifest.json` → top-level `environmentUrl` if present. Normalize by **origin** (lowercase host, drop trailing slash + path/query). If any recorded URL exists and its origin **differs** from `DEV_ENV_URL`'s origin → **mismatch**. 2. **Site-existence probe** (covers declarative/EDM sites that record no URL; only when `DEV_TOKEN` is available): verify the site's `websiteRecordId` actually exists in the connected env: ``` GET {DEV_ENV_URL}/api/data/v9.2/powerpagesites({websiteRecordId})?$select=powerpagesiteid @@ -155,10 +155,11 @@ Steps: | Question | Header | Options | |---|---|---| - | PAC CLI is connected to **{DEV_ENV_NAME}** (`{DEV_ENV_URL}`), which does not match this project's configured environment ({recorded URL, or "this site was not found there"}). Discovery will run against the connected environment. How do you want to proceed? | Env Mismatch | Cancel — switch PAC env, then re-run (Recommended), Continue against {DEV_ENV_NAME} anyway, Cancel | + | PAC CLI is connected to **{DEV_ENV_NAME}** (`{DEV_ENV_URL}`), which does not match this project's configured environment ({recorded URL, or "this site was not found there"}). Discovery will run against the connected environment. How do you want to proceed? | Env Mismatch | Switch PAC env & re-run (Recommended), Continue against {DEV_ENV_NAME} anyway | - - **Cancel — switch PAC env (Recommended)**: stop the skill. Tell the user to point PAC at the right environment (`pac auth select --name ` or `pac org select --environment `) and re-run `/power-pages:plan-alm`. Nothing has been written. - - **Continue anyway**: proceed to Step 7 against `DEV_ENV_URL`, but set `PLAN_QUALITY = "degraded"` and record the cause (*"discovery ran against {DEV_ENV_NAME}, which may not be the project's environment — verify the plan's site settings / size / host before executing"*) so Phase 3 surfaces it as a prominent risk. + Exactly two outcomes (both halt-or-proceed; no separate "cancel" — "Switch & re-run" already stops the skill): + - **Switch PAC env & re-run (Recommended)**: stop the skill. Tell the user to point PAC at the right environment (`pac auth select --name ` or `pac org select --environment `) and re-run `/power-pages:plan-alm`. Nothing has been written. + - **Continue against {DEV_ENV_NAME} anyway**: proceed to Step 7 against `DEV_ENV_URL`, but set `PLAN_QUALITY = "degraded"` and record the cause (*"discovery ran against {DEV_ENV_NAME}, which may not be the project's environment — verify the plan's site settings / size / host before executing"*) so Phase 3 surfaces it as a prominent risk. > **Why this exists**: a real EDM-site run produced a valid-looking plan after PAC had silently stayed connected to a different env than the project targeted. The site-existence probe + recorded-URL comparison catch that at the earliest gate, before any discovery runs.