Skip to content

Commit 98430e9

Browse files
amitjoshi438Amit JoshiclaudeCopilotCopilot
authored
feat(telemetry): 1DS skill-run telemetry infra (#176)
* feat(telemetry) 1/3: shared library — dispatcher, consent, sync, builders (#142) * docs: add design spec for 1DS telemetry infra Design for a shared telemetry library at shared/telemetry/ consumed by the power-pages plugin first, with interactive first-run consent, strict-allowlist payloads, and fail-closed emission via the existing PreToolUse/PostToolUse:Skill hook surface. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(telemetry): add shared 1DS telemetry library Introduces the shared 1DS telemetry infrastructure used by all adopting plugins. Anonymous, default-on, env-var opt-out only (POWER_PLATFORM_SKILLS_TELEMETRY=0). No persistent consent file, no first-run prompt. Library (shared/telemetry/lib/): - events.js — type-aware allowlist builders (PowerPagesPluginEvent envelope, top-level columns, per-event field set, durationMs as int) - emit-dispatcher.js — detached child that POSTs to OneCollector with the Common Schema 4.0 envelope, exits silently on any failure - emit-spawn.js — fire-and-forget spawn helper with restricted child env - emit-from-prompt.js — orchestrator for slash-command skill_started events (reads ikey.json, calls pac auth + agent info, builds event) - pac-auth.js — shells out to `pac auth who` for orgId/tenantId - agent-info.js — detects host AI agent (Claude Code, Copilot CLI) + pac CLI version via env override → auto-detect → empty - prompt-detector.js — matches /plugin:skill from user prompt text - local-log.js — placeholder-iKey JSONL fallback at ~/.power-platform-skills/events.jsonl - with-telemetry.js — runInstrumented() wrapper for script-level signals - correlation.js, session.js, scrubber.js — utilities Config: shared/telemetry/ikey.json carries the iKey, collector URL, per-plugin event_stream_name, and a 'disabled' kill switch. The flag ships true so infrastructure can land before tenant-side annotation + Kusto provisioning is complete; a single PR flips it false later. Sync: shared/telemetry/sync-to-plugin.js copies lib/ + ikey.json into <plugin>/scripts/lib/telemetry/. Adopting plugins never hand-edit the synced copy. Tests: full node:test coverage for every module plus the dispatcher and spawn integration (109 tests, all hermetic; envelope-shape probe via POWER_PLATFORM_SKILLS_FAKE_HTTPS). Docs: rebuild design spec (2026-05-04) + plugin adoption guide (2026-04-29) under docs/superpowers/specs/. Older draft spec (2026-04-20) removed. * docs(telemetry): align README with code — camelCase fields, correct PAC posture shared/telemetry/README.md drifted from the rebuild work: - Field names listed in snake_case (plugin_name, session_id, ...) but the code uses camelCase (pluginName, sessionId, ...). Names match Kusto column names directly. - "What is NEVER sent" claimed tenant IDs are not sent — but pac-auth.js explicitly emits orgId and tenantId GUIDs from the active PAC profile. Moved them to "What is sent" with the rest of the PAC + agent fields. - Missing from "What is sent": pluginVersion, osName/osVersion (was os_family), pacCliVersion, aiAgentName, aiAgentVersion, eventInfo, errorDescription. - Added a note about errorDescription being a 500-char-truncated Error.message — callers must keep PII out of exception messages. - Documented the ikey.json "disabled" repo-side kill switch under Privacy posture. * refactor(telemetry): address review feedback (PR #142) Four fixes from PR review: 1. **iKey full form in envelope** — emit-dispatcher.js now sends the complete iKey value in the Common Schema `iKey` field (`"o:" + IKEY`) instead of truncating to just the first hyphen-separated segment. Aligns with the powerplatform-vscode pattern. Multi-cloud key restructuring deferred to a follow-up PR. 2. **Drop POWER_PLATFORM_SKILLS_BYPASS_KILL_SWITCH** — production code no longer carries a test-only escape hatch. Tests now override the ikey.json path via POWER_PLATFORM_SKILLS_IKEY_JSON (parallels the existing IKEY/COLLECTOR/CONFIG_DIR/FAKE_HTTPS env-var pattern) and supply their own `disabled: false` config when they need to exercise emission paths. emit-spawn.js forwards the new var to the detached child. 3. **Defense-in-depth FIELD_TYPES enforcement in dispatcher** — events.js now exports `pick`. The dispatcher imports both `FIELD_TYPES` and `pick`, then re-runs the allowlist filter on `event.data` before building the envelope. Any field that bypasses the builders is dropped before it reaches the wire. The three reserved meta fields (eventName, eventType, severity) survive via an explicit allow list. 4. **errorDescription emits err.code only, never err.message** — with-telemetry.js no longer reads `err.message` (which can contain file paths, GUIDs, tokens). The field now captures `err.code` — short non-PII metadata like `ENOENT` or `ERR_INVALID_ARG_TYPE`. Test "errorDescription truncated to 500 chars" replaced with one that asserts a PII-bearing message is NOT emitted anywhere when `err.code` is set. README updated accordingly. New dispatcher test: `dispatcher strips unknown fields from event.data` hand-crafts an event with filePath/stackTrace/rawPrompt/tokenValue and asserts none survive the FIELD_TYPES filter. Tests: 110/110. * refactor(telemetry): rename ikey.json key — ikey → instrumentationKey Per PR #142 comment 1: the JSON property is now spelled in its full form (`instrumentationKey`) to match the powerplatform-vscode constants pattern instead of the abbreviated `ikey`. The envelope `iKey` Common Schema field stays at the standard tenant-routing short form (`o:<32-char-tenant>`); the previous attempt to put the full hyphenated key in the envelope is reverted because the full form already lives in the `x-apikey` header. - shared/telemetry/ikey.json: `"ikey"` → `"instrumentationKey"`. - shared/telemetry/lib/emit-from-prompt.js: read `cfg.instrumentationKey` (other JSON keys — `collector_url`, `event_stream_name`, `disabled` — unchanged). - shared/telemetry/lib/emit-dispatcher.js: envelope reverted to `iKey: "o:" + IKEY.split("-")[0]`. - Test fixtures updated across emit-dispatcher / emit-spawn / emit-from-prompt tests. Multi-cloud iKey restructuring (the second half of PR #142 comment 1) remains deferred to a follow-up PR. Tests: 110/110. --------- Co-authored-by: Amit Joshi <amitjoshi@microsoft.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(power-pages): adopt 1DS telemetry — synced library, hooks, valid… (#153) * feat(power-pages): adopt 1DS telemetry — synced library, hooks, validators Wires the shared 1DS telemetry library into the power-pages plugin. Synced copy of the library lives under scripts/lib/telemetry/ and runs at user time. PreToolUse:Skill emits skill_started; PostToolUse:Skill emits skill_completed wrapped around the existing validator chain. Plugin-side additions: - scripts/lib/telemetry/ — synced from shared/telemetry/ via sync-to-plugin.js (12 lib modules + ikey.json carrying the PowerPagesPluginEvent stream name + disabled:true kill switch). - scripts/lib/telemetry-runner.js — runInstrumented() shim plus the PowerPagesPluginEvent envelope name and tracked-script registry. - hooks/run-skill-pretool-telemetry.js, hooks/run-skill-posttool-validation.js — fire-and-forget telemetry around the Skill tool, preserving the existing validator's behavior and exit code. - hooks/hooks.json — registers the two Skill hooks. (UserPromptSubmit is registered separately by the slash-command branch.) - Validators (validate-activation/seo/site/webroles/audit/auth/datamodel/ cloudflow/serverlogic/webapi-integration) and four other plugin scripts (check-activation-status, clear-site-cache, render-audit-report, verify-dataverse-access) wrapped with runInstrumented() so each script run produces script_started/script_completed signals. - scripts/tests/{telemetry-hook-pretool,telemetry-hook-posttool,telemetry-runner}.test.js — hermetic node:test coverage for the new wiring. Repo updates: - AGENTS.md (plugin) gains the Telemetry section pointing at shared/telemetry/ as the canonical source of truth. - README.md (root) mentions anonymous telemetry + env opt-out. - docs/superpowers/handoff/PowerPagesPluginEvent-annotation.xml is the tenant-team handoff describing the EventStreamingAnnotation for the per-plugin Kusto stream. Privacy posture: default-on, env-var opt-out only (POWER_PLATFORM_SKILLS_TELEMETRY=0). No first-run prompt, no consent file. Telemetry stays gated by ikey.json's disabled:true flag until the tenant-side annotation + Kusto table are provisioned. * fix(power-pages): telemetry-runner — pass envelopeName, read instrumentationKey Two regressions caught during E2E verification of the local-log path: 1. **envelopeName was never forwarded** — `runInstrumented()` called `withTelemetry()` without an `envelopeName`, so events shipped with `name: ""`. In production this means the tenant's `EventStreamingAnnotation name="^PowerPagesPluginEvent$"` matcher silently drops every event (`acc:1` from the collector but zero rows in Kusto). Now reads `event_stream_name` from `ikey.json` and passes it through. 2. **iKey read the old JSON property** — `deps.ikeyCfg.ikey` was undefined after the `ikey` → `instrumentationKey` rename in the foundation library (PR #142). The dispatcher saw an empty `POWER_PLATFORM_SKILLS_IKEY` and always took the placeholder / local-log path, even with a real iKey provisioned. Now reads `deps.ikeyCfg.instrumentationKey`. Adds an opt-in `_overrides` test seam on `runInstrumented()` so unit tests can inject a fake `withTelemetry` and assert opts are propagated correctly. New tests: - `runInstrumented forwards envelopeName from ikey.json event_stream_name` - `runInstrumented reads iKey from instrumentationKey property (not legacy 'ikey')` Verified end-to-end by piping a synthetic event directly to the dispatcher: envelope `name` is now `"PowerPagesPluginEvent"`, `scriptName` is correctly set, event lands in the local JSONL. Tests: 4/4 telemetry-runner, full plugin suite still green. * telemetry: detect GitHub Copilot CLI as ai agent host readAiAgent now recognizes COPILOT_CLI=1 and reads COPILOT_CLI_BINARY_VERSION, in addition to Claude Code. Priority remains: explicit AI_AGENT_NAME env > Claude Code > Copilot CLI. Adds test coverage and re-syncs into plugins/power-pages. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(telemetry): drop power-pages testing iKey from shared/telemetry/ikey.json `shared/telemetry/ikey.json` is the dev-time placeholder copied by `sync-to-plugin.js`. It was committed with power-pages's real testing iKey and INT collector URL, which meant any future adopter (canvas-apps, code-apps, model-apps) running the sync would inherit those values and — if they ever flipped `disabled: false` without replacing the iKey — silently route their telemetry to power-pages's Kusto stream. This commit replaces both with explicit placeholders so every plugin is forced to provision its own cluster config before going live. The plugin's own synced copy at `plugins/power-pages/scripts/lib/telemetry/ikey.json` keeps its real plugin-specific values; only the shared template is sanitized. - instrumentationKey: <pp's value> → "PLACEHOLDER_REPLACE_BEFORE_SHIPPING" - collector_url: <INT URL> → "" `event_stream_name` ("PluginEventStreamPlaceholder") and `disabled: true` were already plugin-agnostic and stay unchanged. Tests: 110/110 shared (no behavior change — the placeholder iKey constant matches the dispatcher's existing keyMissing check). * fix(power-pages): address PR #153 review — await, dedup, hook fast-path Three review fixes from priyanshu92 on PR #153: 1. **telemetry-runner.js: `return await asyncFn()`** — added the `await` on the no-telemetry-deps fallback path so stack traces include the `runInstrumented` frame. Behavioural no-op; debug-friendliness only. 2. **validate-auth.js: drop duplicate `runInstrumented` call** — the file invoked `runInstrumented('validate-setup-auth', main)` twice: once unconditionally at module top-level (line 69-72), and again inside the standard `if (require.main === module)` guard (line 137-142). When executed as a script (the actual PostToolUse-hook use case), `main()` ran twice — duplicating validator I/O and emitting two `script_started`/`script_completed` pairs per run. When required as a module (tests), the top-level call still fired, breaking import semantics. Deleted the top-level invocation, kept the require.main guard. 3. **Skill hooks: fast-path opt-out / kill switch BEFORE pac shell-outs** — `run-skill-pretool-telemetry.js` and `run-skill-posttool-validation.js` ran `pac auth who` (3s timeout) and `pac --version` (2s timeout) unconditionally on every tracked Skill invocation, even when the user had opted out (`POWER_PLATFORM_SKILLS_TELEMETRY=0`) or the shipped kill switch was on (`ikey.json` `disabled: true`). Worst case ~10s added latency per Skill tool use, currently hit by every user since `disabled: true` is the shipped default. Both hooks now check, in this order, BEFORE any pac shell-out: a) env opt-out (`POWER_PLATFORM_SKILLS_TELEMETRY === "0"`) b) repo-side kill switch (`ikey.json` `disabled === true`) c) unconfigured (`!ikey.instrumentationKey`) Any of those → exit early. The posttool hook still preserves the validator's exit code regardless of telemetry state. `readIkey()` in the pretool hook now also exposes the `disabled` flag so the gate can read it without re-parsing. Comment #3 (extend agent-info detection to Codex / OpenCode) is pending reviewer confirmation of those hosts' env-var conventions — not implemented here. Any host can already self-identify via the AI_AGENT_NAME / AI_AGENT_VERSION override. Tests: 10/10 plugin telemetry hooks + runner. Shared suite unchanged. --------- Co-authored-by: Amit Joshi <amitjoshi@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(power-pages): emit skill_started for slash-command invocations (#168) Adds telemetry for skills invoked via slash command (e.g. /power-pages:add-seo) at the user-prompt boundary, which the existing PreToolUse:Skill hook does not cover (that fires only for programmatic Skill tool calls). - hooks/run-user-prompt-telemetry.js: UserPromptSubmit hook that matches /<plugin>:<skill> via shared/telemetry/lib/prompt-detector.js and calls emitSkillStartedFromPrompt() with PAC auth + agent info enrichment. Fire-and-forget; never blocks prompt submission. - hooks/hooks.json: registers the new UserPromptSubmit hook alongside the existing PreToolUse / PostToolUse:Skill hooks. - scripts/tests/run-user-prompt-telemetry.test.js: hermetic coverage for the hook (tracked-skill match, no-match no-op, malformed input, env opt-out). - docs/superpowers/specs/2026-04-23-slash-command-telemetry-design.md: design doc covering why this hook exists and how it composes with the Skill-tool path. skill_completed remains the responsibility of the PostToolUse:Skill hook for skills that flow through the Skill tool. For slash-only skills, completion is tracked indirectly via script-level runInstrumented() spans. Co-authored-by: Amit Joshi <amitjoshi@microsoft.com> * docs(telemetry): remove implementation specs + handoff XML The 1DS telemetry foundation has landed and is in active use. The design specs (slash-command-telemetry-design, telemetry-rebuild-design, plugin adoption guide) and the PowerPagesPluginEvent annotation XML were implementation-time artifacts — the live integration is now documented in shared/telemetry/README.md, which covers what is sent, the privacy posture, and how to adopt the library in a new plugin. Updates plugins/power-pages/AGENTS.md to point readers at the README instead of the deleted spec. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(telemetry): honor disabled flag before PAC + spawn in user-prompt + script paths The pretool and posttool hooks already short-circuit on cfg.disabled before invoking pac auth who (~3s) and pac --version (~2s). The two remaining entry points — emit-from-prompt (used by the UserPromptSubmit hook) and telemetry-runner (used by scripts wrapped in runInstrumented) — were calling PAC and spawning the detached dispatcher regardless; the dispatcher caught disabled later but the user had already paid the shellout cost. Add three fast-path gates at both entry points, in this order: - POWER_PLATFORM_SKILLS_TELEMETRY=0 (env opt-out) - cfg.disabled === true (repo-side kill switch) - empty instrumentationKey (unprovisioned) Any of these returns immediately and runs the user's original async fn (telemetry-runner) or a no-op result (emit-from-prompt). Zero PAC, zero agent-info, zero spawn. Also fix a pre-existing destructuring bug in emit-from-prompt.test.js's mkTelemetryDir helper: callers passed 'instrumentationKey' but the helper destructured 'ikey', so every test ran with an empty iKey field by accident. Fix the helper and add test coverage for all three new fast paths in both modules. Synced shared/telemetry/lib/emit-from-prompt.js into the power-pages plugin copy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(telemetry): rewrite README as integration guide for new adopters Reorganize around the questions a plugin author actually asks: - What does the library do? (data flow diagram) - What goes on the wire? (fields + privacy posture) - Where do the modules live? (lib/ layout) - How do I adopt this in my plugin? (5 numbered steps) - How do I update the shared lib without clobbering my ikey.json? - What test seams exist? Drop the dangling reference to the deleted design spec. The new "Adopting in a new plugin" section walks through sync, ikey.json config, hook registration, optional script wrapping, and local verification — covering everything a new plugin needs without requiring the implementation history. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(telemetry): make PAC shellouts actually capture data on Windows Two real bugs causing every emitted event to silently drop orgId, tenantId, and pacCliVersion: 1) pac-auth.js TIMEOUT_MS was 3000, but `pac auth who` cold-start on Windows consistently takes 3.5-4s (.NET runtime + cached-token validation). Every call timed out, the catch handler swallowed it, readPacAuth returned null, and the hooks emitted events without the PAC identity fields. Bumped to 8000 — still well under the 30s hook budget. 2) agent-info.js readPacCliVersion ran `pac --version`, but PAC 2.x treats `--version` as an unknown command: it prints the version banner ("Version: 2.7.4+g06bb2eb ...") to stdout but exits with status 1. execFileSync throws on non-zero exit, the old catch discarded everything, and the version was lost. Now we read err.stdout from the thrown error and parse the banner. Tightened the regex from a bare semver match to one anchored on the "Version:" prefix so we don't accidentally pick up the .NET Framework version from the same line. Bumped its timeout to 8s too. Verified on this machine: cold-start total of both reads is ~5.7s, returning real orgId/tenantId/cloud and pacCliVersion "2.7.4". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(telemetry): stabilize sessionId across hook invocations in a Claude Code session session.getSessionId() cached a random UUID in module-level state, but each hook spawns as a fresh Node process — the cache reset every invocation, so every event in a Claude Code session carried a different sessionId. That broke session-scoped joins in Kusto. Claude Code sends session_id in the hook stdin payload. Wire it through: - session.js: getSessionId(override?) — if a truthy string is passed, prime the cache with it. Subsequent calls (with or without override) return the primed value. _resetCache() exported for tests. - pretool / posttool / user-prompt hooks: extract parsed.session_id from the hook payload and pass it to getSessionId / through emitSkillStartedFromPrompt opts. - emit-from-prompt.js: accept opts.sessionId and forward to getSessionId. Synced into the plugin copy. New session tests cover override winning over a previously-cached UUID, multi-process stability when callers pass the same primed value, and falsy-override safety. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(telemetry): design spec for skill-only events + session/correlation cleanup Records the audit findings and approved design for collapsing the telemetry event vocabulary to skill_started + skill_completed only. Removes the script-wrapping path (with-telemetry, telemetry-runner, runInstrumented) and adds a TTL sweep to correlation.js to bound leaks when PostToolUse never fires. Ratifies the just-shipped sessionId fix that sources Claude Code's session_id from hook stdin. Finding E (missing skill_completed in local trace) deferred to a separate investigation after this cleanup lands. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(telemetry): make sessionId sourcing host-agnostic (Claude Code + Copilot CLI) Per review feedback: sessionId should come from whichever host agent the user is running under. Add extractSessionIdFromPayload helper that checks both session_id (Claude Code convention) and sessionId (camelCase fallback for hosts that use that). aiAgentName already detects both hosts via agent-info.js; this brings sessionId into the same multi-host pattern. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(telemetry): resolve session_id from COPILOT_SESSION_ID env for Copilot CLI Copilot CLI exposes its session ID via the env var COPILOT_SESSION_ID, not through the hook stdin payload. Rename the helper to resolveHostSessionId and have it consult, in precedence order: payload.session_id, payload.sessionId, env.COPILOT_SESSION_ID, empty. Each hook stays a single line; new hosts can be added by extending the resolver without touching hook code. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(telemetry): drop COPILOT_SESSION_ID env path; both hosts use payload Per user feedback: GitHub Copilot CLI uses the same hook stdin payload mechanism as Claude Code to surface its session id. Drop the COPILOT_SESSION_ID env var fallback; resolveHostSessionId now only inspects payload.session_id and payload.sessionId. Simpler resolver, one fewer test, no env coupling. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(telemetry): implementation plan for skill-only events + session/correlation cleanup Bite-sized TDD plan covering: resolveHostSessionId helper, correlation TTL sweep, removal of buildScript builders, deletion of with-telemetry + telemetry-runner, unwrap runInstrumented from 10 validators + 4 standalone scripts, switch 3 hooks to resolveHostSessionId, sync, README update, manual verification, push. 12 tasks total. Spec reference: docs/superpowers/specs/2026-05-28-session-correlation-cleanup-design.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(telemetry): add resolveHostSessionId helper for multi-host session sourcing * feat(telemetry): TTL-sweep stale correlation files in correlation.write * feat(telemetry): drop buildScript builders + scriptName field * feat(telemetry): delete with-telemetry script wrapper * feat(power-pages): delete telemetry-runner plugin wrapper * feat(power-pages): unwrap runInstrumented from validators * feat(power-pages): unwrap runInstrumented from standalone scripts * feat(power-pages): hooks use resolveHostSessionId for multi-host session sourcing * chore(power-pages): sync session/correlation cleanup into plugin copy * docs(telemetry): drop script-wrapper section from README; align with skill-only events * fix(power-pages): hoist input parse so posttool emits skill_completed The const input was scoped to the first try block while the telemetry block (second try) also references it for resolveHostSessionId. The ReferenceError was silently swallowed by the outer telemetry catch, preventing all skill_completed events from emitting. This is the root cause of Finding E (9 skill_started, 0 skill_completed). Hoist to outer async scope so both blocks see the parsed payload. * fix(telemetry): backfill aiAgentVersion from CLAUDECODE when only AI_AGENT_NAME is set If the host sets AI_AGENT_NAME (e.g., via .claude/settings.local.json) but omits AI_AGENT_VERSION, the previous code returned aiAgentVersion="" and the hook silently dropped the field. Now: when AI_AGENT_NAME is explicit but the version is empty, fall through to the same built-in detection used when AI_AGENT_NAME isn't set at all — read the version from CLAUDE_CODE_EXECPATH/../package.json for Claude Code, or COPILOT_CLI_BINARY_VERSION for Copilot CLI. Explicit AI_AGENT_VERSION still wins over the backfill when present. Extracted readClaudeCodeVersion(env) helper to share the package.json read path between the explicit-name and built-in detection branches. Synced into the plugin copy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(telemetry): correct aiAgent detection description in README Reflect actual agent-info logic: Claude Code version is read from the installed package.json via CLAUDE_CODE_EXECPATH, Copilot CLI version from COPILOT_CLI_BINARY_VERSION, and an explicit AI_AGENT_NAME backfills its version from the matching built-in detector when AI_AGENT_VERSION is empty. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(telemetry): remove session-correlation cleanup design + plan The session/correlation cleanup work has shipped; drop the superpowers design spec and implementation plan that tracked it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * revert(power-pages): restore main's validators + standalone scripts These 10 validators and 4 standalone scripts diverged from main only by the leftover runInstrumented-unwrap shape (async main() wrapper + require.main guard + module.exports). That structure is a vestige of an earlier per-script telemetry approach that was dropped in favor of hook-based skill telemetry, so it's no longer needed. Restore main's versions to keep the telemetry PR scoped to telemetry-only changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(telemetry): derive Claude Code version from AI_AGENT on non-npm installs aiAgentVersion was derived solely from CLAUDE_CODE_EXECPATH -> ../package.json, which only exists in the npm-global install layout. On native-installer Claude Code (standalone binary, now the default), that read fails and the version is dropped while aiAgentName still logs as "Claude Code" -- name present, version blank. Add an install-method-independent fallback: when the package.json read yields nothing, parse the dotted semver out of AI_AGENT (claude-code_<maj>-<min>-<patch>_agent), which Claude Code sets regardless of install method. The npm path stays primary, so npm installs are unaffected. Edited shared/telemetry and re-synced to plugins/power-pages. Adds test coverage for native-install fallback, EXECPATH-unset fallback, npm-wins precedence, and malformed AI_AGENT. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(telemetry): address Copilot review on PR #176 - emit-dispatcher: pass url.port to https.request so a non-443 collector URL POSTs to the configured port instead of always 443 - events: tighten isPlainStructured() to only accept arrays and plain objects (prototype Object.prototype/null), rejecting class instances like Error from the dynamic eventInfo field - emit-from-prompt + posttool hook: add POWER_PLATFORM_SKILLS_IKEY_JSON override seam so tests point at a temp ikey.json instead of mutating the checked-in config (removes a parallel-test race) - posttool hook: skip telemetry emission when no tracked skill was resolved (malformed stdin / pre-detection error) so no incomplete skill_completed event is emitted; exit code unchanged - run-user-prompt-telemetry test: replace busy-wait loops with a non-spinning Atomics.wait sleep + waitForFile helper, and use the ikey override instead of mutating the shipped ikey.json Shared lib edits propagated to the power-pages synced copy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(telemetry): design for removing skill_completed logging flow Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(telemetry): stop emitting skill_completed from posttool hook Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(telemetry): inline correlationId in pretool hook, drop disk correlation Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(telemetry): delete unused correlation.js and its test Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(telemetry): remove orphaned synced correlation.js Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(telemetry): document removal of skill_completed flow Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(telemetry): describe only live telemetry; drop skill_completed references Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: remove brainstorm/plan docs for skill_completed removal Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Clarify shared telemetry README event scope * fix(telemetry): preserve real ikey on resync; drop unused test consent artifact Address PR #176 review: - sync-to-plugin.js no longer clobbers a provisioned ikey.json. It is preserved when the target carries a non-placeholder instrumentationKey and only seeded when missing or still on the placeholder, so the documented `sync-to-plugin --target plugins/power-pages` is safe to re-run. Adds preserve-on-resync test coverage. - run-user-prompt-telemetry.test.js: simplify mkConfigDir to a tmpdir (the telemetry.json consent file and `enabled` arg were never read; live gates are POWER_PLATFORM_SKILLS_TELEMETRY=0 and ikey.json.disabled). - Raise the enabled-path spawn timeout 10s->30s to match the hook's documented pac budget; the 10s timeout flaked on the ~10s pac shellout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(telemetry): fail closed on unreadable kill-switch config; refresh README Address PR #176 second review pass: - emit-dispatcher.js isDisabledByConfig() now fails CLOSED when ikey.json is missing/unreadable (treat as disabled) instead of fail-open. The config is a kill switch; if it can't be read we can't confirm emission is authorized, so we suppress rather than risk a POST / local log in an unexpected state. Matches the repo's documented "fail closed" telemetry posture. Adds an emit-dispatcher test covering the missing-config path. Synced to the power-pages copy via sync-to-plugin.js. - README: the sync section claimed sync overwrites ikey.json and to git-checkout it back; sync now preserves a provisioned key, so describe the preserve/seed behavior instead. - README: corrected the session.js test seam entry — session.js has no opts.configDir / filesystem state; the seams are _resetCache() and the getSessionId(override) parameter. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(telemetry): propagate fail-closed + ikey override seam to readIkey copies Self-review follow-up to e20c6d3 (which only fixed emit-dispatcher.js): - emit-from-prompt.js and the run-skill-pretool-telemetry.js hook now fail CLOSED (disabled: true) when ikey.json is missing/unreadable, matching emit-dispatcher.js's isDisabledByConfig(). The `ikey: ""` guard already blocked emission, so this is a semantics fix that keeps all three readIkey paths consistent and prevents the kill switch from being bypassed by a missing/corrupt config. - The pretool hook's readIkey() now honors POWER_PLATFORM_SKILLS_IKEY_JSON, the same test/override seam the dispatcher and emit-from-prompt already respect. This lets a test redirect ikey.json without mutating the checked-in file (avoiding the parallel-test race fixed elsewhere). - Adds pretool-hook tests covering the enabled emit path (via the override seam) and the fail-closed path (missing override file → no emit). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(telemetry): make readIkey self-protecting against missing telemetryDir emit-from-prompt.js readIkey() computed path.join(telemetryDir, "ikey.json") outside the try, so a caller that omits telemetryDir (with no IKEY_JSON override) would throw a TypeError instead of failing closed — contradicting the library's fail-closed contract. Move path resolution inside the try so any bad input falls through to disabled: true. Production callers already pass telemetryDir and wrap the call, so this is defense-in-depth at the library boundary. Adds a test for the missing-telemetryDir path. Synced to power-pages. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(telemetry): detect bare /<skill> slash commands, not just /power-pages:<skill> Per review feedback: the host tool controls how a slash command is surfaced in the UserPromptSubmit payload and may emit the bare form (`/add-seo`) without the plugin namespace. Make the `pluginName:` prefix optional in the detector regex so both forms are recognized. The captured skill name is still validated against trackedSkills, so a bare `/foo` only counts when `foo` is a known skill, and the trailing boundary lookahead still rejects another plugin's namespaced command (`/other-plugin:add-seo` → null). Adds tests for the bare form, non-tracked bare commands, and bare substrings. Synced to power-pages. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(telemetry): update power-pages ikey.json to int collector + PagesAIPluginEvent stream Points the synced config at the int 1DS instrumentation key / us-mobile collector and renames the event stream to PagesAIPluginEvent. Stays disabled: true (inert) until the tenant-side stream is provisioned. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(telemetry): broaden skill tracking - Detect additional AI agent hosts from hook env variables - Derive Power Pages tracked skills from skill folders - Discover optional validators and refresh hook docs/tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(telemetry): write a local diagnostic mirror of every event Always append a faithful, on-disk mirror of each emitted event to ~/.power-platform-skills/events.jsonl, irrespective of iKey presence and irrespective of the POWER_PLATFORM_SKILLS_TELEMETRY=0 opt-out. - Mirror shape is {time, name, data} where `data` is the sanitized payload (== Kusto column names) and `time` matches the wire envelope; the dispatcher computes both once and shares them with the POST so they can never diverge. - Previously the local log was written ONLY when the iKey was missing; a real-iKey event was transmitted but never recorded locally. Now it is mirrored on every path that clears the kill switch. - POWER_PLATFORM_SKILLS_TELEMETRY=0 now suppresses TRANSMISSION only — the local mirror is still written. The opt-out fast-paths were removed from emit-from-prompt.js and run-skill-pretool-telemetry.js so the enriched event still reaches the dispatcher; it writes the mirror then skips the POST. Trade-off: opted-out runs now incur the same event-building (pac) cost as enabled runs. - `disabled: true` in ikey.json remains the one true hard-off (no mirror, no POST), checked before any side effect. - Spell out POWER_PLATFORM_SKILLS_TELEMETRY=0 in full everywhere (was mixed with TELEMETRY=0 shorthand) so it is not confused with the ikey.json `disabled` flag. - Update README flow + privacy posture; add/adjust tests (mirror==wire equality, opt-out-still-mirrors); re-sync shared/telemetry into the power-pages plugin. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * experiment(telemetry): symlink plugin lib to shared/telemetry/lib instead of copying Replaces the synced copy at plugins/power-pages/scripts/lib/telemetry/lib with a relative symlink -> ../../../../../shared/telemetry/lib. ikey.json is kept as a real file (it carries the plugin's real config, not shared's placeholder). Git tracks the link as mode 120000; Node require() resolves through it and all 12 telemetry hook tests pass. KNOWN CAVEAT (not yet addressed): emit-dispatcher.js finds its kill-switch config via path.join(__dirname, '..', 'ikey.json'). Node resolves __dirname through the symlink to the REAL dir (shared/telemetry/lib), so the default resolves to shared/telemetry/ikey.json (the placeholder, disabled:true) rather than the plugin's real ikey.json. Masked today (both disabled:true); once the plugin config is flipped to disabled:false this would keep emission suppressed. A complete symlink implementation must pass POWER_PLATFORM_SKILLS_IKEY_JSON (the plugin's real ikey.json) through emit-spawn so the dispatcher reads the right config regardless of realpath. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(telemetry): dispatcher reads the calling plugin's ikey.json under shared lib When lib/ is shared (symlink or relative require), emit-dispatcher.js runs from shared/telemetry/lib, so its __dirname-based default kill-switch config resolves to shared/'s placeholder (disabled:true) instead of the plugin's real ikey.json. Once a plugin flips its config to enabled, emission would stay suppressed. Fix: emit-spawn forwards POWER_PLATFORM_SKILLS_IKEY_JSON from a new opts.ikeyJsonPath (an explicit env override still wins, for tests); the pretool hook and emit-from-prompt pass the calling plugin's ikey.json path. Adds an emit-spawn regression test that drives the opts path with no env override. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(telemetry): design spec for interactive telemetry toggle Brainstormed design for a per-plugin `/<plugin>:telemetry on|off|status` command backed by a host-neutral ~/.power-platform-skills/config.json (per-plugin keys). Replaces the POWER_PLATFORM_SKILLS_TELEMETRY env var with the config file as the single user off-switch; "off" stops transmission only (local mirror kept). Internal ikey.json `disabled` flag stays hidden; every command prints an anonymity reassurance. Pre-implementation; no code changed yet. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(telemetry): implementation plan for the telemetry toggle 8-task TDD plan from the approved spec: new user-config.js module + CLI, dispatcher gate rewired to per-plugin config.json (env var removed), shared telemetry skill + power-pages wrapper, tracking exclusion, user-facing README, re-sync + live verification. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(telemetry): add user-config module for per-plugin opt-out * feat(telemetry): gate transmission on per-plugin config instead of env var * feat(telemetry): add CLI to read/write the telemetry toggle * feat(telemetry): add shared telemetry skill + power-pages wrapper * feat(telemetry): exclude the telemetry skill from usage tracking * refactor(telemetry): remove POWER_PLATFORM_SKILLS_TELEMETRY env var entirely * docs(telemetry): document the telemetry toggle in the user-facing README * chore(telemetry): sync user-config + CLI into power-pages * chore(telemetry): remove design spec and implementation plan docs * refactor(telemetry): bundle skill workflow via symlink like report-issue Switch the telemetry skill from a repo-root-relative workflow reference (${CLAUDE_PLUGIN_ROOT}/../../shared/...) to the bundled-path pattern used by report-issue: SKILL.md points at ${CLAUDE_PLUGIN_ROOT}/skills/telemetry/ telemetry-workflow.md, backed by a per-plugin symlink to the shared source. Marketplace installs copy only the plugin directory, so the old ../../shared path did not resolve at user time. The symlink keeps a single source of truth in-repo while letting installers dereference it into the plugin cache. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(telemetry): finalize symlink approach (drop sync-to-plugin, update docs) Make the symlink the single source of truth for the shared telemetry lib, replacing the synced-copy mechanism: - remove shared/telemetry/sync-to-plugin.js and its test (superseded by the per-plugin lib symlink; the script would have copied lib/ onto itself) - rewrite the telemetry docs to the symlink model across root AGENTS.md, plugins/power-pages/AGENTS.md, and shared/telemetry/README.md, including the symlink adoption recipe and the ikey.json posture rule (committed stays disabled:true; working-tree disabled:false is a local experiment only) - gitignore .omc/ scratch directories Verified: shared telemetry suite 147/147, power-pages telemetry hooks 23/23. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(telemetry): address PR review (array config, null-proto skills, test comment) - user-config.readConfig: ignore an array config.json. typeof [] === "object" passed the old guard, so setTelemetryChoice set .telemetry on the array and JSON.stringify silently dropped it — returning true while persisting nothing. Now guarded with !Array.isArray + regression test. - powerpages-hook-utils: build TRACKED_SKILLS with Object.create(null). Membership is tested via bracket access, so a plain {} let inherited keys (toString, constructor, __proto__) test truthy and emit bogus skill names. Added a test. - events.test.js: fix the misleading "drop guarded by clamp" comment — boolean true is kept and coerced to 1 (Number(true)), not dropped. Addresses copilot review comments on PR #176. Shared telemetry 148/148, power-pages telemetry hooks 24/24. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Amit Joshi <amitjoshi@microsoft.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Priyanshu Agrawal <priyanshuag@microsoft.com>
1 parent 36fcbc7 commit 98430e9

47 files changed

Lines changed: 4229 additions & 99 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,3 +48,6 @@ ENV/
4848
Thumbs.db
4949

5050
.playwright-mcp/
51+
52+
# oh-my-claudecode scratch state
53+
.omc/

AGENTS.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,14 @@ Skills that apply to all plugins live in `shared/skills/<skill-name>/`. The work
6363

6464
This keeps the skill discoverable in each plugin while preserving install-time portability. Marketplace installs copy only the plugin directory, so per-plugin wrappers must not reference repo-root `shared/` paths at runtime. Instead, point the wrapper at `${CLAUDE_PLUGIN_ROOT}/skills/<skill-name>/<workflow>.md` and keep a symlink from that per-plugin path to the repo-root shared workflow; marketplace installers dereference same-marketplace symlinks into the installed plugin cache. When updating a shared skill, edit the workflow file and/or `SKILL.template.md` in `shared/`, then update the per-plugin wrappers (frontmatter + bundled workflow reference, with `{{PLUGIN_NAME}}` substituted) and ensure any per-plugin symlinks still resolve under `plugins/<plugin>/skills/<skill-name>/`. Commit the shared source and per-plugin symlinks together.
6565

66+
## Shared Telemetry
67+
68+
1DS telemetry code for all plugins lives at `shared/telemetry/`. Each adopting plugin **symlinks** the library into its own tree — `plugins/<plugin>/scripts/lib/telemetry/lib` is a symlink to `shared/telemetry/lib`. The marketplace installer dereferences that symlink into the installed plugin at install time, so the shared code ships without copying it into each plugin. Each plugin keeps its own real `ikey.json` next to the symlink.
69+
70+
Edit `shared/telemetry/` directly — the symlink makes changes live for every adopting plugin immediately; there is nothing to re-sync.
71+
72+
Current adopters: `power-pages`. Others adopt on demand.
73+
6674
## Code Conventions
6775

6876
**DRY (Don't Repeat Yourself):** Never duplicate logic across files. Each plugin has shared utilities (e.g., `scripts/lib/`) and shared reference docs (e.g., `references/`). Always check for and reuse existing helpers before writing new code. When adding shared logic, put it in the plugin's shared modules — not in individual skill directories.

README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -211,3 +211,7 @@ trademarks or logos is subject to and must follow
211211
[Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/legal/intellectualproperty/trademarks/usage/general).
212212
Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.
213213
Any use of third-party trademarks or logos are subject to those third-party's policies.
214+
215+
## Telemetry
216+
217+
Plugins that ship 1DS telemetry (currently: `power-pages`) gather anonymous usage signals. Telemetry is default-on; users opt out per-plugin via the `/<plugin>:telemetry off` command (e.g. `/power-pages:telemetry off`), stored in `~/.power-platform-skills/config.json`. See `shared/telemetry/README.md` for what is sent.

plugins/power-pages/AGENTS.md

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -153,15 +153,15 @@ Skills are defined in `SKILL.md` files with YAML frontmatter (name, description,
153153

154154
### Hooks
155155

156-
Hook registration is centralized in `hooks/hooks.json` — a single PostToolUse hook (matcher `Skill`) runs `hooks/run-skill-posttool-validation.js` after every Skill tool call. The runner consults the `TRACKED_SKILLS` map in `scripts/lib/powerpages-hook-utils.js`, looks up the validator for the skill that just completed, and invokes it with the current cwd.
156+
Hook registration is centralized in `hooks/hooks.json` — a single PostToolUse hook (matcher `Skill`) runs `hooks/run-skill-posttool-validation.js` after every Skill tool call. The runner derives tracked skills directly from `skills/*/SKILL.md` via `scripts/lib/powerpages-hook-utils.js`, looks up an optional `skills/<skill>/scripts/validate*.js` validator for the skill that just completed, and invokes it with the current cwd.
157157

158158
To wire a new skill into validation:
159159

160160
1. Write the validator at `skills/<skill>/scripts/validate-<skill>.js` using the `runValidation((cwd) => { ... })` pattern from `scripts/lib/validation-helpers.js`.
161-
2. Register the skill in `TRACKED_SKILLS` (in `scripts/lib/powerpages-hook-utils.js`) with its `validatorScript` path.
162-
3. Add test coverage in `scripts/tests/powerpages-hook-utils.test.js` so an unregistered skill is caught in CI.
161+
2. No manual tracked-skill registration is needed. Any folder with `skills/<skill>/SKILL.md` is automatically tracked for telemetry and hook detection.
162+
3. Add or update test coverage in `scripts/tests/powerpages-hook-utils.test.js` if you introduce a new validator naming pattern.
163163

164-
Skills currently registered with command-backed validators: `activate-site`, `add-cloud-flow`, `add-seo`, `add-server-logic`, `audit-permissions`, `configure-env-variables`, `create-site`, `create-webroles`, `deploy-pipeline`, `ensure-pipelines-host`, `export-solution`, `force-link-environment`, `import-solution`, `integrate-webapi`, `plan-alm`, `setup-auth`, `setup-datamodel`, `setup-pipeline`, `setup-solution`. `add-sample-data` and `test-site` are tracked without command validators (no artifacts to verify). `diagnose-deployment` is intentionally not tracked — it's read-only and produces no artifacts to verify.
164+
All skill folders are tracked. Skills without a `scripts/validate*.js` file are tracked for telemetry/detection but skip validation.
165165

166166
**Anti-patterns** (see `PLUGIN_DEVELOPMENT_GUIDE.md` for the rationale): do not add `hooks: Stop:` blocks to individual SKILL.md frontmatter — they duplicate the centralized PostToolUse hook and fire too often. Do not use `type: prompt` Stop hooks for skill-completion checks — they create runaway forced-continuation loops.
167167

@@ -342,7 +342,7 @@ This runs a lightweight check comparing the local plugin version against `origin
342342

343343
- **Approval Gates** — Every load-bearing `AskUserQuestion` is an **Approval Gate**. Pause at minimum after gathering requirements, after presenting a plan, after implementation, and before deployment (Three-Point Approval Pattern). For ALM skills, every gate must (a) be catalogued in `references/approval-gates.md` §6 with a stable `gate-id`, and (b) be marked in SKILL.md with the explicit-pairing comment `<!-- gate: skill:phase | category=<intent|plan|progress|consent|final|pause> | cancel-leaves=<vocab> -->` followed by a human-readable `> 🚦 **Gate (...)**` block. New ALM skills must extend the catalog in the same PR that introduces the skill. Non-ALM skills should follow the same convention as the catalog is extended in a follow-up; lint runs warn-only on non-ALM skills until then. Do not coin alternative terms ("review gate", "approval checkpoint", "manual step" etc.) — the canonical term is **Approval Gate**.
344344
- **Deployment prompt** — Skills that modify site artifacts should end by asking "Ready to deploy?" and invoke `/deploy-site` if yes.
345-
- **Lifecycle hooks**If a skill needs command validation or checklist enforcement, update `hooks/hooks.json` and `scripts/lib/powerpages-hook-utils.js`. Do not define hook registration in individual `SKILL.md` files.
345+
- **Lifecycle hooks**Hook registration is centralized in `hooks/hooks.json`; `scripts/lib/powerpages-hook-utils.js` derives tracked skills from `skills/*/SKILL.md` and discovers optional `scripts/validate*.js` validators. Do not define hook registration in individual `SKILL.md` files.
346346
- **Graceful failure** — Track API call results, never auto-rollback, report failures clearly, continue with remaining items.
347347
- **Token refresh** — Refresh Azure CLI token every ~20 records / 3-4 tables / ~60 seconds.
348348
- **Git commits** — Commit after every significant milestone (each page/component, design foundations, phase completion).
@@ -388,6 +388,17 @@ These patterns have caused repeated PR review feedback. Check for them before su
388388
- **Template placeholders in `<script>` blocks need special care**`render-template.js` injects string values as-is (no encoding), which is safe for HTML text contexts but risky inside JavaScript. Avoid declaring JS variables with `"__PLACEHOLDER__"` in script blocks; prefer reading from the DOM or using `JSON.stringify` for JS contexts.
389389
- **Guidance must be consistent within a skill** — If one section says "always use raw fetch", a framework-specific table in the same file must not recommend a different HTTP client without qualification. Reviewers will flag contradictions.
390390

391+
## Telemetry
392+
393+
This plugin ships 1DS telemetry for skill-run and script-run signals. The shared library lives at the repo-root `shared/telemetry/`; `scripts/lib/telemetry/lib` is a **symlink** to `shared/telemetry/lib`, so the shared code is the live code. Zero npm dependencies — nothing to install.
394+
395+
- **`scripts/lib/telemetry/lib` is a symlink** to the repo-root `shared/telemetry/lib` — edit `shared/telemetry/lib/` directly; there is no copy to re-sync. The one real file under `scripts/lib/telemetry/` is `ikey.json` (this plugin's config). **Posture:** the committed `ikey.json` ships `disabled: true`; a working-tree `disabled: false` is a local experiment only — never commit it.
396+
- **Privacy posture:** anonymous telemetry is **default-on**. There is no consent prompt in skills. Users opt out via `/power-pages:telemetry off`, which stores a per-plugin choice in `~/.power-platform-skills/config.json` (`telemetry["power-pages"] = "off"`). Opting out stops transmission only; the local diagnostic mirror is still written. Re-enable with `/power-pages:telemetry on`.
397+
- **Strict allowlist:** `shared/telemetry/lib/events.js` enforces exactly the fields listed in the spec. Never add a field to a builder without first adding it to the allowlist and documenting it in the design doc.
398+
- **Fail closed:** telemetry code must never change a script's exit code or break a skill run. Emission is fire-and-forget via a detached dispatcher child, so the hook or script returns before the HTTPS POST completes.
399+
400+
See `shared/telemetry/README.md` for the integration guide.
401+
391402
## Maintaining This File
392403

393404
Update when plugin structure or conventions change or you learn something which can be useful for new skills or agents.

plugins/power-pages/PLUGIN_DEVELOPMENT_GUIDE.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -197,7 +197,7 @@ LLMs are probabilistic. When an LLM constructs inline bash commands for Datavers
197197
|--------|---------|
198198
| `validation-helpers.js` | `runValidation()`, `findPath()`, `findProjectRoot()`, `approve()`, `block()` — shared boilerplate for all validators |
199199
| `powerpages-config.js` | Loads `.powerpages-site` YAML files (table permissions, site settings, web roles) with consistent parsing |
200-
| `powerpages-hook-utils.js` | Maps skill names to validator scripts for the hook dispatcher |
200+
| `powerpages-hook-utils.js` | Discovers skill folders and optional validator scripts for the hook dispatcher |
201201
| `powerpages-schema-validator.js` | Validates permission/site-setting YAML schema |
202202
| `table-permissions-validator.js` | Validates table permission YAML |
203203
| `web-roles-validator.js` | Validates web role YAML |
@@ -410,7 +410,7 @@ echo '{"reason":"ni-dev — ALM handled by infra"}' > .alm-deferred
410410

411411
PostToolUse on the `Skill` tool fires **once per skill invocation**. Stop fires on **every assistant pause** (including user-input waits — every "Continue?" prompt fires it).
412412

413-
This plugin uses PostToolUse via `hooks/hooks.json``run-skill-posttool-validation.js` → per-skill validator. Skill frontmatter must NOT declare its own `hooks: Stop:` block — those duplicate the centralized PostToolUse hook and fire too often. To wire validation for a new skill, register it in the `TRACKED_SKILLS` map in `scripts/lib/powerpages-hook-utils.js` (see `AGENTS.md` → "Hooks" for the registration steps).
413+
This plugin uses PostToolUse via `hooks/hooks.json``run-skill-posttool-validation.js` → per-skill validator. Skill frontmatter must NOT declare its own `hooks: Stop:` block — those duplicate the centralized PostToolUse hook and fire too often. `scripts/lib/powerpages-hook-utils.js` automatically tracks every `skills/*/SKILL.md` folder and discovers an optional `skills/<skill>/scripts/validate*.js` validator, so new skills do not need manual hook registration.
414414

415415
#### 4. Skills write explicit status, not just artifact presence
416416

plugins/power-pages/README.md

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ This keeps hook behavior in one place and avoids relying on skill-frontmatter ho
3838

3939
## Skills
4040

41-
The plugin provides 29 skills that cover the full lifecycle of a Power Pages code site — scaffolding, deployment, data modeling, backend integration, authentication, ALM and CI/CD, security review, testing, and auditing. Each skill is invoked conversationally — just describe what you want to do.
41+
The plugin provides 30 skills that cover the full lifecycle of a Power Pages code site — scaffolding, deployment, data modeling, backend integration, authentication, ALM and CI/CD, security review, testing, and auditing. Each skill is invoked conversationally — just describe what you want to do.
4242

4343
### Site scaffolding and deployment
4444

@@ -373,6 +373,17 @@ Collects context about the current session and opens a pre-filled GitHub issue a
373373
- Attaches relevant file paths and environment info
374374
- Opens the issue in your browser for final review
375375

376+
#### `/telemetry`
377+
378+
> "Turn off telemetry" · "Disable telemetry" · "Telemetry status"
379+
380+
Enables, disables, or checks the status of anonymous usage telemetry. Per-user and per-plugin; the choice is stored in `~/.power-platform-skills/config.json`. See [Telemetry & privacy](#telemetry--privacy) below.
381+
382+
- `/power-pages:telemetry status` — show the current setting
383+
- `/power-pages:telemetry off` — stop sending telemetry (nothing leaves your machine)
384+
- `/power-pages:telemetry on` — resume sending telemetry
385+
- No personal data is ever collected (anonymous: skill name, plugin version, OS, Node version)
386+
376387
## Agents
377388

378389
The plugin includes 4 specialized agents that are spawned automatically by skills when needed:
@@ -476,6 +487,27 @@ node plugins/power-pages/scripts/validate-permissions-schema.js --projectRoot /p
476487

477488
This Dataverse relationship check is intended for local validation only and should not be used in CI.
478489

490+
## Telemetry & privacy
491+
492+
This plugin sends **anonymous** usage telemetry by default to help Microsoft
493+
improve it. **No personal data is ever collected** — only things like skill name,
494+
plugin version, OS, and Node version. It never includes file paths, prompts, tool
495+
inputs, site names, URLs, credentials, usernames, or hostnames.
496+
497+
**Turn it on or off (per-user, applies to every project):**
498+
499+
```bash
500+
/power-pages:telemetry status # show the current setting
501+
/power-pages:telemetry off # stop sending telemetry
502+
/power-pages:telemetry on # resume sending telemetry
503+
```
504+
505+
When **off**, nothing leaves your machine. A local diagnostic copy of each event
506+
is still written to `~/.power-platform-skills/events.jsonl` so you can see exactly
507+
what would have been sent; delete it anytime. The setting is stored at
508+
`~/.power-platform-skills/config.json` (`{ "telemetry": { "power-pages": "off" } }`),
509+
so CI/headless environments can opt out by writing that file directly.
510+
479511
## License
480512

481513
[MIT](../../LICENSE)

plugins/power-pages/hooks/hooks.json

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,17 @@
11
{
22
"hooks": {
3+
"PreToolUse": [
4+
{
5+
"matcher": "Skill",
6+
"hooks": [
7+
{
8+
"type": "command",
9+
"command": "node \"${CLAUDE_PLUGIN_ROOT}/hooks/run-skill-pretool-telemetry.js\"",
10+
"timeout": 30
11+
}
12+
]
13+
}
14+
],
315
"PostToolUse": [
416
{
517
"matcher": "Skill",
@@ -11,6 +23,17 @@
1123
}
1224
]
1325
}
26+
],
27+
"UserPromptSubmit": [
28+
{
29+
"hooks": [
30+
{
31+
"type": "command",
32+
"command": "node \"${CLAUDE_PLUGIN_ROOT}/hooks/run-user-prompt-telemetry.js\"",
33+
"timeout": 30
34+
}
35+
]
36+
}
1437
]
1538
}
1639
}

plugins/power-pages/hooks/run-skill-posttool-validation.js

Lines changed: 21 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -23,41 +23,36 @@ process.stdin.on('data', (chunk) => {
2323

2424
process.stdin.on('end', () => {
2525
debug(`[power-pages hook] stdin closed, received ${inputData.length} bytes\n`);
26+
27+
let validatorStatus = 0;
28+
let skillName = null;
29+
let input = null;
30+
2631
try {
27-
const input = JSON.parse(inputData);
28-
const skillName = getTrackedSkillFromToolInput(input.tool_input);
32+
input = JSON.parse(inputData);
33+
skillName = getTrackedSkillFromToolInput(input.tool_input);
2934
if (!skillName) {
3035
debug('[power-pages hook] No tracked skill detected — skipping validation\n');
3136
process.exit(0);
3237
}
3338

3439
const validatorScript = getValidatorScript(skillName);
35-
if (!validatorScript) {
36-
debug(`[power-pages hook] Skill "${skillName}" has no validator — skipping\n`);
37-
process.exit(0);
40+
if (validatorScript) {
41+
const validatorPath = path.join(__dirname, '..', validatorScript);
42+
const result = spawnSync(process.execPath, [validatorPath], {
43+
input: inputData,
44+
encoding: 'utf8',
45+
cwd: input.cwd || process.cwd(),
46+
});
47+
if (result.stdout) process.stdout.write(result.stdout);
48+
if (result.stderr) process.stderr.write(result.stderr);
49+
validatorStatus = result.status ?? 0;
50+
debug(`[power-pages hook] Validator exited with code ${validatorStatus}\n`);
3851
}
39-
40-
debug(`[power-pages hook] Running validator for skill "${skillName}": ${validatorScript}\n`);
41-
42-
const validatorPath = path.join(__dirname, '..', validatorScript);
43-
const result = spawnSync(process.execPath, [validatorPath], {
44-
input: inputData,
45-
encoding: 'utf8',
46-
cwd: input.cwd || process.cwd(),
47-
});
48-
49-
if (result.stdout) {
50-
process.stdout.write(result.stdout);
51-
}
52-
53-
if (result.stderr) {
54-
process.stderr.write(result.stderr);
55-
}
56-
57-
debug(`[power-pages hook] Validator exited with code ${result.status ?? 0}\n`);
58-
process.exit(result.status ?? 0);
5952
} catch (err) {
6053
process.stderr.write(`[power-pages hook] Unexpected error: ${err.message}\n`);
61-
process.exit(0);
54+
validatorStatus = 0;
6255
}
56+
57+
process.exit(validatorStatus);
6358
});

0 commit comments

Comments
 (0)