Commit 98430e9
feat(telemetry): 1DS skill-run telemetry infra (#176)
* feat(telemetry) 1/3: shared library — dispatcher, consent, sync, builders (#142)
* docs: add design spec for 1DS telemetry infra
Design for a shared telemetry library at shared/telemetry/ consumed
by the power-pages plugin first, with interactive first-run consent,
strict-allowlist payloads, and fail-closed emission via the existing
PreToolUse/PostToolUse:Skill hook surface.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(telemetry): add shared 1DS telemetry library
Introduces the shared 1DS telemetry infrastructure used by all
adopting plugins. Anonymous, default-on, env-var opt-out only
(POWER_PLATFORM_SKILLS_TELEMETRY=0). No persistent consent file,
no first-run prompt.
Library (shared/telemetry/lib/):
- events.js — type-aware allowlist builders (PowerPagesPluginEvent
envelope, top-level columns, per-event field set, durationMs as int)
- emit-dispatcher.js — detached child that POSTs to OneCollector
with the Common Schema 4.0 envelope, exits silently on any failure
- emit-spawn.js — fire-and-forget spawn helper with restricted child env
- emit-from-prompt.js — orchestrator for slash-command skill_started
events (reads ikey.json, calls pac auth + agent info, builds event)
- pac-auth.js — shells out to `pac auth who` for orgId/tenantId
- agent-info.js — detects host AI agent (Claude Code, Copilot CLI)
+ pac CLI version via env override → auto-detect → empty
- prompt-detector.js — matches /plugin:skill from user prompt text
- local-log.js — placeholder-iKey JSONL fallback at
~/.power-platform-skills/events.jsonl
- with-telemetry.js — runInstrumented() wrapper for script-level signals
- correlation.js, session.js, scrubber.js — utilities
Config: shared/telemetry/ikey.json carries the iKey, collector URL,
per-plugin event_stream_name, and a 'disabled' kill switch. The flag
ships true so infrastructure can land before tenant-side annotation
+ Kusto provisioning is complete; a single PR flips it false later.
Sync: shared/telemetry/sync-to-plugin.js copies lib/ + ikey.json
into <plugin>/scripts/lib/telemetry/. Adopting plugins never
hand-edit the synced copy.
Tests: full node:test coverage for every module plus the dispatcher
and spawn integration (109 tests, all hermetic; envelope-shape probe
via POWER_PLATFORM_SKILLS_FAKE_HTTPS).
Docs: rebuild design spec (2026-05-04) + plugin adoption guide
(2026-04-29) under docs/superpowers/specs/. Older draft spec
(2026-04-20) removed.
* docs(telemetry): align README with code — camelCase fields, correct PAC posture
shared/telemetry/README.md drifted from the rebuild work:
- Field names listed in snake_case (plugin_name, session_id, ...) but
the code uses camelCase (pluginName, sessionId, ...). Names match
Kusto column names directly.
- "What is NEVER sent" claimed tenant IDs are not sent — but
pac-auth.js explicitly emits orgId and tenantId GUIDs from the
active PAC profile. Moved them to "What is sent" with the rest of
the PAC + agent fields.
- Missing from "What is sent": pluginVersion, osName/osVersion (was
os_family), pacCliVersion, aiAgentName, aiAgentVersion, eventInfo,
errorDescription.
- Added a note about errorDescription being a 500-char-truncated
Error.message — callers must keep PII out of exception messages.
- Documented the ikey.json "disabled" repo-side kill switch under
Privacy posture.
* refactor(telemetry): address review feedback (PR #142)
Four fixes from PR review:
1. **iKey full form in envelope** — emit-dispatcher.js now sends the
complete iKey value in the Common Schema `iKey` field
(`"o:" + IKEY`) instead of truncating to just the first
hyphen-separated segment. Aligns with the powerplatform-vscode
pattern. Multi-cloud key restructuring deferred to a follow-up PR.
2. **Drop POWER_PLATFORM_SKILLS_BYPASS_KILL_SWITCH** — production code
no longer carries a test-only escape hatch. Tests now override the
ikey.json path via POWER_PLATFORM_SKILLS_IKEY_JSON (parallels the
existing IKEY/COLLECTOR/CONFIG_DIR/FAKE_HTTPS env-var pattern) and
supply their own `disabled: false` config when they need to
exercise emission paths. emit-spawn.js forwards the new var to the
detached child.
3. **Defense-in-depth FIELD_TYPES enforcement in dispatcher** — events.js
now exports `pick`. The dispatcher imports both `FIELD_TYPES` and
`pick`, then re-runs the allowlist filter on `event.data` before
building the envelope. Any field that bypasses the builders is
dropped before it reaches the wire. The three reserved meta fields
(eventName, eventType, severity) survive via an explicit allow list.
4. **errorDescription emits err.code only, never err.message** —
with-telemetry.js no longer reads `err.message` (which can contain
file paths, GUIDs, tokens). The field now captures `err.code` —
short non-PII metadata like `ENOENT` or `ERR_INVALID_ARG_TYPE`.
Test "errorDescription truncated to 500 chars" replaced with one
that asserts a PII-bearing message is NOT emitted anywhere when
`err.code` is set. README updated accordingly.
New dispatcher test: `dispatcher strips unknown fields from event.data`
hand-crafts an event with filePath/stackTrace/rawPrompt/tokenValue
and asserts none survive the FIELD_TYPES filter.
Tests: 110/110.
* refactor(telemetry): rename ikey.json key — ikey → instrumentationKey
Per PR #142 comment 1: the JSON property is now spelled in its full
form (`instrumentationKey`) to match the powerplatform-vscode
constants pattern instead of the abbreviated `ikey`. The envelope
`iKey` Common Schema field stays at the standard tenant-routing
short form (`o:<32-char-tenant>`); the previous attempt to put the
full hyphenated key in the envelope is reverted because the full
form already lives in the `x-apikey` header.
- shared/telemetry/ikey.json: `"ikey"` → `"instrumentationKey"`.
- shared/telemetry/lib/emit-from-prompt.js: read
`cfg.instrumentationKey` (other JSON keys —
`collector_url`, `event_stream_name`, `disabled` — unchanged).
- shared/telemetry/lib/emit-dispatcher.js: envelope reverted to
`iKey: "o:" + IKEY.split("-")[0]`.
- Test fixtures updated across emit-dispatcher / emit-spawn /
emit-from-prompt tests.
Multi-cloud iKey restructuring (the second half of PR #142
comment 1) remains deferred to a follow-up PR.
Tests: 110/110.
---------
Co-authored-by: Amit Joshi <amitjoshi@microsoft.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(power-pages): adopt 1DS telemetry — synced library, hooks, valid… (#153)
* feat(power-pages): adopt 1DS telemetry — synced library, hooks, validators
Wires the shared 1DS telemetry library into the power-pages plugin.
Synced copy of the library lives under scripts/lib/telemetry/ and runs
at user time. PreToolUse:Skill emits skill_started; PostToolUse:Skill
emits skill_completed wrapped around the existing validator chain.
Plugin-side additions:
- scripts/lib/telemetry/ — synced from shared/telemetry/ via
sync-to-plugin.js (12 lib modules + ikey.json carrying the
PowerPagesPluginEvent stream name + disabled:true kill switch).
- scripts/lib/telemetry-runner.js — runInstrumented() shim plus the
PowerPagesPluginEvent envelope name and tracked-script registry.
- hooks/run-skill-pretool-telemetry.js, hooks/run-skill-posttool-validation.js
— fire-and-forget telemetry around the Skill tool, preserving the
existing validator's behavior and exit code.
- hooks/hooks.json — registers the two Skill hooks. (UserPromptSubmit
is registered separately by the slash-command branch.)
- Validators (validate-activation/seo/site/webroles/audit/auth/datamodel/
cloudflow/serverlogic/webapi-integration) and four other plugin
scripts (check-activation-status, clear-site-cache, render-audit-report,
verify-dataverse-access) wrapped with runInstrumented() so each
script run produces script_started/script_completed signals.
- scripts/tests/{telemetry-hook-pretool,telemetry-hook-posttool,telemetry-runner}.test.js
— hermetic node:test coverage for the new wiring.
Repo updates:
- AGENTS.md (plugin) gains the Telemetry section pointing at
shared/telemetry/ as the canonical source of truth.
- README.md (root) mentions anonymous telemetry + env opt-out.
- docs/superpowers/handoff/PowerPagesPluginEvent-annotation.xml
is the tenant-team handoff describing the EventStreamingAnnotation
for the per-plugin Kusto stream.
Privacy posture: default-on, env-var opt-out only
(POWER_PLATFORM_SKILLS_TELEMETRY=0). No first-run prompt, no
consent file. Telemetry stays gated by ikey.json's disabled:true
flag until the tenant-side annotation + Kusto table are provisioned.
* fix(power-pages): telemetry-runner — pass envelopeName, read instrumentationKey
Two regressions caught during E2E verification of the local-log path:
1. **envelopeName was never forwarded** — `runInstrumented()` called
`withTelemetry()` without an `envelopeName`, so events shipped with
`name: ""`. In production this means the tenant's
`EventStreamingAnnotation name="^PowerPagesPluginEvent$"` matcher
silently drops every event (`acc:1` from the collector but zero rows
in Kusto). Now reads `event_stream_name` from `ikey.json` and passes
it through.
2. **iKey read the old JSON property** — `deps.ikeyCfg.ikey` was
undefined after the `ikey` → `instrumentationKey` rename in the
foundation library (PR #142). The dispatcher saw an empty
`POWER_PLATFORM_SKILLS_IKEY` and always took the placeholder /
local-log path, even with a real iKey provisioned. Now reads
`deps.ikeyCfg.instrumentationKey`.
Adds an opt-in `_overrides` test seam on `runInstrumented()` so unit
tests can inject a fake `withTelemetry` and assert opts are propagated
correctly. New tests:
- `runInstrumented forwards envelopeName from ikey.json event_stream_name`
- `runInstrumented reads iKey from instrumentationKey property (not legacy 'ikey')`
Verified end-to-end by piping a synthetic event directly to the
dispatcher: envelope `name` is now `"PowerPagesPluginEvent"`,
`scriptName` is correctly set, event lands in the local JSONL.
Tests: 4/4 telemetry-runner, full plugin suite still green.
* telemetry: detect GitHub Copilot CLI as ai agent host
readAiAgent now recognizes COPILOT_CLI=1 and reads COPILOT_CLI_BINARY_VERSION, in addition to Claude Code. Priority remains: explicit AI_AGENT_NAME env > Claude Code > Copilot CLI. Adds test coverage and re-syncs into plugins/power-pages.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(telemetry): drop power-pages testing iKey from shared/telemetry/ikey.json
`shared/telemetry/ikey.json` is the dev-time placeholder copied by
`sync-to-plugin.js`. It was committed with power-pages's real testing
iKey and INT collector URL, which meant any future adopter
(canvas-apps, code-apps, model-apps) running the sync would inherit
those values and — if they ever flipped `disabled: false` without
replacing the iKey — silently route their telemetry to power-pages's
Kusto stream.
This commit replaces both with explicit placeholders so every plugin
is forced to provision its own cluster config before going live. The
plugin's own synced copy at `plugins/power-pages/scripts/lib/telemetry/ikey.json`
keeps its real plugin-specific values; only the shared template is sanitized.
- instrumentationKey: <pp's value> → "PLACEHOLDER_REPLACE_BEFORE_SHIPPING"
- collector_url: <INT URL> → ""
`event_stream_name` ("PluginEventStreamPlaceholder") and `disabled: true`
were already plugin-agnostic and stay unchanged.
Tests: 110/110 shared (no behavior change — the placeholder iKey
constant matches the dispatcher's existing keyMissing check).
* fix(power-pages): address PR #153 review — await, dedup, hook fast-path
Three review fixes from priyanshu92 on PR #153:
1. **telemetry-runner.js: `return await asyncFn()`** — added the `await`
on the no-telemetry-deps fallback path so stack traces include the
`runInstrumented` frame. Behavioural no-op; debug-friendliness only.
2. **validate-auth.js: drop duplicate `runInstrumented` call** — the
file invoked `runInstrumented('validate-setup-auth', main)` twice:
once unconditionally at module top-level (line 69-72), and again
inside the standard `if (require.main === module)` guard (line
137-142). When executed as a script (the actual PostToolUse-hook
use case), `main()` ran twice — duplicating validator I/O and
emitting two `script_started`/`script_completed` pairs per run.
When required as a module (tests), the top-level call still fired,
breaking import semantics. Deleted the top-level invocation, kept
the require.main guard.
3. **Skill hooks: fast-path opt-out / kill switch BEFORE pac shell-outs**
— `run-skill-pretool-telemetry.js` and `run-skill-posttool-validation.js`
ran `pac auth who` (3s timeout) and `pac --version` (2s timeout)
unconditionally on every tracked Skill invocation, even when the
user had opted out (`POWER_PLATFORM_SKILLS_TELEMETRY=0`) or the
shipped kill switch was on (`ikey.json` `disabled: true`). Worst
case ~10s added latency per Skill tool use, currently hit by every
user since `disabled: true` is the shipped default.
Both hooks now check, in this order, BEFORE any pac shell-out:
a) env opt-out (`POWER_PLATFORM_SKILLS_TELEMETRY === "0"`)
b) repo-side kill switch (`ikey.json` `disabled === true`)
c) unconfigured (`!ikey.instrumentationKey`)
Any of those → exit early. The posttool hook still preserves the
validator's exit code regardless of telemetry state.
`readIkey()` in the pretool hook now also exposes the `disabled`
flag so the gate can read it without re-parsing.
Comment #3 (extend agent-info detection to Codex / OpenCode) is
pending reviewer confirmation of those hosts' env-var conventions —
not implemented here. Any host can already self-identify via the
AI_AGENT_NAME / AI_AGENT_VERSION override.
Tests: 10/10 plugin telemetry hooks + runner. Shared suite unchanged.
---------
Co-authored-by: Amit Joshi <amitjoshi@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(power-pages): emit skill_started for slash-command invocations (#168)
Adds telemetry for skills invoked via slash command (e.g.
/power-pages:add-seo) at the user-prompt boundary, which the
existing PreToolUse:Skill hook does not cover (that fires only for
programmatic Skill tool calls).
- hooks/run-user-prompt-telemetry.js: UserPromptSubmit hook that
matches /<plugin>:<skill> via shared/telemetry/lib/prompt-detector.js
and calls emitSkillStartedFromPrompt() with PAC auth + agent info
enrichment. Fire-and-forget; never blocks prompt submission.
- hooks/hooks.json: registers the new UserPromptSubmit hook alongside
the existing PreToolUse / PostToolUse:Skill hooks.
- scripts/tests/run-user-prompt-telemetry.test.js: hermetic coverage
for the hook (tracked-skill match, no-match no-op, malformed input,
env opt-out).
- docs/superpowers/specs/2026-04-23-slash-command-telemetry-design.md:
design doc covering why this hook exists and how it composes with
the Skill-tool path.
skill_completed remains the responsibility of the PostToolUse:Skill
hook for skills that flow through the Skill tool. For slash-only
skills, completion is tracked indirectly via script-level
runInstrumented() spans.
Co-authored-by: Amit Joshi <amitjoshi@microsoft.com>
* docs(telemetry): remove implementation specs + handoff XML
The 1DS telemetry foundation has landed and is in active use. The design
specs (slash-command-telemetry-design, telemetry-rebuild-design, plugin
adoption guide) and the PowerPagesPluginEvent annotation XML were
implementation-time artifacts — the live integration is now documented
in shared/telemetry/README.md, which covers what is sent, the privacy
posture, and how to adopt the library in a new plugin.
Updates plugins/power-pages/AGENTS.md to point readers at the README
instead of the deleted spec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(telemetry): honor disabled flag before PAC + spawn in user-prompt + script paths
The pretool and posttool hooks already short-circuit on cfg.disabled
before invoking pac auth who (~3s) and pac --version (~2s). The two
remaining entry points — emit-from-prompt (used by the UserPromptSubmit
hook) and telemetry-runner (used by scripts wrapped in runInstrumented)
— were calling PAC and spawning the detached dispatcher regardless;
the dispatcher caught disabled later but the user had already paid the
shellout cost.
Add three fast-path gates at both entry points, in this order:
- POWER_PLATFORM_SKILLS_TELEMETRY=0 (env opt-out)
- cfg.disabled === true (repo-side kill switch)
- empty instrumentationKey (unprovisioned)
Any of these returns immediately and runs the user's original async fn
(telemetry-runner) or a no-op result (emit-from-prompt). Zero PAC, zero
agent-info, zero spawn.
Also fix a pre-existing destructuring bug in
emit-from-prompt.test.js's mkTelemetryDir helper: callers passed
'instrumentationKey' but the helper destructured 'ikey', so every test
ran with an empty iKey field by accident. Fix the helper and add
test coverage for all three new fast paths in both modules.
Synced shared/telemetry/lib/emit-from-prompt.js into the power-pages
plugin copy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(telemetry): rewrite README as integration guide for new adopters
Reorganize around the questions a plugin author actually asks:
- What does the library do? (data flow diagram)
- What goes on the wire? (fields + privacy posture)
- Where do the modules live? (lib/ layout)
- How do I adopt this in my plugin? (5 numbered steps)
- How do I update the shared lib without clobbering my ikey.json?
- What test seams exist?
Drop the dangling reference to the deleted design spec. The new
"Adopting in a new plugin" section walks through sync, ikey.json
config, hook registration, optional script wrapping, and local
verification — covering everything a new plugin needs without
requiring the implementation history.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(telemetry): make PAC shellouts actually capture data on Windows
Two real bugs causing every emitted event to silently drop orgId,
tenantId, and pacCliVersion:
1) pac-auth.js TIMEOUT_MS was 3000, but `pac auth who` cold-start on
Windows consistently takes 3.5-4s (.NET runtime + cached-token
validation). Every call timed out, the catch handler swallowed it,
readPacAuth returned null, and the hooks emitted events without
the PAC identity fields. Bumped to 8000 — still well under the
30s hook budget.
2) agent-info.js readPacCliVersion ran `pac --version`, but PAC 2.x
treats `--version` as an unknown command: it prints the version
banner ("Version: 2.7.4+g06bb2eb ...") to stdout but exits with
status 1. execFileSync throws on non-zero exit, the old catch
discarded everything, and the version was lost. Now we read
err.stdout from the thrown error and parse the banner. Tightened
the regex from a bare semver match to one anchored on the
"Version:" prefix so we don't accidentally pick up the .NET
Framework version from the same line. Bumped its timeout to 8s
too.
Verified on this machine: cold-start total of both reads is ~5.7s,
returning real orgId/tenantId/cloud and pacCliVersion "2.7.4".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(telemetry): stabilize sessionId across hook invocations in a Claude Code session
session.getSessionId() cached a random UUID in module-level state, but
each hook spawns as a fresh Node process — the cache reset every
invocation, so every event in a Claude Code session carried a different
sessionId. That broke session-scoped joins in Kusto.
Claude Code sends session_id in the hook stdin payload. Wire it
through:
- session.js: getSessionId(override?) — if a truthy string is passed,
prime the cache with it. Subsequent calls (with or without override)
return the primed value. _resetCache() exported for tests.
- pretool / posttool / user-prompt hooks: extract parsed.session_id
from the hook payload and pass it to getSessionId / through
emitSkillStartedFromPrompt opts.
- emit-from-prompt.js: accept opts.sessionId and forward to
getSessionId.
Synced into the plugin copy. New session tests cover override winning
over a previously-cached UUID, multi-process stability when callers
pass the same primed value, and falsy-override safety.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(telemetry): design spec for skill-only events + session/correlation cleanup
Records the audit findings and approved design for collapsing the
telemetry event vocabulary to skill_started + skill_completed only.
Removes the script-wrapping path (with-telemetry, telemetry-runner,
runInstrumented) and adds a TTL sweep to correlation.js to bound
leaks when PostToolUse never fires. Ratifies the just-shipped
sessionId fix that sources Claude Code's session_id from hook stdin.
Finding E (missing skill_completed in local trace) deferred to a
separate investigation after this cleanup lands.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(telemetry): make sessionId sourcing host-agnostic (Claude Code + Copilot CLI)
Per review feedback: sessionId should come from whichever host agent
the user is running under. Add extractSessionIdFromPayload helper that
checks both session_id (Claude Code convention) and sessionId
(camelCase fallback for hosts that use that). aiAgentName already
detects both hosts via agent-info.js; this brings sessionId into the
same multi-host pattern.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(telemetry): resolve session_id from COPILOT_SESSION_ID env for Copilot CLI
Copilot CLI exposes its session ID via the env var COPILOT_SESSION_ID,
not through the hook stdin payload. Rename the helper to
resolveHostSessionId and have it consult, in precedence order:
payload.session_id, payload.sessionId, env.COPILOT_SESSION_ID, empty.
Each hook stays a single line; new hosts can be added by extending
the resolver without touching hook code.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(telemetry): drop COPILOT_SESSION_ID env path; both hosts use payload
Per user feedback: GitHub Copilot CLI uses the same hook stdin payload
mechanism as Claude Code to surface its session id. Drop the
COPILOT_SESSION_ID env var fallback; resolveHostSessionId now only
inspects payload.session_id and payload.sessionId. Simpler resolver,
one fewer test, no env coupling.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(telemetry): implementation plan for skill-only events + session/correlation cleanup
Bite-sized TDD plan covering: resolveHostSessionId helper, correlation
TTL sweep, removal of buildScript builders, deletion of with-telemetry
+ telemetry-runner, unwrap runInstrumented from 10 validators + 4
standalone scripts, switch 3 hooks to resolveHostSessionId, sync,
README update, manual verification, push. 12 tasks total. Spec
reference: docs/superpowers/specs/2026-05-28-session-correlation-cleanup-design.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(telemetry): add resolveHostSessionId helper for multi-host session sourcing
* feat(telemetry): TTL-sweep stale correlation files in correlation.write
* feat(telemetry): drop buildScript builders + scriptName field
* feat(telemetry): delete with-telemetry script wrapper
* feat(power-pages): delete telemetry-runner plugin wrapper
* feat(power-pages): unwrap runInstrumented from validators
* feat(power-pages): unwrap runInstrumented from standalone scripts
* feat(power-pages): hooks use resolveHostSessionId for multi-host session sourcing
* chore(power-pages): sync session/correlation cleanup into plugin copy
* docs(telemetry): drop script-wrapper section from README; align with skill-only events
* fix(power-pages): hoist input parse so posttool emits skill_completed
The const input was scoped to the first try block while the telemetry
block (second try) also references it for resolveHostSessionId. The
ReferenceError was silently swallowed by the outer telemetry catch,
preventing all skill_completed events from emitting. This is the root
cause of Finding E (9 skill_started, 0 skill_completed). Hoist to outer
async scope so both blocks see the parsed payload.
* fix(telemetry): backfill aiAgentVersion from CLAUDECODE when only AI_AGENT_NAME is set
If the host sets AI_AGENT_NAME (e.g., via .claude/settings.local.json)
but omits AI_AGENT_VERSION, the previous code returned aiAgentVersion=""
and the hook silently dropped the field. Now: when AI_AGENT_NAME is
explicit but the version is empty, fall through to the same built-in
detection used when AI_AGENT_NAME isn't set at all — read the version
from CLAUDE_CODE_EXECPATH/../package.json for Claude Code, or
COPILOT_CLI_BINARY_VERSION for Copilot CLI.
Explicit AI_AGENT_VERSION still wins over the backfill when present.
Extracted readClaudeCodeVersion(env) helper to share the package.json
read path between the explicit-name and built-in detection branches.
Synced into the plugin copy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(telemetry): correct aiAgent detection description in README
Reflect actual agent-info logic: Claude Code version is read from the
installed package.json via CLAUDE_CODE_EXECPATH, Copilot CLI version from
COPILOT_CLI_BINARY_VERSION, and an explicit AI_AGENT_NAME backfills its
version from the matching built-in detector when AI_AGENT_VERSION is empty.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(telemetry): remove session-correlation cleanup design + plan
The session/correlation cleanup work has shipped; drop the superpowers
design spec and implementation plan that tracked it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* revert(power-pages): restore main's validators + standalone scripts
These 10 validators and 4 standalone scripts diverged from main only by
the leftover runInstrumented-unwrap shape (async main() wrapper +
require.main guard + module.exports). That structure is a vestige of an
earlier per-script telemetry approach that was dropped in favor of
hook-based skill telemetry, so it's no longer needed. Restore main's
versions to keep the telemetry PR scoped to telemetry-only changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(telemetry): derive Claude Code version from AI_AGENT on non-npm installs
aiAgentVersion was derived solely from CLAUDE_CODE_EXECPATH -> ../package.json,
which only exists in the npm-global install layout. On native-installer Claude
Code (standalone binary, now the default), that read fails and the version is
dropped while aiAgentName still logs as "Claude Code" -- name present, version
blank.
Add an install-method-independent fallback: when the package.json read yields
nothing, parse the dotted semver out of AI_AGENT (claude-code_<maj>-<min>-<patch>_agent),
which Claude Code sets regardless of install method. The npm path stays primary,
so npm installs are unaffected.
Edited shared/telemetry and re-synced to plugins/power-pages. Adds test coverage
for native-install fallback, EXECPATH-unset fallback, npm-wins precedence, and
malformed AI_AGENT.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(telemetry): address Copilot review on PR #176
- emit-dispatcher: pass url.port to https.request so a non-443 collector
URL POSTs to the configured port instead of always 443
- events: tighten isPlainStructured() to only accept arrays and plain
objects (prototype Object.prototype/null), rejecting class instances
like Error from the dynamic eventInfo field
- emit-from-prompt + posttool hook: add POWER_PLATFORM_SKILLS_IKEY_JSON
override seam so tests point at a temp ikey.json instead of mutating
the checked-in config (removes a parallel-test race)
- posttool hook: skip telemetry emission when no tracked skill was
resolved (malformed stdin / pre-detection error) so no incomplete
skill_completed event is emitted; exit code unchanged
- run-user-prompt-telemetry test: replace busy-wait loops with a
non-spinning Atomics.wait sleep + waitForFile helper, and use the
ikey override instead of mutating the shipped ikey.json
Shared lib edits propagated to the power-pages synced copy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(telemetry): design for removing skill_completed logging flow
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(telemetry): stop emitting skill_completed from posttool hook
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(telemetry): inline correlationId in pretool hook, drop disk correlation
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(telemetry): delete unused correlation.js and its test
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(telemetry): remove orphaned synced correlation.js
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(telemetry): document removal of skill_completed flow
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(telemetry): describe only live telemetry; drop skill_completed references
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: remove brainstorm/plan docs for skill_completed removal
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Clarify shared telemetry README event scope
* fix(telemetry): preserve real ikey on resync; drop unused test consent artifact
Address PR #176 review:
- sync-to-plugin.js no longer clobbers a provisioned ikey.json. It is
preserved when the target carries a non-placeholder instrumentationKey
and only seeded when missing or still on the placeholder, so the
documented `sync-to-plugin --target plugins/power-pages` is safe to
re-run. Adds preserve-on-resync test coverage.
- run-user-prompt-telemetry.test.js: simplify mkConfigDir to a tmpdir
(the telemetry.json consent file and `enabled` arg were never read;
live gates are POWER_PLATFORM_SKILLS_TELEMETRY=0 and ikey.json.disabled).
- Raise the enabled-path spawn timeout 10s->30s to match the hook's
documented pac budget; the 10s timeout flaked on the ~10s pac shellout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(telemetry): fail closed on unreadable kill-switch config; refresh README
Address PR #176 second review pass:
- emit-dispatcher.js isDisabledByConfig() now fails CLOSED when ikey.json
is missing/unreadable (treat as disabled) instead of fail-open. The
config is a kill switch; if it can't be read we can't confirm emission
is authorized, so we suppress rather than risk a POST / local log in an
unexpected state. Matches the repo's documented "fail closed" telemetry
posture. Adds an emit-dispatcher test covering the missing-config path.
Synced to the power-pages copy via sync-to-plugin.js.
- README: the sync section claimed sync overwrites ikey.json and to
git-checkout it back; sync now preserves a provisioned key, so describe
the preserve/seed behavior instead.
- README: corrected the session.js test seam entry — session.js has no
opts.configDir / filesystem state; the seams are _resetCache() and the
getSessionId(override) parameter.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(telemetry): propagate fail-closed + ikey override seam to readIkey copies
Self-review follow-up to e20c6d3 (which only fixed emit-dispatcher.js):
- emit-from-prompt.js and the run-skill-pretool-telemetry.js hook now fail
CLOSED (disabled: true) when ikey.json is missing/unreadable, matching
emit-dispatcher.js's isDisabledByConfig(). The `ikey: ""` guard already
blocked emission, so this is a semantics fix that keeps all three readIkey
paths consistent and prevents the kill switch from being bypassed by a
missing/corrupt config.
- The pretool hook's readIkey() now honors POWER_PLATFORM_SKILLS_IKEY_JSON,
the same test/override seam the dispatcher and emit-from-prompt already
respect. This lets a test redirect ikey.json without mutating the
checked-in file (avoiding the parallel-test race fixed elsewhere).
- Adds pretool-hook tests covering the enabled emit path (via the override
seam) and the fail-closed path (missing override file → no emit).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(telemetry): make readIkey self-protecting against missing telemetryDir
emit-from-prompt.js readIkey() computed path.join(telemetryDir, "ikey.json")
outside the try, so a caller that omits telemetryDir (with no IKEY_JSON
override) would throw a TypeError instead of failing closed — contradicting
the library's fail-closed contract. Move path resolution inside the try so any
bad input falls through to disabled: true. Production callers already pass
telemetryDir and wrap the call, so this is defense-in-depth at the library
boundary. Adds a test for the missing-telemetryDir path. Synced to power-pages.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(telemetry): detect bare /<skill> slash commands, not just /power-pages:<skill>
Per review feedback: the host tool controls how a slash command is surfaced
in the UserPromptSubmit payload and may emit the bare form (`/add-seo`)
without the plugin namespace. Make the `pluginName:` prefix optional in the
detector regex so both forms are recognized. The captured skill name is still
validated against trackedSkills, so a bare `/foo` only counts when `foo` is a
known skill, and the trailing boundary lookahead still rejects another
plugin's namespaced command (`/other-plugin:add-seo` → null). Adds tests for
the bare form, non-tracked bare commands, and bare substrings. Synced to
power-pages.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(telemetry): update power-pages ikey.json to int collector + PagesAIPluginEvent stream
Points the synced config at the int 1DS instrumentation key / us-mobile
collector and renames the event stream to PagesAIPluginEvent. Stays
disabled: true (inert) until the tenant-side stream is provisioned.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(telemetry): broaden skill tracking
- Detect additional AI agent hosts from hook env variables
- Derive Power Pages tracked skills from skill folders
- Discover optional validators and refresh hook docs/tests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(telemetry): write a local diagnostic mirror of every event
Always append a faithful, on-disk mirror of each emitted event to
~/.power-platform-skills/events.jsonl, irrespective of iKey presence and
irrespective of the POWER_PLATFORM_SKILLS_TELEMETRY=0 opt-out.
- Mirror shape is {time, name, data} where `data` is the sanitized payload
(== Kusto column names) and `time` matches the wire envelope; the dispatcher
computes both once and shares them with the POST so they can never diverge.
- Previously the local log was written ONLY when the iKey was missing; a
real-iKey event was transmitted but never recorded locally. Now it is mirrored
on every path that clears the kill switch.
- POWER_PLATFORM_SKILLS_TELEMETRY=0 now suppresses TRANSMISSION only — the local
mirror is still written. The opt-out fast-paths were removed from
emit-from-prompt.js and run-skill-pretool-telemetry.js so the enriched event
still reaches the dispatcher; it writes the mirror then skips the POST.
Trade-off: opted-out runs now incur the same event-building (pac) cost as
enabled runs.
- `disabled: true` in ikey.json remains the one true hard-off (no mirror,
no POST), checked before any side effect.
- Spell out POWER_PLATFORM_SKILLS_TELEMETRY=0 in full everywhere (was mixed with
TELEMETRY=0 shorthand) so it is not confused with the ikey.json `disabled` flag.
- Update README flow + privacy posture; add/adjust tests (mirror==wire equality,
opt-out-still-mirrors); re-sync shared/telemetry into the power-pages plugin.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* experiment(telemetry): symlink plugin lib to shared/telemetry/lib instead of copying
Replaces the synced copy at plugins/power-pages/scripts/lib/telemetry/lib
with a relative symlink -> ../../../../../shared/telemetry/lib. ikey.json is
kept as a real file (it carries the plugin's real config, not shared's
placeholder). Git tracks the link as mode 120000; Node require() resolves
through it and all 12 telemetry hook tests pass.
KNOWN CAVEAT (not yet addressed): emit-dispatcher.js finds its kill-switch
config via path.join(__dirname, '..', 'ikey.json'). Node resolves __dirname
through the symlink to the REAL dir (shared/telemetry/lib), so the default
resolves to shared/telemetry/ikey.json (the placeholder, disabled:true) rather
than the plugin's real ikey.json. Masked today (both disabled:true); once the
plugin config is flipped to disabled:false this would keep emission suppressed.
A complete symlink implementation must pass POWER_PLATFORM_SKILLS_IKEY_JSON
(the plugin's real ikey.json) through emit-spawn so the dispatcher reads the
right config regardless of realpath.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(telemetry): dispatcher reads the calling plugin's ikey.json under shared lib
When lib/ is shared (symlink or relative require), emit-dispatcher.js runs from
shared/telemetry/lib, so its __dirname-based default kill-switch config resolves
to shared/'s placeholder (disabled:true) instead of the plugin's real ikey.json.
Once a plugin flips its config to enabled, emission would stay suppressed.
Fix: emit-spawn forwards POWER_PLATFORM_SKILLS_IKEY_JSON from a new
opts.ikeyJsonPath (an explicit env override still wins, for tests); the pretool
hook and emit-from-prompt pass the calling plugin's ikey.json path. Adds an
emit-spawn regression test that drives the opts path with no env override.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(telemetry): design spec for interactive telemetry toggle
Brainstormed design for a per-plugin `/<plugin>:telemetry on|off|status`
command backed by a host-neutral ~/.power-platform-skills/config.json
(per-plugin keys). Replaces the POWER_PLATFORM_SKILLS_TELEMETRY env var with the
config file as the single user off-switch; "off" stops transmission only (local
mirror kept). Internal ikey.json `disabled` flag stays hidden; every command
prints an anonymity reassurance. Pre-implementation; no code changed yet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(telemetry): implementation plan for the telemetry toggle
8-task TDD plan from the approved spec: new user-config.js module + CLI,
dispatcher gate rewired to per-plugin config.json (env var removed), shared
telemetry skill + power-pages wrapper, tracking exclusion, user-facing README,
re-sync + live verification.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(telemetry): add user-config module for per-plugin opt-out
* feat(telemetry): gate transmission on per-plugin config instead of env var
* feat(telemetry): add CLI to read/write the telemetry toggle
* feat(telemetry): add shared telemetry skill + power-pages wrapper
* feat(telemetry): exclude the telemetry skill from usage tracking
* refactor(telemetry): remove POWER_PLATFORM_SKILLS_TELEMETRY env var entirely
* docs(telemetry): document the telemetry toggle in the user-facing README
* chore(telemetry): sync user-config + CLI into power-pages
* chore(telemetry): remove design spec and implementation plan docs
* refactor(telemetry): bundle skill workflow via symlink like report-issue
Switch the telemetry skill from a repo-root-relative workflow reference
(${CLAUDE_PLUGIN_ROOT}/../../shared/...) to the bundled-path pattern used
by report-issue: SKILL.md points at ${CLAUDE_PLUGIN_ROOT}/skills/telemetry/
telemetry-workflow.md, backed by a per-plugin symlink to the shared source.
Marketplace installs copy only the plugin directory, so the old ../../shared
path did not resolve at user time. The symlink keeps a single source of truth
in-repo while letting installers dereference it into the plugin cache.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(telemetry): finalize symlink approach (drop sync-to-plugin, update docs)
Make the symlink the single source of truth for the shared telemetry lib,
replacing the synced-copy mechanism:
- remove shared/telemetry/sync-to-plugin.js and its test (superseded by the
per-plugin lib symlink; the script would have copied lib/ onto itself)
- rewrite the telemetry docs to the symlink model across root AGENTS.md,
plugins/power-pages/AGENTS.md, and shared/telemetry/README.md, including the
symlink adoption recipe and the ikey.json posture rule (committed stays
disabled:true; working-tree disabled:false is a local experiment only)
- gitignore .omc/ scratch directories
Verified: shared telemetry suite 147/147, power-pages telemetry hooks 23/23.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(telemetry): address PR review (array config, null-proto skills, test comment)
- user-config.readConfig: ignore an array config.json. typeof [] === "object"
passed the old guard, so setTelemetryChoice set .telemetry on the array and
JSON.stringify silently dropped it — returning true while persisting nothing.
Now guarded with !Array.isArray + regression test.
- powerpages-hook-utils: build TRACKED_SKILLS with Object.create(null). Membership
is tested via bracket access, so a plain {} let inherited keys (toString,
constructor, __proto__) test truthy and emit bogus skill names. Added a test.
- events.test.js: fix the misleading "drop guarded by clamp" comment — boolean
true is kept and coerced to 1 (Number(true)), not dropped.
Addresses copilot review comments on PR #176. Shared telemetry 148/148,
power-pages telemetry hooks 24/24.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Amit Joshi <amitjoshi@microsoft.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Priyanshu Agrawal <priyanshuag@microsoft.com>1 parent 36fcbc7 commit 98430e9
47 files changed
Lines changed: 4229 additions & 99 deletions
File tree
- plugins/power-pages
- hooks
- scripts
- lib
- telemetry
- tests
- skills/telemetry
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
63 | 63 | | |
64 | 64 | | |
65 | 65 | | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
66 | 74 | | |
67 | 75 | | |
68 | 76 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
211 | 211 | | |
212 | 212 | | |
213 | 213 | | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
153 | 153 | | |
154 | 154 | | |
155 | 155 | | |
156 | | - | |
| 156 | + | |
157 | 157 | | |
158 | 158 | | |
159 | 159 | | |
160 | 160 | | |
161 | | - | |
162 | | - | |
| 161 | + | |
| 162 | + | |
163 | 163 | | |
164 | | - | |
| 164 | + | |
165 | 165 | | |
166 | 166 | | |
167 | 167 | | |
| |||
342 | 342 | | |
343 | 343 | | |
344 | 344 | | |
345 | | - | |
| 345 | + | |
346 | 346 | | |
347 | 347 | | |
348 | 348 | | |
| |||
388 | 388 | | |
389 | 389 | | |
390 | 390 | | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
391 | 402 | | |
392 | 403 | | |
393 | 404 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
197 | 197 | | |
198 | 198 | | |
199 | 199 | | |
200 | | - | |
| 200 | + | |
201 | 201 | | |
202 | 202 | | |
203 | 203 | | |
| |||
410 | 410 | | |
411 | 411 | | |
412 | 412 | | |
413 | | - | |
| 413 | + | |
414 | 414 | | |
415 | 415 | | |
416 | 416 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
41 | | - | |
| 41 | + | |
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| |||
373 | 373 | | |
374 | 374 | | |
375 | 375 | | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
376 | 387 | | |
377 | 388 | | |
378 | 389 | | |
| |||
476 | 487 | | |
477 | 488 | | |
478 | 489 | | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
479 | 511 | | |
480 | 512 | | |
481 | 513 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
3 | 15 | | |
4 | 16 | | |
5 | 17 | | |
| |||
11 | 23 | | |
12 | 24 | | |
13 | 25 | | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
14 | 37 | | |
15 | 38 | | |
16 | 39 | | |
Lines changed: 21 additions & 26 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
26 | 31 | | |
27 | | - | |
28 | | - | |
| 32 | + | |
| 33 | + | |
29 | 34 | | |
30 | 35 | | |
31 | 36 | | |
32 | 37 | | |
33 | 38 | | |
34 | 39 | | |
35 | | - | |
36 | | - | |
37 | | - | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
38 | 51 | | |
39 | | - | |
40 | | - | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | 52 | | |
60 | 53 | | |
61 | | - | |
| 54 | + | |
62 | 55 | | |
| 56 | + | |
| 57 | + | |
63 | 58 | | |
0 commit comments