You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Browse filesBrowse the repository at this point in the historyBrowse files
Nikhil Agrawal
committed
feat(mobile-apps): emit skill-started telemetry from plugin hooks
Wires the Mobile Apps plugin into the vendored 1DS telemetry library so we
can see which mobile skills developers actually run.
- Register two hooks: PreToolUse(Skill) for agent-invoked skills and
UserPromptSubmit for manual slash commands, which Copilot pre-expands
into <skill-context> and never reports as a Skill tool call.
- Add mobile-telemetry.js, the plugin adapter that gates on a provisioned
ikey, resolves the session, and builds the skill-started event.
- Correlate nested Copilot agents, which receive a transient call_* session
id, back to their owning UUID session via a bounded tail read of the local
session-state log, cached as a short-lived alias file. Fails open.
- Discover tracked skills from the skills directory instead of hardcoding.
- Add the telemetry skill so users can inspect or opt out.
- Add script tests plus a CI workflow that sets the plugin opt-out env var
so test runs never post to the production collector.
scripts/ ← shared helpers, including validate-mobile-files.js for skill-owned changed-file validation
30
-
hooks/ ← Validator implementations invoked explicitly by mobile workflows
29
+
scripts/ ← shared helpers, including validate-mobile-files.js and bundled telemetry
30
+
hooks/ ← Telemetry start hooks plus validators invoked explicitly by mobile workflows
31
31
```
32
32
33
33
## Template source
@@ -56,7 +56,7 @@ Do not add preparation rewrites for `scheme`, `package`, `bundleIdentifier`, `sr
56
56
7.**Persisted plan** — Write `native-app-plan.md` (Mermaid ER + per-screen specs + native capabilities matrix) as the source of truth that sub-skills `Read`.
57
57
8.**CLI compatibility** — Use `npx power-apps ...` for code-app lifecycle and data-source commands. Use `scripts/resolve-environment.js` plus `az` tokens for Dataverse environment URL/tenant discovery and Azure/Entra operations. See [`shared/shared-instructions.md`](./shared/shared-instructions.md).
58
58
9.**Agent invocation namespace** — All `Task` invocations of agents in this plugin MUST use the fully-qualified `mobile-app:<agent-name>` form (e.g. `mobile-app:native-app-planner`, `mobile-app:screen-builder`). Bare names like `native-app-planner` return `Agent type 'native-app-planner' not found` because Claude Code namespaces all plugin agents by plugin name.
59
-
10.**Plugin isolation** — Do not add `hooks/hooks.json`: Claude loads plugin hooks during unrelated workflows, so a mobile write hook can block Canvas Apps tool calls. Mutating skills follow the changed-file gate in `shared/shared-instructions.md`, and final-artifact agents invoke `scripts/validate-mobile-files.js` directly.
59
+
10.**Plugin isolation** — `hooks/hooks.json` is limited to fail-open telemetry start hooks. They never validate, mutate, or block tool calls. Do not add write/validation hooks: mutating skills follow the changed-file gate in `shared/shared-instructions.md`, and final-artifact agents invoke `scripts/validate-mobile-files.js` directly.
60
60
11.**Invocation metadata** — Public entry skills use `user-invocable: true` and remain model-invocable. Bundled implementation helpers use both `user-invocable: false` and `disable-model-invocation: true`; their owner reads `SKILL.md` directly. Hidden standalone workflows such as `assign-offline-profile` and `preview-offline-scope` use `user-invocable: false` without disabling model invocation because no owner reads them directly. Agents use `user-invocable: false` without `disable-model-invocation` so qualified `Task` delegation remains available.
61
61
12.**Sub-agent return-status protocol** — Every agent in this plugin (`native-app-planner`, `data-model-architect`, `screen-planner`, `screen-builder`) MUST return a status code as the **literal first line** of its final message. Orchestrators (skills that invoke agents via `Task`) MUST parse the first line and branch:
62
62
@@ -74,6 +74,17 @@ Do not add preparation rewrites for `scheme`, `package`, `bundleIdentifier`, `sr
74
74
- Special early-return signals (`INDUSTRY_CONFIRM_REQUESTED:`, `DESIGN_VIBE_REQUESTED:`) pre-date this protocol and remain in effect — they are special-cased "ask the user one question and re-spawn me" handoffs, not terminal returns.
75
75
- The canonical orchestrator handler lives in [`skills/create-mobile-app/SKILL.md`](./skills/create-mobile-app/SKILL.md) Step 3.0. Future skills that spawn agents should reference it rather than duplicating the switch.
76
76
77
+
## Telemetry
78
+
79
+
Mobile Apps bundles the canonical stdlib-only 1DS transport from the repo-root `shared/telemetry/lib` at `scripts/lib/telemetry/lib`. Edit the shared source first, then refresh this physical copy in the same change; never copy another plugin's `ikey.json` or resolver.
80
+
81
+
-**Start-only lifecycle:**`UserPromptSubmit` records explicit slash-command starts and `PreToolUse(Skill)` records programmatic Skill-tool starts; both may fire for one visible slash command. `UserPromptSubmit` payloads differ by host — Claude Code passes the raw `/mobile-app:<skill>` text, Copilot CLI pre-expands it to a `<skill-context name="<skill>">` wrapper and emits no Skill pre-tool event — so both shapes must stay recognized or manual runs go uncaptured. Do not add `skill_completed`, duration, outcome, or persisted correlation state: Power Pages deliberately removed that flow because the hook boundary does not prove the workflow completed.
82
+
-**Coverage and attribution:**`scripts/lib/mobileapp-hook-utils.js` discovers every user- or model-invocable top-level skill, including `telemetry`. Direct-read helpers with `disable-model-invocation: true` are not independently invoked and are excluded. Bare and `mobile-app:`-qualified names are both attributed; explicitly foreign plugin namespaces are excluded.
83
+
-**Session correlation:** Stable host session ids pass through unchanged. Copilot CLI reports a transient `call_*` id to nested-agent hooks, so `resolveCopilotRootSessionId` in `scripts/lib/mobile-telemetry.js` resolves it to the unique recent UUID session whose local `~/.copilot/session-state/<uuid>/events.jsonl` structurally owns that `agentId`, reading only a bounded tail. Keep host-specific quirks contained in that one function. The verified root is cached as one atomic 30-minute alias file per hashed call id so fresh hook processes reuse it; aliases hold no prompts, cwd, or tool arguments and are never transmitted. Missing, stale, malformed, or ambiguous state fails open to the original id, and Claude Code and Codex ids are not rewritten.
84
+
-**Privacy:** Mobile Apps sends no prompt, tool input, cwd, path, URL, credential, username, hostname, Dataverse org/tenant ID, or Entra object ID. The dynamic `eventInfo` contains only `invocationSource` (`prompt` or `pretool`).
85
+
-**Controls:**`scripts/lib/telemetry/ikey.json` remains `disabled: true` with a Mobile-specific placeholder until its own 1DS key, collector, stream annotation, and Kusto mapping are provisioned. That is a true hard-off: no local log and no POST. Once enabled, `/mobile-app:telemetry off` and `POWER_PLATFORM_SKILLS_TELEMETRY_MOBILE_APP_OPTOUT=1` suppress transmission while preserving the local diagnostic mirror.
86
+
-**CI:** Every Mobile Apps test job must set `POWER_PLATFORM_SKILLS_TELEMETRY_MOBILE_APP_OPTOUT=1`. The single positive wire test clears that backstop only in its child process and routes the event to `POWER_PLATFORM_SKILLS_FAKE_HTTPS`; all other positive tests remain opted out and exercise the local mirror.
87
+
77
88
## Decisions made
78
89
79
90
- ✅ Markdown plan with Mermaid (no HTML rendering)
Copy file name to clipboardExpand all lines: plugins/mobile-apps/README.md
+19Lines changed: 19 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -216,6 +216,7 @@ Example edit flows:
216
216
|`/deploy`| ✅ v0 | Build + push — `npm run build`then`npx power-apps push` to the env in`power.config.json`. **Does not** drive `expo run:ios` or `expo run:android` (out of scope for v0). |
217
217
|`/open-wrap-url`| ✅ v0 | Opens the Wrap URL in browser for an app ID using `https://make.powerapps.com/environments/<envID>/wrap?appID=<appID>`. Requires both `--app-id` and `--env-id`. |
|`/telemetry`| ✅ v0 | Enable, disable, or show the per-user Mobile Apps telemetry transmission preference. |
219
220
|`/design-system`| ✅ v0 | End-to-end design system — collects brand inputs (logo, brand doc, website, free text, canvas app, code app, Figma), runs a 3-style visual picker, writes `brand/design-system.md` + `brand/tokens.ts`, renders branded screen previews. Auto-invoked at Step 6.75 of `/create-mobile-app`; also standalone. |
220
221
|`/preview-screens`| ✅ v0 | Renders generated TSX screens as a browser-viewable HTML preview (no Metro needed). Uses Tamagui → HTML mapping. |
221
222
|`/add-datasource`| ✅ v0 | Alias for`/add-connector` — discoverable name for"how do I connect to X?"|
@@ -237,6 +238,24 @@ Example edit flows:
237
238
| `screen-builder` | Mutation — writes ONE TSX file per assigned screen, runs N in parallel |
238
239
| `offline-profile-architect` | Read-only — proposes per-table row scope, relationships, selected columns, sync frequency; returns `_offline_section.md` for `/setup-offline-profile` to embed in `native-app-plan.md` |
239
240
241
+
## Telemetry and privacy
242
+
243
+
The Mobile Apps plugin includes start-only usage telemetry built on the same shared 1DS transport as Power Pages. The checked-in Mobile Apps configuration is currently `disabled: true` with a placeholder key, so it is hard-off until a dedicated Mobile Apps stream and instrumentation key are provisioned. While hard-off, it performs no telemetry shellouts, writes no local event log, and sends nothing.
244
+
245
+
Once provisioned, a start event can include the skill name, plugin version, session and per-start correlation IDs, OS/Node versions, AI-agent name/version, and whether the host observed the start through `UserPromptSubmit` or `PreToolUse(Skill)`. It never includes prompts, skill arguments, tool inputs, file paths, cwd, app/site names, URLs, credentials, usernames, hostnames, Dataverse organization or tenant IDs, or Entra object IDs.
246
+
247
+
Both host surfaces are covered — an explicit slash command and a programmatic Skill-tool call — so some hosts may produce two `skill_started` records for one visible run. The plugin does not emit `skill_completed`, success/failure, error, or duration data because the available hook boundary does not prove that the workflow itself completed.
248
+
249
+
Control the per-user transmission preference with:
250
+
251
+
```text
252
+
/mobile-app:telemetry status
253
+
/mobile-app:telemetry off
254
+
/mobile-app:telemetry on
255
+
```
256
+
257
+
After provisioning, `off` stops network transmission but retains the sanitized local diagnostic mirror under `~/.power-platform-skills/telemetry/mobile-app/sessions/<sessionId>/events.jsonl`. Automation can force transmission off with `POWER_PLATFORM_SKILLS_TELEMETRY_MOBILE_APP_OPTOUT=1`; this overrides the saved preference and `on`.
"_comment": "Telemetry-only start hooks. They never validate, mutate, or block tool calls. File validation stays owned by each workflow through scripts/validate-mobile-files.js.",
0 commit comments