Skip to content

Commit 31868b8

Browse files
committed
fix(ce-product-pulse): close schema gap and tighten config-driven prose
Addresses functional gaps and copy nits surfaced in a trace-through of both first-run and subsequent-run flows. Schema gap: - Add pulse_pending_metrics and pulse_excluded_metrics CSV keys to config-template.yaml and the rendered example. The interview promised these dispositions for un-instrumented strategy metrics, but the prior flat YAML had no slot for them. Matches the pulse_completion_events CSV pattern. - Update references/interview.md section 6.0 to write to those keys. - Update references/report-template.md to honor both keys when rendering strategy metrics. Config-driven prose tightening: - Phase 2.2: stale "see interview config" reference replaced with the precise key check (pulse_quality_scoring is true). - Phase 1.0: product-name extraction guidance now specifies stripping the trailing " Strategy" suffix from the H1 title; seed message rewritten as a directive to the agent rather than a literal-template string. Issue 1 (CWD vs repo-root for docs/ paths) was checked against repo precedent and intentionally NOT changed: ce-brainstorm, ce-plan, and ce-compound all use bare relative docs/ paths and rely on target-announcement for cross-repo cases. ce-strategy and ce-product-pulse already announce, so the existing pattern holds.
1 parent 0ef57af commit 31868b8

5 files changed

Lines changed: 20 additions & 9 deletions

File tree

.compound-engineering/config.local.example.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,3 +26,5 @@
2626
# pulse_tracing_source: sentry # sentry | datadog | custom (no default)
2727
# pulse_payments_source: stripe # stripe | custom (no default)
2828
# pulse_db_enabled: false # true | false (default: false; read-only DB if true)
29+
# pulse_pending_metrics: "retention_d7,nps" # comma-separated strategy metrics awaiting instrumentation; render as 'no data'
30+
# pulse_excluded_metrics: "north_star" # comma-separated strategy metrics intentionally not in pulse

plugins/compound-engineering/skills/ce-product-pulse/SKILL.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,8 @@ If it shows an unresolved command string, read `.compound-engineering/config.loc
7070
- `pulse_tracing_source` -- string identifying tracing provider (e.g., `sentry`, `datadog`, `custom`)
7171
- `pulse_payments_source` -- string identifying payments provider (e.g., `stripe`, `custom`); omit if not used
7272
- `pulse_db_enabled` -- `true` or default `false`; when `true`, read-only DB access is part of the pulse
73+
- `pulse_pending_metrics` -- comma-separated string of strategy-doc metric names awaiting instrumentation; rendered as `no data` in each pulse report until instrumentation lands
74+
- `pulse_excluded_metrics` -- comma-separated string of strategy-doc metric names intentionally excluded from the pulse; the metric stays in `docs/strategy.md` but is not surfaced in pulse reports
7375

7476
**Routing:**
7577

@@ -84,12 +86,12 @@ If the argument was `setup`, `reconfigure`, or `edit config`, go to Phase 1 rega
8486

8587
Before asking any questions, read `docs/strategy.md` using the native file-read tool. If the file exists, extract:
8688

87-
- The product name (from the title)
88-
- The list of key metrics (from the `## Key metrics` section)
89+
- The product name from the H1 title, stripping the trailing ` Strategy` suffix (e.g., `# Spiral Strategy` -> `Spiral`)
90+
- The list of key metrics from the `## Key metrics` section, one per line
8991

90-
Open the interview by showing the user what was pulled forward: "I see you have a strategy doc on file - I'll seed the product name as `{{name}}` and carry these key metrics into the event/data setup: {{metric list}}. Say so if any of that needs to change."
92+
Open the interview by surfacing what was extracted: announce that a strategy doc was found, show the seeded product name and the list of key metrics that will be carried into event/data setup, and invite the user to correct any of it before continuing.
9193

92-
If `docs/strategy.md` does not exist, note that too: "No strategy doc on file. Running the full setup from scratch. (If you want strategy to seed this later, run `/ce-strategy` first.)"
94+
If `docs/strategy.md` does not exist, note that explicitly in chat: no strategy doc on file, running setup from scratch, and mention that `ce-strategy` can seed pulse later if run first.
9395

9496
#### 1.1 Interview
9597

@@ -132,7 +134,7 @@ Run these **serially**, after the parallel batch:
132134

133135
#### 2.2 Optional: Sample Quality Scoring
134136

135-
If quality scoring is opted in (AI products only, see interview config), sample up to 10 sessions or conversations from the window and score each 1-5 on the dimension the user specified at setup.
137+
If `pulse_quality_scoring` is `true` (AI products only), sample up to 10 sessions or conversations from the window and score each 1-5 on the dimension recorded in `pulse_quality_dimension`.
136138

137139
**Scoring discipline:** Default to 4 or 5 when the session looks normal. Reserve 1-3 for sessions with a clear failure mode (product gave wrong answer, user got stuck, error surfaced). If every session is scoring 3, the bar is too strict; if every session is scoring 5, the bar is too loose.
138140

plugins/compound-engineering/skills/ce-product-pulse/references/interview.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -126,10 +126,10 @@ The answer produces (tool name, query shape). If multiple entries land in the sa
126126

127127
**If the user says "we don't have that instrumented yet"** (common for strategy-seeded metrics like retention or NPS): offer two off-ramps and let them pick.
128128

129-
- **Defer** - capture the metric as `instrumentation-pending` with a one-line note. The metric will appear in the config and render as `no data` in each pulse report until instrumentation lands. This is the right call if the metric matters and they'll fix it.
130-
- **Drop from pulse** - capture the metric as `strategy-only - not in pulse`. The metric stays in `docs/strategy.md` but the pulse ignores it. This is the right call if the metric is aspirational and won't have data any time soon.
129+
- **Defer** - append the metric name to `pulse_pending_metrics` (CSV). The metric renders as `no data` in each pulse report until instrumentation lands. Right call when the metric matters and the team will instrument it.
130+
- **Drop from pulse** - append the metric name to `pulse_excluded_metrics` (CSV). The metric stays in `docs/strategy.md` but the pulse skips it entirely. Right call when the metric is aspirational and won't have data any time soon.
131131

132-
Do not silently skip. An unmeasured strategy metric should either show up as a `no data` slot (forcing visibility) or be explicitly dropped.
132+
Do not silently skip. Every un-instrumented strategy metric must land in exactly one of `pulse_pending_metrics` (visible as `no data`) or `pulse_excluded_metrics` (omitted from the report).
133133

134134
### 6.1 MCP nudge
135135

@@ -242,6 +242,8 @@ pulse_analytics_source: {{posthog | mixpanel | custom | omit}}
242242
pulse_tracing_source: {{sentry | datadog | custom | omit}}
243243
pulse_payments_source: {{stripe | custom | omit}} # omit if not used
244244
pulse_db_enabled: {{true | false}} # default false; read-only DB access only
245+
pulse_pending_metrics: "{{metric,metric}}" # strategy metrics deferred for instrumentation; render as 'no data'; omit if none
246+
pulse_excluded_metrics: "{{metric,metric}}" # strategy metrics intentionally not in pulse; omit if none
245247
~~~
246248

247249
**Notes on what is NOT persisted in config:**

plugins/compound-engineering/skills/ce-product-pulse/references/report-template.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,10 @@ Loaded by `SKILL.md` at Phase 2.3 after queries have returned. Fill the template
99
- No hardcoded thresholds. Do not label things "high" or "low" or color anything red unless the reader asked for threshold-based annotation at setup.
1010
- No PII. No emails, no account IDs, no message content.
1111
- Headlines are the top of the page. If a reader only reads the first 3 lines, they should know the most important thing that happened.
12-
- If `docs/strategy.md` exists, re-read its `## Key metrics` section before assembling the report and render each strategy metric in the Usage section with its current value and delta. If a strategy metric has no current value, include it anyway and mark it `no data`.
12+
- If `docs/strategy.md` exists, re-read its `## Key metrics` section before assembling the report. For each strategy metric, decide what to render:
13+
- If the metric name appears in `pulse_excluded_metrics`, omit it from the report.
14+
- If the metric name appears in `pulse_pending_metrics`, include it in the Usage section marked `no data (instrumentation pending)`.
15+
- Otherwise, render the metric with its current value and delta. If the query returns no value, include it anyway and mark it `no data`.
1316

1417
## Template
1518

plugins/compound-engineering/skills/ce-setup/references/config-template.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,3 +26,5 @@
2626
# pulse_tracing_source: sentry # sentry | datadog | custom (no default)
2727
# pulse_payments_source: stripe # stripe | custom (no default)
2828
# pulse_db_enabled: false # true | false (default: false; read-only DB if true)
29+
# pulse_pending_metrics: "retention_d7,nps" # comma-separated strategy metrics awaiting instrumentation; render as 'no data'
30+
# pulse_excluded_metrics: "north_star" # comma-separated strategy metrics intentionally not in pulse

0 commit comments

Comments
 (0)