Commit 44a3cc8
fix(telemetry): emit billing.operation.succeeded for all operation types and forward billing events to the desktop host sink (#14310)
## Summary
Two coupled billing-telemetry correctness fixes. Both make billing
dashboards under-report silently rather than fail loudly, which is why
they ship together.
Plan doc: [Billing telemetry program
dossier](https://app.notion.com/p/3ad6d73d365081fea125e284b6ac9f13)
## Changes
- **What**:
1. `billingOperationStore.handleSuccess` now emits
`billing.operation.succeeded` for **all three** operation types
(`subscription`, `topup`, `cancel`). It previously only emitted it for
`cancel`, while `billing.operation.failed` and
`billing.operation.timeout` are emitted for all three. Any `succeeded /
(succeeded + failed)` ratio grouped by `operation_type` therefore read a
**permanent 0%** for `subscription` and `topup` — a metric that can
never be non-zero, so an anomaly monitor on it pages immediately and
forever.
2. `HostTelemetrySink` now implements `trackBillingEvent`. It did not,
and `initHostTelemetry` *replaces* the provider registry rather than
extending it, so a cloud build running inside the desktop shell emitted
**zero** canonical `billing.*` events to any provider — an entire user
segment missing from every billing funnel.
- **Breaking**: no. Both changes are additive at the event level. No
event was renamed, removed, or had a property removed.
### Event contract after this change
`billing.operation.succeeded` is now emitted once per successful billing
operation, for every operation type, with the property set below. This
matches `billing.operation.failed` / `billing.operation.timeout`
exactly, so numerator and denominator of a success-rate ratio are
groupable by the same tags:
| Property | Type | Presence |
| --- | --- | --- |
| `operation` | `'operation'` | always |
| `stage` | `'succeeded'` | always |
| `outcome` | `'success'` | always |
| `billing_op_id` | string | always |
| `operation_type` | `'subscription' \| 'topup' \| 'cancel'` | always |
| `tier` | tier key or `'team'` | when the caller supplied it |
| `cycle` | `'monthly' \| 'yearly'` | when the caller supplied it |
| `checkout_type` | `'new' \| 'change'` | when the caller supplied it |
| `payment_intent_source` | attribution enum | when the caller supplied
it |
Notes for anyone querying this:
- A success-rate monitor should now be `succeeded / (succeeded +
failed)` over `billing.operation.*`, grouped by `operation_type`, and it
will be non-zero for all three types.
- `.timeout` is a separate stage from `.failed`; a ratio that should
treat timeouts as failures must include both in the denominator.
- The flow-specific success events
`billing.subscription_checkout.succeeded` and `billing.topup.succeeded`
are unchanged and still fire. A `subscription` success therefore emits
both `billing.operation.succeeded` and
`billing.subscription_checkout.succeeded`. These are distinct event
names — do not sum them into one success count.
- The four optional grouping keys are omitted from the payload when the
caller passed no metadata (the workspace top-up path currently starts
its poller without metadata), so group-bys on them must tolerate a
missing tag rather than assume it is present.
## Review Focus
**How to review this, and what it is part of.** This is one step in a
broader effort to make the billing/subscription funnel observable end to
end. The two fixes here are both "the dashboard lies" bugs rather than
user-visible bugs, so the thing to check is the *event contract*, not UI
behavior.
Suggested reading order:
1. `billingOperationStore.ts` — the whole fix is one moved emit. The new
`billing.operation.succeeded` call is unconditional at the top of
`handleSuccess`'s telemetry block; the old `else` branch that emitted it
only for `cancel` is deleted. Confirm the `cancel` payload is unchanged
in practice (`cancel` operations are started without metadata, so the
four new optional keys serialize away) and that `subscription` / `topup`
now additionally emit it.
2. `HostTelemetrySink.ts` — a four-line method delegating to the same
`getBillingTelemetryEventName` / `getBillingTelemetryEventPayload`
helpers the two cloud providers use, so the host rail receives a
byte-identical property set.
3. The three test files — each new test fails against the unfixed source
and passes with it (verified by reverting only the source and
re-running).
**The design question worth a second opinion (fix 2).** There were two
candidate fixes: implement `trackBillingEvent` on the sink, or make
`initHostTelemetry` compose with the existing registry instead of
replacing it. This PR implements the sink method, because that is what
the surrounding code's intent points at: the sink already forwards 46 of
the 54 `TelemetryProvider` methods through a single `capture()` bridge,
and `trackBillingEvent` postdates the sink's creation — it was never a
deliberate exclusion, just a method added later and not backfilled.
Registry *replacement*, by contrast, looks deliberate: in the desktop
shell the host process is the single telemetry egress point, and
composing would double-emit every event on two rails with two different
identities, affecting all ~40 forwarded events rather than just billing.
That is a much larger blast radius and a cross-surface decision, so it
is left as a follow-up question rather than decided here: **should a
cloud build inside the desktop shell fan out to both the host bridge and
the cloud providers?** Worth a reviewer's opinion, not blocking this
fix.
Also out of scope but noted while in here: `trackAuthFailed`,
`trackExecutionOutcome`, `trackOnboardingTour`, `trackShellLayout`,
`trackWidgetFavoriteToggled`, `trackWorkspaceInviteSent`, and
`trackWorkspaceInviteFailed` are likewise absent from the host sink.
Only `trackBillingEvent` is in scope here.
**Why these two are worth landing promptly.** Fix 1 unblocks
success-rate monitoring: a private, access-controlled infra repository
has monitors pending review that are built on exactly the `succeeded /
(succeeded + failed)` shape, and they cannot be applied against the
current event stream without alerting permanently on an unreachable
metric. Fix 2 closes a whole-segment blind spot — desktop-in-cloud users
are currently absent from every billing funnel, on every provider.
**Overlap with open PRs.** #14216 also edits `handleSuccess` in this
file, adding a dual-emit *inside* the `type === 'subscription'` branch.
This PR's hunk is an insertion *before* that branch plus the deletion of
the trailing `else`. The two are semantically independent and
compatible; expect at most a trivial textual conflict depending on merge
order. This PR deliberately does **not** touch
`useDowngradeToPersonal.ts`, whose three remaining `failure_category:
'unknown'` sites are already claimed by #14229.
## Testing
Six new regression tests, all confirmed to fail against the unfixed
source:
- `billingOperationStore.test.ts` — parameterized over all three
operation types, asserting `billing.operation.succeeded` fires with the
full grouping-key payload. This is the test that pins the parity
invariant.
- `HostTelemetrySink.test.ts` — a success event and a failure event are
forwarded to the host bridge with the derived event name and payload.
- `initHostTelemetry.test.ts` — end-to-end through the registry: after
`initHostTelemetry()`, a `trackBillingEvent` call reaches the host
bridge.
No e2e added: this is provider-dispatch-level code with no UI surface,
and exercising it in Playwright would require driving a real billing
flow against live payment rails. Per `browser_tests/README.md` this
belongs at the unit level.
Gates: `pnpm typecheck`, `pnpm lint`, `pnpm format:check`, `pnpm knip`
all clean; full unit suite green (1067 files, 14204 passed / 8 skipped).
Co-authored-by: Connor Byrne <c.byrne@comfy.org>1 parent d622d43 commit 44a3cc8
5 files changed
Lines changed: 136 additions & 9 deletions
File tree
- src/platform
- telemetry
- providers/host
- workspace/stores
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
54 | 80 | | |
Lines changed: 48 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
156 | 156 | | |
157 | 157 | | |
158 | 158 | | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
159 | 207 | | |
160 | 208 | | |
161 | 209 | | |
| |||
Lines changed: 14 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| 16 | + | |
16 | 17 | | |
17 | 18 | | |
18 | 19 | | |
| |||
50 | 51 | | |
51 | 52 | | |
52 | 53 | | |
53 | | - | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
54 | 60 | | |
55 | 61 | | |
56 | 62 | | |
| |||
121 | 127 | | |
122 | 128 | | |
123 | 129 | | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
124 | 137 | | |
125 | 138 | | |
126 | 139 | | |
| |||
Lines changed: 36 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
362 | 362 | | |
363 | 363 | | |
364 | 364 | | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
365 | 401 | | |
366 | 402 | | |
367 | 403 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
278 | 278 | | |
279 | 279 | | |
280 | 280 | | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
281 | 293 | | |
282 | 294 | | |
283 | 295 | | |
| |||
308 | 320 | | |
309 | 321 | | |
310 | 322 | | |
311 | | - | |
312 | | - | |
313 | | - | |
314 | | - | |
315 | | - | |
316 | | - | |
317 | | - | |
318 | | - | |
319 | 323 | | |
320 | 324 | | |
321 | 325 | | |
| |||
0 commit comments