Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ verified in their own repositories.

## Current compatibility

- package version: `0.4.12`;
- package version: `0.4.13`;
- session JSON schema: `7`;
- fingerprint schema: `6`;
- minimum Dart SDK: `3.9.2`;
Expand Down
217 changes: 217 additions & 0 deletions docs/integration/production-replay-acceptance-0.4.13.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,217 @@
# Production replay acceptance: 0.4.12 → 0.4.13

Use this after shipping SDK **0.4.13** (and collector build passthrough) and
running Blend through a similar flow to the baseline session.

## Baseline (locked)

| Field | Value |
| --- | --- |
| Session | `session-1785142623932166` |
| SDK | `0.4.12` |
| App | `to.blend.mobile_app` |
| Build | `3.17.177+1472` |

| Metric | Baseline |
| --- | ---: |
| Raw `tap` | 86 |
| Swipe-consumed taps | 63 |
| Settled taps | 23 |
| Truly orphaned taps | 0 |
| `tap_settled` `result=unknown` | 6 |
| unknown `superseded_route_epoch` | 2 |
| `missing_frame` with zero local/normalized | 1 |
| `capture_diagnostic` `missing_context_graph_build_identity` | 37 / 37 |

## Manual run checklist

1. Install / point Blend at tugboat **0.4.13** (includes same-turn claims,
deferred taps, session-end pointer fence, duplicate-down coalesce).
2. Confirm collector with event `build` passthrough is deployed (Gate 8).
3. Drive: scroll/flick on home, Get Pro / paywall, StageIt / sheet / chooser,
one rapid double-tap during settle.
4. Paste the new `sessionId` (+ Blend build) in chat for scoring.

## Hard gates (all must PASS)

1. **Identity** — `metadata.sdkVersion = '0.4.13'` on event groups.
2. **No phantom taps** — `swipe_consumed_taps = 0`.
3. **Settle coverage** — `settled / raw_taps >= 0.95`.
4. **No same-position bursts** — no UTC second with `tap_count >= 5` and
`distinct_positions = 1`.
5. **Unknown settles** — `unknown / tap_settled <= 0.10` **and**
`superseded_route_epoch` count = 0.
6. **After frames** — 100% of `navigated`/`changed` settles have `afterFrame`.
7. **Missing-frame geometry** — any `missing_frame` tap has
`normalizedX/Y ∈ [0,1]` and `boundaryWidth/Height > 0`.
8. **Diagnostics identity** — zero
`missing_context_graph_build_identity` on `capture_diagnostic`
(BLOCKED if collector not deployed).
9. **Swipe start geometry** — every `swipe` has `payload.startCaptureCoordinate`.

Overall verdict: **ACCEPT** only if every gate PASSes; otherwise **REJECT** or
**BLOCKED** with the first failing gate id.

## ClickHouse queries

Replace `{newSession}` with the new session id. Service: pmkit ClickHouse.

### Gate 1 — SDK version

```sql
SELECT
argMax(metadata.sdkVersion::Nullable(String), receivedAt) AS sdkVersion,
count() AS rows
FROM pmkit.raw_events
WHERE sessionId = {newSession}
GROUP BY eventType
ORDER BY eventType
```

### Gates 2–3 — tap fate

```sql
WITH events AS (
SELECT id,
argMax(eventType, receivedAt) AS eventType,
argMax(metadata.relatedEventId::Nullable(String), receivedAt) AS related
FROM pmkit.raw_events
WHERE sessionId = {newSession}
GROUP BY id
),
taps AS (SELECT id FROM events WHERE eventType = 'tap'),
swipeRefs AS (SELECT related FROM events WHERE eventType = 'swipe' AND related IS NOT NULL),
settleRefs AS (SELECT related FROM events WHERE eventType = 'tap_settled' AND related IS NOT NULL)
SELECT
count() AS totalTaps,
countIf(id IN (SELECT related FROM swipeRefs)) AS consumedBySwipe,
countIf(id IN (SELECT related FROM settleRefs)) AS settled,
countIf(
id NOT IN (SELECT related FROM swipeRefs)
AND id NOT IN (SELECT related FROM settleRefs)
) AS orphaned
FROM taps
```

### Gate 4 — bursts

```sql
WITH taps AS (
SELECT
argMax(triggeredAt, receivedAt) AS triggeredAt,
argMax(payload.x::Nullable(Float64), receivedAt) AS x,
argMax(payload.y::Nullable(Float64), receivedAt) AS y
FROM pmkit.raw_events
WHERE sessionId = {newSession} AND eventType = 'tap'
GROUP BY id
)
SELECT
toStartOfSecond(triggeredAt) AS sec,
count() AS tapCount,
uniqExact((round(x, 1), round(y, 1))) AS distinctPositions
FROM taps
GROUP BY sec
HAVING tapCount >= 5 AND distinctPositions = 1
ORDER BY sec
```

### Gate 5 — unknown settles

```sql
WITH settles AS (
SELECT
argMax(result, receivedAt) AS result,
argMax(toJSONString(payload), receivedAt) AS payloadJson
FROM pmkit.raw_events
WHERE sessionId = {newSession} AND eventType = 'tap_settled'
GROUP BY id
)
SELECT
count() AS totalSettled,
countIf(result = 'unknown') AS unknownSettles,
countIf(
JSONExtractString(payloadJson, 'settleObservation', 'captureFailure')
= 'superseded_route_epoch'
) AS supersededRouteEpoch
FROM settles
```

### Gate 6 — after frames on navigated/changed

```sql
WITH settles AS (
SELECT
argMax(result, receivedAt) AS result,
argMax(afterFrame, receivedAt) AS afterFrame
FROM pmkit.raw_events
WHERE sessionId = {newSession} AND eventType = 'tap_settled'
GROUP BY id
)
SELECT
countIf(result IN ('navigated', 'changed')) AS outcomeRows,
countIf(result IN ('navigated', 'changed') AND afterFrame IS NULL) AS missingAfter
FROM settles
```

### Gate 7 — missing_frame geometry

```sql
WITH taps AS (
SELECT argMax(toJSONString(payload), receivedAt) AS payloadJson
FROM pmkit.raw_events
WHERE sessionId = {newSession} AND eventType = 'tap'
GROUP BY id
)
SELECT
JSONExtractString(payloadJson, 'captureCoordinate', 'unavailableReason') AS reason,
JSONExtractFloat(payloadJson, 'captureCoordinate', 'normalizedX') AS nx,
JSONExtractFloat(payloadJson, 'captureCoordinate', 'normalizedY') AS ny,
JSONExtractFloat(payloadJson, 'captureCoordinate', 'boundaryWidth') AS bw,
JSONExtractFloat(payloadJson, 'captureCoordinate', 'boundaryHeight') AS bh
FROM taps
WHERE JSONExtractString(payloadJson, 'captureCoordinate', 'unavailableReason')
= 'missing_frame'
```

### Gate 8 — diagnostics enrichment

```sql
WITH diags AS (
SELECT argMax(toJSONString(payload), receivedAt) AS payloadJson
FROM pmkit.raw_events
WHERE sessionId = {newSession} AND eventType = 'capture_diagnostic'
GROUP BY id
)
SELECT
count() AS diagnostics,
countIf(
JSONExtractString(payloadJson, 'contextEnrichment', 'reason')
= 'missing_context_graph_build_identity'
) AS missingBuildIdentity
FROM diags
```

### Gate 9 — swipe startCaptureCoordinate

```sql
WITH swipes AS (
SELECT argMax(toJSONString(payload), receivedAt) AS payloadJson
FROM pmkit.raw_events
WHERE sessionId = {newSession} AND eventType = 'swipe'
GROUP BY id
)
SELECT
count() AS swipes,
countIf(JSONHas(payloadJson, 'startCaptureCoordinate')) AS withStartCoord
FROM swipes
```

## Deliverable

After scoring, write
`docs/integration/production-replay-compare-0.4.12-vs-0.4.13.md` with:

- session ids + SDK versions
- side-by-side metric table
- per-gate PASS/FAIL with deciding counts
- single overall `ACCEPT` / `REJECT` / `BLOCKED`
65 changes: 65 additions & 0 deletions docs/integration/production-replay-acceptance-0.4.15.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Production replay acceptance: interaction consolidation (0.4.15)

Use this after shipping SDK **0.4.15** (canonical interactions + delayed
reconciliation) and running Blend through the acceptance flow.

## Baseline (locked)

Prefer the nearest prior Blend session against SDK **0.4.12 / 0.4.13** for
side-by-side scoring. Record the new session id and Blend build before scoring.

## What changed in the SDK

| Concern | 0.4.13 behavior | 0.4.15 behavior |
| --- | --- | --- |
| Gesture identity | `tap` + `tap_settled` peers | one `interaction` (`stream: semantic`) + legacy projection |
| Claim window | microtask same-turn only | default 1,250 ms delayed reconciliation |
| Diagnostics | mixed into normal events | `stream: diagnostic` |
| Origin | frozen on pending tap | immutable `InteractionOrigin` on the transaction |
| Swipe state | refreshed at pointer-up | frozen to pointer-down origin |

## Manual run checklist

1. Point Blend at tugboat **0.4.15**.
2. Drive: home scroll/flick, Get Pro / paywall, full-screen navigation, modal
bottom sheet, asynchronous onboarding transition, rapid double-tap,
automatic redirect after a settled tap.
3. Paste the new `sessionId` (+ Blend build) for scoring.

## Hard gates (all must PASS)

1. **Identity** — `metadata.sdkVersion = '0.4.15'`.
2. **Canonical coverage** — one `stream: semantic` `interaction` per completed
user gesture (tap / swipe / scroll / cancelled).
3. **Origin correctness** — interaction `origin.route` / `origin.targetAnchor`
match the pointer-down screen/component, never the destination.
4. **Delayed attribution** — delayed navigation / bottom sheet inside 1,250 ms
has `attribution.kind = delayed_likely` (or `direct`) and
`result.status = navigated|changed`, with matching
`route_change.causedByInteractionId`.
5. **Automatic false-claim rate** — timer/auth redirects after the window, and
routes with competing pointers, stay `navigationOrigin =
automatic_or_unknown`.
6. **No semantic tap for scrolls/swipes** — completed scroll/swipe produces no
`stream: semantic` tap; one `interaction` with `gesture=scroll|swipe`.
7. **Diagnostic isolation** — enrichment selection of `stream: semantic`
excludes `capture_diagnostic`.
8. **Rage-tap precision** — three no-result taps on the same origin target flag
once; three scrolls or three successful navigation taps do not.

## Soft / observational

- Semantic event count per completed gesture should drop vs 0.4.0 raw
`tap`+`tap_settled`+scroll peer inflation.
- Legacy projection remains present until collector/graph cut over; do not
delete `tap`/`tap_settled` selection until two representative Blend flows pass
on canonical interactions alone.
- Instrument pending-to-success latency; retune `interactionClaimWindow` from
production evidence if 1,250 ms is too short/long.

## Consumer follow-ups (separate PRs)

- Collector / Context Graph enrichment select `stream: semantic` `interaction`
and map components via `origin.targetAnchor`.
- Build causal edges from `result` / `causedByInteractionId`.
- Update dashboard rage-tap detectors to the definition above.
21 changes: 13 additions & 8 deletions docs/integration/production-replay-acceptance.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,19 @@ database receipt alone as proof that a replay is correct.

## Current acceptance status

Production acceptance #13/#14 remains open. The SDK's route-epoch and frame
provenance behavior is an intended invariant, but rapid/nested modal chains and
programmatic/automatic navigation can still be absent or degraded in a
production replay. Record those observations as SDK capture gaps; do not infer
route/action coherence from the intended contract or repair the evidence in the
dashboard. Stored tap coordinates are global logical pixels and are not
capture-boundary-normalized for playback, so fractional overlay drift is also a
known limitation.
Interaction consolidation shipped in SDK **0.4.15** (canonical `interaction`
events, 1,250 ms delayed claim window, diagnostic stream isolation). Use
[`production-replay-acceptance-0.4.15.md`](./production-replay-acceptance-0.4.15.md)
for the Blend scoring gates. Collector/Context Graph migration onto
`stream: semantic` interactions remains a follow-up before legacy
`tap`/`tap_settled` projection can be removed.

Production acceptance #13/#14 remains open for rapid/nested modal chains and
programmatic/automatic navigation gaps. Record those observations as SDK
capture gaps; do not infer route/action coherence from the intended contract or
repair the evidence in the dashboard. Stored tap coordinates are global logical
pixels and are not capture-boundary-normalized for playback, so fractional
overlay drift is also a known limitation.

## Roles and evidence

Expand Down
Loading
Loading