From d2294014a7b5bac6d0f0f23f2ede6fe0db589974 Mon Sep 17 00:00:00 2001 From: Chinmay-KB Date: Sun, 26 Jul 2026 15:01:38 +0530 Subject: [PATCH 1/2] docs(replay): reconcile the 0.4.10 capture contract --- docs/README.md | 2 +- docs/design/capture-and-fingerprint.md | 56 +++++++++++++++++-- .../production-replay-acceptance.md | 15 ++++- packages/tugboat/README.md | 40 ++++++++++++- 4 files changed, 105 insertions(+), 8 deletions(-) diff --git a/docs/README.md b/docs/README.md index e34a591..380bfa7 100644 --- a/docs/README.md +++ b/docs/README.md @@ -28,7 +28,7 @@ verified in their own repositories. ## Current compatibility -- package version: `0.4.9`; +- package version: `0.4.10`; - session JSON schema: `7`; - fingerprint schema: `6`; - minimum Dart SDK: `3.9.2`; diff --git a/docs/design/capture-and-fingerprint.md b/docs/design/capture-and-fingerprint.md index fa0708e..cc89fb3 100644 --- a/docs/design/capture-and-fingerprint.md +++ b/docs/design/capture-and-fingerprint.md @@ -1,6 +1,6 @@ # Capture and fingerprint architecture -Status: current implementation audit (2026-07-16) +Status: current implementation audit (2026-07-26) Scope: `packages/tugboat` in this repository This document describes what the Flutter SDK implements today. It deliberately @@ -107,6 +107,52 @@ Events may carry `beforeFrame`, `afterFrame`, `stateAnchor`, `targetAnchor`, type-specific `data`. Route transition values live in `route_change.data`, not in a session-level route dictionary. +### Capture lifecycle and attribution + +`wrapApp` starts a session only after its repaint boundary has a non-zero +viewport. The session begins with `session_start` and an initial capture +request. Pointer-down records `tap` plus a compatible pre-interaction frame, +then pointer-up either records a swipe or creates one `tap_settled` outcome. +The settled event refers to the initial tap through `relatedEventId` and is +intended to attach an after-frame only when that frame's provenance matches the +observed route epoch. A capture that is unavailable, cancelled, superseded, or +timed out is represented by bounded capture/attachment diagnostics instead of +borrowing the latest frame from another screen. + +Frame requests are serialized, may coalesce, and use fresh-paint/readback +checks before publishing. Their provenance records the capture context and +completion state, so exact-content and perceptual deduplication reuse frames +only within a compatible context. `paused` and `hidden` request a delivery +flush after 500 ms; `resumed` cancels that pending flush; `detached`, wrapper +disposal, and deactivation end the session once and initiate sink shutdown. + +Input event coordinates (`x`, `y`, and swipe start/delta values) originate as +Flutter global logical-pixel coordinates. The resolver converts a global point +to capture-boundary local coordinates only to hit-test and normalize it against +the viewport; stored coordinates are not capture-boundary-normalized for replay +playback, are neither device pixels nor widget-local, and can therefore produce +fractional overlay drift. + +### Navigator and modal routes + +Installing `TugboatReplay.navigatorObserver` is intended to record the standard +Navigator push, pop, replace, and remove callbacks automatically. No +per-navigation SDK call is required. Dialogs and `showModalBottomSheet` +instances can participate when their routes use that observed Navigator. Nested +navigators require their own observer integration; native/system overlays +remain outside the Flutter Navigator and repaint-boundary surface. + +Every visible navigation change advances a route epoch and has a single route +capture barrier after transition settlement. A later visible navigation +supersedes stale work. This is the implemented intended invariant: a delayed +destination capture or tap-settle operation should not attach the prior route's +frame, and an unavailable compatible frame should remain explicitly degraded. + +It is not yet a production-accepted guarantee. Production acceptance #13/#14 +remains open: rapid/nested modal chains and programmatic/automatic navigation +can still be absent or degraded. Treat those observations as SDK capture gaps, +not as coherent replay evidence. + Pointer work and post-interaction capture are serialized through a controller queue. An error in one queued task is caught and logged so later taps, scrolls, and route captures continue. @@ -229,11 +275,13 @@ frames captured before connection are still sent. | --- | --- | --- | --- | | `off` | no | no | no | | `tapResolutionOnly` | yes for active profiles | no | no | -| `full` | yes | yes | no | -| `fullWithDebugLogs` | yes | yes | yes | +| `full` | yes | exploration only | no | +| `fullWithDebugLogs` | yes | exploration only | exploration only | Exploration holds a persistent Flutter `SemanticsHandle` when semantics are -enabled. Production uses transient semantics. Emitted maps are bounded by +enabled. Production uses transient semantics and never emits semantic-map +events, even in `full` modes; it can still build maps locally for tap +resolution. Emitted exploration maps are bounded by `viewportSemanticMapMaxNodes` (default `120`) and `viewportSemanticMapMaxBytes` (default `48000`). diff --git a/docs/integration/production-replay-acceptance.md b/docs/integration/production-replay-acceptance.md index b4331c0..dfd550e 100644 --- a/docs/integration/production-replay-acceptance.md +++ b/docs/integration/production-replay-acceptance.md @@ -10,6 +10,17 @@ The procedure is intentionally strict about build identity. Do not mix sessions from different SDK revisions, infer the SDK revision from capture time, or use database receipt alone as proof that a replay is correct. +## Current acceptance status + +Production acceptance #13/#14 remains open. The SDK's route-epoch and frame +provenance behavior is an intended invariant, but rapid/nested modal chains and +programmatic/automatic navigation can still be absent or degraded in a +production replay. Record those observations as SDK capture gaps; do not infer +route/action coherence from the intended contract or repair the evidence in the +dashboard. Stored tap coordinates are global logical pixels and are not +capture-boundary-normalized for playback, so fractional overlay drift is also a +known limitation. + ## Roles and evidence Record these people before starting: @@ -140,7 +151,7 @@ flows share a session, list the event IDs or timestamps that delimit each flow. Wait until the collector session has finalized and the replay is available in the production website. Filter to the recorded Blend build and SDK version -`0.4.9` (or the version under test), then open every recorded session. +`0.4.10` (or the version under test), then open every recorded session. For each interaction, inspect the actual replay UI and verify: @@ -169,7 +180,7 @@ Use one row per production session: | Session ID | UTC range | Blend build | SDK version / SHA | Flows | Frame availability | Route/action coherence | Verdict | Follow-up | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| `` | ` - ` | `` | `0.4.9 / ` | `` | pass/fail | pass/fail | accept/reject | `` | +| `` | ` - ` | `` | `0.4.10 / ` | `` | pass/fail | pass/fail | accept/reject | `` | The cohort passes only when: diff --git a/packages/tugboat/README.md b/packages/tugboat/README.md index 7e85db3..916dba3 100644 --- a/packages/tugboat/README.md +++ b/packages/tugboat/README.md @@ -5,7 +5,7 @@ checkpoints around meaningful interactions, compact structural anchors, route transitions, scrolling evidence, and optional viewport semantic maps. Capture can be sent to the local exploration WebSocket, the HTTP collector, or both. -The current package version is `0.4.9`. Session JSON uses schema version `7` +The current package version is `0.4.10`. Session JSON uses schema version `7` (readers still accept `6`), and structural fingerprints use fingerprint schema version `6`. @@ -43,6 +43,27 @@ work, but route-change events and route-backed anchors are incomplete. Without `wrapApp`, no capture controller, repaint boundary, or input/scroll listener is installed. +### Navigation and overlays + +The supplied observer is intended to record standard Navigator `push`, `pop`, +`replace`, and `remove` callbacks without application code calling the replay +controller for each navigation. Dialog and modal-bottom-sheet routes can +participate when they use that observed Navigator. Nested navigators need their +own observer wiring, and native/system overlays are outside the Flutter +Navigator/repaint-boundary contract. + +Each visible route change creates a route epoch and waits for the transition +plus the configured settle delay before taking the destination capture. A newer +visible route supersedes an older pending capture. Consequently, a route event +and a related `tap_settled` event are intended to reference a frame compatible +with that route epoch, or report bounded degraded/capture diagnostics rather +than attach an origin-route frame merely because it was the latest frame. + +This is an implemented SDK invariant, not a production-accepted guarantee. +Production acceptance for #13/#14 remains open: rapid or nested modal chains +and programmatic/automatic navigation can still be absent or degraded. Treat +those cases as an SDK capture gap, not as coherent replay evidence. + ## Capture profiles and runtime state `TugboatReplayConfig.profile` controls whether the wrapper installs capture @@ -177,6 +198,23 @@ The SDK first skips repeated state signatures, then uses a small dHash to avoid PNG encoding for visually unchanged content, and finally deduplicates encoded frames by content hash. +Pointer coordinates in event data (`x`, `y`, and swipe `startX`/`startY`) are +Flutter global logical-pixel coordinates from the pointer event. The SDK +converts a copy into its capture boundary's local space only for hit-testing and +normalizing target/viewport-semantic bounds; stored event coordinates are not +capture-boundary-normalized for replay playback. Do not interpret them as +physical pixels or as coordinates relative to an individual widget. Fractional +overlay drift is therefore still possible. + +For a tap, `beforeFrame` is selected at pointer-down only if its provenance is +compatible with the tap's route epoch. At pointer-up, a single `tap_settled` +event links back to the `tap` via `relatedEventId` and is intended to contain +an `afterFrame` only when the settled capture is compatible with the observed +route. A missing attachment is explicit in `frameAttachment`/settle diagnostics +rather than a fallback to an unrelated frame. Acceptance remains open for the +navigation cases above, so consumers must still treat absence and degradation +as a capture gap. + During local WebSocket exploration, connecting without an HTTP collector suppresses new Flutter screenshot capture for UI-thread performance. Events, anchors, inventories, and semantic evidence continue to stream; the CLI's ADB From 4aabdd9a5b46fe2965a1f2ac763166d7628b6d57 Mon Sep 17 00:00:00 2001 From: Chinmay-KB Date: Sun, 26 Jul 2026 15:03:19 +0530 Subject: [PATCH 2/2] chore: bump tugboat to 0.4.11 --- docs/README.md | 2 +- packages/tugboat/CHANGELOG.md | 4 ++++ packages/tugboat/README.md | 2 +- packages/tugboat/lib/src/sdk_version.dart | 2 +- packages/tugboat/pubspec.yaml | 2 +- 5 files changed, 8 insertions(+), 4 deletions(-) diff --git a/docs/README.md b/docs/README.md index 380bfa7..62389f2 100644 --- a/docs/README.md +++ b/docs/README.md @@ -28,7 +28,7 @@ verified in their own repositories. ## Current compatibility -- package version: `0.4.10`; +- package version: `0.4.11`; - session JSON schema: `7`; - fingerprint schema: `6`; - minimum Dart SDK: `3.9.2`; diff --git a/packages/tugboat/CHANGELOG.md b/packages/tugboat/CHANGELOG.md index 34a473b..5662517 100644 --- a/packages/tugboat/CHANGELOG.md +++ b/packages/tugboat/CHANGELOG.md @@ -38,6 +38,10 @@ ### Changed +- **Replay capture contract documentation** — documents the implemented route/frame + attribution invariant separately from the still-open production acceptance + gaps for rapid modal chains, automatic navigation, and playback tap-coordinate + alignment. - **Route-capture barrier** — tap settlement now joins the matching route epoch instead of reusing the previous route's latest frame. Supersession, lifecycle cancellation, capture failure, and bounded timeout outcomes diff --git a/packages/tugboat/README.md b/packages/tugboat/README.md index 916dba3..1552b07 100644 --- a/packages/tugboat/README.md +++ b/packages/tugboat/README.md @@ -5,7 +5,7 @@ checkpoints around meaningful interactions, compact structural anchors, route transitions, scrolling evidence, and optional viewport semantic maps. Capture can be sent to the local exploration WebSocket, the HTTP collector, or both. -The current package version is `0.4.10`. Session JSON uses schema version `7` +The current package version is `0.4.11`. Session JSON uses schema version `7` (readers still accept `6`), and structural fingerprints use fingerprint schema version `6`. diff --git a/packages/tugboat/lib/src/sdk_version.dart b/packages/tugboat/lib/src/sdk_version.dart index 287d2e5..5ce58f4 100644 --- a/packages/tugboat/lib/src/sdk_version.dart +++ b/packages/tugboat/lib/src/sdk_version.dart @@ -1,3 +1,3 @@ // Keep this in sync with packages/tugboat/pubspec.yaml. The SDK version test // reads pubspec.yaml directly so release bumps fail fast if this drifts. -const tugboatSdkVersion = '0.4.10'; +const tugboatSdkVersion = '0.4.11'; diff --git a/packages/tugboat/pubspec.yaml b/packages/tugboat/pubspec.yaml index 36e5dd4..5b2ce3a 100644 --- a/packages/tugboat/pubspec.yaml +++ b/packages/tugboat/pubspec.yaml @@ -1,7 +1,7 @@ name: tugboat description: >- Screenshot-based session replay with compact interaction anchors for Tugboat. -version: 0.4.10 +version: 0.4.11 repository: https://github.com/blendto/tugboat-flutter issue_tracker: https://github.com/blendto/tugboat-flutter/issues homepage: https://github.com/blendto/tugboat-flutter