Skip to content
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 21 additions & 24 deletions docs/design/capture-and-fingerprint.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,12 @@ arbitrary visible text as structural identity.

- State and target identity are deterministic within one build and fingerprint
schema version.
- Exploration and production profiles use the same fingerprint algorithm.
- Optional evidence capabilities use the same fingerprint algorithm.
- Locale is emitted as session and event evidence but never enters identity.
- Developer tags strengthen matching but are optional.
- Screenshots remain visual evidence and are masked before leaving the app.
- Capture failures and sink failures must not interrupt the host app.
- Dormant and disabled modes return the host child unchanged.
- Disabled lifecycle configuration returns the host child unchanged.

Cross-build stability is not promised. Consumers should namespace identity by
build metadata and `fingerprintSchemaVersion`; remapping identities across app
Expand All @@ -46,21 +46,20 @@ NavigatorObserver + global/local pointer input + scroll notifications
```

`TugboatReplay.wrapApp` installs the controller, repaint boundary, input
capture, scroll listener, and lifecycle observer only for an active profile.
capture, scroll listener, and lifecycle observer only when capture is enabled.
`TugboatReplay.navigatorObserver` supplies route changes and navigator context.
Both are needed for complete capture.

### Profiles and activation
### Lifecycle and capabilities

| Profile | Implemented behavior |
| --- | --- |
| `dormant` | Always-mounted lightweight gate; no pointer/screenshot/sink machinery until `activate` |
| `exploration` | full interaction capture, scene inventory, and optional emitted semantic maps |
| `productionLean` | interaction capture, no scene-inventory events, production screenshot masking by default |
`TugboatReplayConfig.enabled` controls the capture lifecycle. Scene inventory,
semantic-map, diagnostic, and action-context evidence use separate additive
capabilities. All optional capabilities are false by default. The default
screenshot mask is always `allTextAndMedia`.

`TugboatReplay.disabled = true` is a global kill switch. It deactivates the
current controller and keeps future calls to `wrapApp` inert. Runtime
`activate(activationRequestId:, profile:)` notifies the mounted gate without
`activate(activationRequestId:)` notifies the mounted gate without
requiring a host rebuild. `deactivate()` tears capture down through the same
gate. Pause/hidden flush pending delivery; detach ends the session once.

Expand Down Expand Up @@ -261,10 +260,9 @@ the SDK collects mask rectangles using the shared anchor resolver and applies
them as opaque fills on the RGBA buffer inside the encode isolate (avoiding a
second full-size GPU raster).

The default mask policy is profile-dependent:

- `exploration`: explicit `TugboatSensitive` subtrees only;
- `productionLean`: all rendered text, editable inputs, and images.
The default mask policy covers all rendered text, editable inputs, and images.
Hosts can select a stricter or narrower explicit mask policy. Evidence
capabilities never change the mask.

The public mask levels are `explicitOnly`, `allTextAndMedia`, `allText`,
`allTextExceptActionable`, and `sensitiveInputsOnly`.
Expand Down Expand Up @@ -336,19 +334,18 @@ device-tier gates.

## Viewport semantics

`viewportSemanticMode` resolves with the capture profile:
`viewportSemanticMode` resolves with `emitViewportSemanticMap`:

| Mode | Engine | Emits map events | Debug logs |
| --- | --- | --- | --- |
| `off` | no | no | no |
| `tapResolutionOnly` | yes for active profiles | no | no |
| `full` | yes | exploration only | no |
| `fullWithDebugLogs` | yes | exploration only | exploration only |

Exploration holds a persistent Flutter `SemanticsHandle` when semantics are
enabled. Production uses transient semantics and never emits semantic-map
events, even in `full` modes; it can still build maps locally for tap
resolution. Emitted exploration maps are bounded by
| `tapResolutionOnly` | yes while enabled | no | no |
| `full` | yes | when capability is true | no |
| `fullWithDebugLogs` | yes | when capability is true | when capability is true |

The SDK holds a persistent Flutter `SemanticsHandle` only while semantic-map
emission is enabled. It can still build maps locally for tap resolution.
Emitted maps are bounded by
`viewportSemanticMapMaxNodes` (default `120`) and
`viewportSemanticMapMaxBytes` (default `48000`).

Expand Down Expand Up @@ -412,7 +409,7 @@ Unit thresholds live in `benchmark/screenshot_budget_baseline.dart`. The default
rolling budget is 60 ms per 5 s window so eligible captures skip under load
sooner now that paint-generation / dHash coalesce replace the old post-capture
state-signature short circuit. Record multi-tier device measurements before
enabling more aggressive degradation in production profiles.
enabling more aggressive degradation in normal app capture.

### 3. Deferred capture surfaces

Expand Down
20 changes: 15 additions & 5 deletions docs/integration/collector.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ Transport failures are isolated from the host app.

## Required app integration

Install the wrapper and navigator observer. Capture is dormant by default, so
select an active profile for startup capture:
Install the wrapper and navigator observer. Capture is disabled by default.
Set `enabled: true` for startup capture, or call `TugboatReplay.activate` later:

```dart
MaterialApp(
Expand All @@ -27,6 +27,10 @@ MaterialApp(
);
```

The default configuration masks all text and media. Optional inventory,
semantic-map, diagnostic, and action-context evidence stays off until the host
enables each matching capability.

Without the observer, pointer and scroll evidence still works but route-change
events and route-backed anchors are incomplete. Without the wrapper no capture
controller or transport is installed.
Expand All @@ -42,7 +46,10 @@ Use the exploration destination for an interactive local run:

```dart
const config = TugboatReplayConfig(
profile: TugboatCaptureProfile.exploration,
enabled: true,
emitSceneInventory: true,
emitViewportSemanticMap: true,
acceptActionContext: true,
explorationCollectorUrl: 'ws://127.0.0.1:7832/sdk',
explorationRunId: 'optional-run-id',
appInfo: TugboatCollectorAppInfo(
Expand Down Expand Up @@ -115,7 +122,7 @@ final collector = await TugboatCollectorHost.fromPlatform(
);

final config = TugboatReplayConfig(
profile: TugboatCaptureProfile.productionLean,
enabled: true,
collector: collector,
);
```
Expand Down Expand Up @@ -301,7 +308,10 @@ stamping or clearing a newer session's evidence.

```dart
final config = TugboatReplayConfig(
profile: TugboatCaptureProfile.exploration,
enabled: true,
emitSceneInventory: true,
emitViewportSemanticMap: true,
acceptActionContext: true,
explorationCollectorUrl: 'ws://127.0.0.1:7832/sdk',
collector: productionCollectorConfig,
);
Expand Down
2 changes: 1 addition & 1 deletion docs/releases/compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@

| Adapter | Adapter version | Native runtime |
| --- | --- | --- |
| Flutter `tugboat` | 0.9.0 | Android `capture-runtime` `0.1.0`; Apple `TugboatCaptureRuntime` `0.1.1`; mode-free capture API with additive evidence capabilities. |
| Flutter `tugboat` | 0.8.17 | Android `capture-runtime` `0.1.0`; Apple `TugboatCaptureRuntime` `0.1.1`; blank iOS engine captures retry with explicit hierarchy coverage. |
| Apple `TugboatCaptureRuntime` | 0.1.1 | Rejects transparent and near-white captures and validates explicit view-hierarchy capture before encoding. |
| Flutter `tugboat` | 0.8.16 | Same hosted runtimes as 0.8.15. iOS plugin looks up `registrar.viewController` at runtime so Flutter 3.35 hosts compile. |
| Flutter `tugboat` | 0.8.15 | Android `com.gettugboat.sdk:capture-runtime:0.1.0` from Maven Central. Apple `TugboatCaptureRuntime` `0.1.0` from CocoaPods trunk. Plugin iOS floor 15. |
| Flutter `tugboat` | 0.8.14 | Android `com.gettugboat.sdk:capture-runtime:0.1.0` from Maven Central. Apple `TugboatCaptureRuntime` 0.1.x compiled from monorepo sources (unpublished CocoaPod is not required). Plugin iOS floor 12; native capture still reports unsupported below iOS 15. |
| Flutter `tugboat` | 0.8.13 | Android `capture-runtime` 0.1.x compiled from monorepo sources (hosted Maven is not required). Apple `TugboatCaptureRuntime` 0.1.x compiled from monorepo sources (unpublished CocoaPod is not required). Plugin iOS floor 12; native capture still reports unsupported below iOS 15. |
| Flutter `tugboat` | 0.8.12 | none (Flutter `flutterRepaintBoundary` only; session identity stamp patch) |
| Flutter `tugboat` | 0.9.0 (planned) | Android `capture-runtime` 0.1.x and Apple `TugboatCaptureRuntime` 0.1.x |
| Apple `TugboatCaptureRuntime` | 0.1.0 | CocoaPods trunk / SwiftPM git tag; iOS 15 live Flutter-layer CPU path |
| `@tugboat/react-native` | — | not started |

Expand Down
24 changes: 24 additions & 0 deletions sdks/flutter/packages/tugboat/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,27 @@
## 0.9.0

### Changed

- Replace `TugboatCaptureProfile` with one privacy-safe capture policy.
- Use `TugboatReplayConfig.enabled` for lifecycle configuration.
- Add explicit, additive capabilities for scene inventory, viewport semantic
maps, capture diagnostics, and external action context.
- Keep `allTextAndMedia` as the default screenshot mask when capabilities are
enabled.
- Make `TugboatParameterPolicy.allowAll` mode-free and remove the duplicate
production-specific parameter policy.
- Remove capture-profile fields from SDK health and sink session context.
- Rename the controller action-window API to `setActionContext` and
`clearActionContext`.

### Migration

- Replace `profile: TugboatCaptureProfile.dormant` with `enabled: false`.
- Replace `profile: TugboatCaptureProfile.productionLean` with `enabled: true`.
- Replace exploration mode with `enabled: true` plus only the required
additive capabilities.
- Remove the `profile` argument from `TugboatReplay.activate`.

## 0.8.17

### Fixed
Expand Down
Loading
Loading