diff --git a/docs/README.md b/docs/README.md index f46df41..3bb2c34 100644 --- a/docs/README.md +++ b/docs/README.md @@ -28,7 +28,7 @@ should be verified in their own repositories. ## Current compatibility -- package version: `0.7.0`; +- package version: `0.7.1`; - session JSON schema: `9`; - fingerprint schema: `6`; - minimum Dart SDK: `3.9.2`; diff --git a/docs/design/capture-and-fingerprint.md b/docs/design/capture-and-fingerprint.md index 2acc96d..9369f59 100644 --- a/docs/design/capture-and-fingerprint.md +++ b/docs/design/capture-and-fingerprint.md @@ -83,7 +83,7 @@ those fields are optional historic data in older sessions only. The session stores: -- frame metadata plus PNG bytes; +- frame metadata plus JPEG bytes; - ordered events; - optional scroll samples; - app/platform/viewport metadata; @@ -245,8 +245,9 @@ telemetry and avoid putting user data in them. ## Screenshot pipeline Screenshots are taken from the SDK `RepaintBoundary` at the configured pixel -ratio (default `0.75`). Before PNG encoding the SDK collects mask rectangles -using the shared anchor resolver and paints them onto the raster. +ratio (default `0.75`). Before JPEG encoding the SDK collects mask rectangles +using the shared anchor resolver and applies them as opaque fills on the RGBA +buffer inside the encode isolate (avoiding a second full-size GPU raster). The default mask policy is profile-dependent: @@ -256,14 +257,23 @@ The default mask policy is profile-dependent: The public mask levels are `explicitOnly`, `allTextAndMedia`, `allText`, `allTextExceptActionable`, and `sensitiveInputsOnly`. -Capture uses a 9x8 perceptual dHash before PNG encoding to skip a visually -unchanged raster, then SHA-256 content hashing to deduplicate encoded frames. -Capture requests are serialized and coalesced; repeated state signatures are -also skipped unless a caller forces capture. - -PNG readback and encoding still happen through Flutter image APIs on the UI -isolate. Platform views, video textures, maps, and native overlays may be absent -or incomplete in repaint-boundary output. +Capture computes a 9x8 perceptual dHash from the masked RGBA buffer inside the +encode isolate and skips JPEG encoding when the Hamming distance to the last +accepted hash is at most 2 bits (tolerating minor anti-alias shimmer). SHA-256 +content hashing then deduplicates encoded frames. Capture requests are +serialized and coalesced. When the capture subtree's paint signature has not +changed since the last accepted frame (outer capture boundary paint generation +plus nested [RepaintBoundary] layer/picture identity), the controller skips the +entire GPU readback/encode path and reuses a compatible frame (unless the +caller forces capture or requires a fresh paint). + +Mask fills, dHash, JPEG encoding, and content hashing run on a persistent +background isolate after a full-frame RGBA readback on the UI isolate. RGBA +bytes are packed into [TransferableTypedData] for the worker: packing still +copies once on the UI isolate, but the worker materializes the buffer without +a second full-frame copy (unlike a plain isolate/`compute` send). Platform +views, video textures, maps, and native overlays may be absent or incomplete +in repaint-boundary output. When the exploration WebSocket connects and there is no HTTP collector, the controller suppresses new Flutter screenshots for UI-thread performance. @@ -345,9 +355,11 @@ cross-build equivalence. ### 2. Screenshot budget device baselines -Unit thresholds live in `benchmark/screenshot_budget_baseline.dart`. Record -multi-tier device measurements before enabling aggressive degradation in -production profiles. +Unit thresholds live in `benchmark/screenshot_budget_baseline.dart`. The default +rolling budget is 60 ms per 5 s window so eligible captures skip under load +sooner now that paint-generation / dHash coalesce replace the old post-capture +state-signature short circuit. Record multi-tier device measurements before +enabling more aggressive degradation in production profiles. ### 3. Stronger collector acknowledgement diff --git a/docs/integration/collector.md b/docs/integration/collector.md index 459d789..99df3be 100644 --- a/docs/integration/collector.md +++ b/docs/integration/collector.md @@ -65,7 +65,7 @@ The SDK sends: `fingerprintSchemaVersion`; - `type: event`: serialized event payload plus available session/run/action correlation fields; -- `type: frame`: frame metadata followed by a binary PNG message; +- `type: frame`: frame metadata followed by a binary JPEG message; - `type: control_ack`: acknowledgement for supported exploration commands. Incoming JSON control messages are forwarded to the controller. The current @@ -161,7 +161,7 @@ The SDK calls: | --- | --- | | `POST /v1/sessions` | Session lifecycle and identity: `session_start`, `session_identify`, `session_end`, `traits_updated`, `user_changed` | | `POST /v1/events/batch` | JSON event batches | -| `POST /v1/frames` | multipart PNG frame upload | +| `POST /v1/frames` | multipart JPEG frame upload | Every request includes both `X-PMKit-API-Key` and `X-Tugboat-API-Key`, plus platform, build number, version name, and app ID headers. Mobile API keys are @@ -204,8 +204,13 @@ Event payloads contain: schema version. Frame uploads are sorted by numeric frame suffix and sent as multipart files -named `.png`, with `sessionId` and comma-separated `frameNos` fields. +named `.jpg`, with `sessionId` and comma-separated `frameNos` fields. Malformed frame IDs and frames belonging to a stale SDK session are dropped. +Queued frames are uploaded as-is: events reference exact `beforeFrame` / +`afterFrame` IDs, and the multipart protocol has no hash alias, so intermediate +scroll or duplicate-content captures cannot be dropped without breaking those +refs. Backpressure may still drop the oldest pending frames when +`maxPendingFrames` is exceeded. ### Batching, retry, and backpressure diff --git a/docs/integration/production-replay-acceptance.md b/docs/integration/production-replay-acceptance.md index 0b941d6..eb0458b 100644 --- a/docs/integration/production-replay-acceptance.md +++ b/docs/integration/production-replay-acceptance.md @@ -12,7 +12,7 @@ database receipt alone as proof that a replay is correct. ## Current acceptance status -The current SDK release candidate is **0.7.0**, which writes session schema +The current SDK release candidate is **0.7.1**, which writes session schema **v9**. It preserves structural interaction replay while no longer emitting `controlValue`, `controlValueTransition`, or `semanticAnnotation` in event data. Treat the absence of those fields as the expected privacy boundary, not @@ -164,7 +164,7 @@ flows share a session, list the event IDs or timestamps that delimit each flow. Wait until the collector session has finalized and the replay is available in the production website. Filter to the recorded Blend build and SDK version -under test (`0.7.0` for this release), then open every recorded session. +under test (`0.7.1` for this release), then open every recorded session. For each interaction, inspect the actual replay UI and verify: diff --git a/packages/tugboat/CHANGELOG.md b/packages/tugboat/CHANGELOG.md index a84035f..193e748 100644 --- a/packages/tugboat/CHANGELOG.md +++ b/packages/tugboat/CHANGELOG.md @@ -1,3 +1,19 @@ +## 0.7.1 + +### Changed + +- **Screenshot capture performance** — remove the post-capture state-signature + short circuit and replace it with a paint-signature gate that skips the full + GPU readback/encode path when the capture subtree (including nested + `RepaintBoundary`s) has not painted. Diagnostic outcome + `state_signature_short_circuit` is replaced by `paint_generation_unchanged`. +- **Encode path** — JPEG encoding, SHA-256, mask fills, and dHash now run on a + persistent background isolate with transferable RGBA input. dHash coalesce + tolerates Hamming distance ≤ 2. Default screenshot budget is 60 ms / 5 s. +- **Collector uploads** — frame wire format docs corrected to JPEG. Pending + frames are not superseded on enqueue: events reference exact frame IDs and + multipart upload has no hash alias. + ## 0.7.0 ### Added diff --git a/packages/tugboat/README.md b/packages/tugboat/README.md index 8c377eb..0817492 100644 --- a/packages/tugboat/README.md +++ b/packages/tugboat/README.md @@ -5,7 +5,7 @@ checkpoints around meaningful interactions, compact structural anchors, route transitions, scrolling evidence, and optional viewport semantic maps. Capture can be sent to the local exploration WebSocket, the HTTP collector, or both. -The current package version is `0.7.0`. Session JSON writers emit schema +The current package version is `0.7.1`. Session JSON writers emit schema version `9`; compatibility readers should accept versions `6` through `9`. Structural fingerprints use fingerprint schema version `6`. @@ -23,7 +23,7 @@ The package requires Dart 3.9.2 or newer and Flutter 3.35.0 or newer. ```yaml dependencies: - tugboat_dio: ^0.7.0 + tugboat_dio: ^0.7.1 ``` See `packages/tugboat_dio/README.md`. @@ -303,7 +303,7 @@ Call `TugboatReplay.clearDurableOutbox()` on logout/consent revocation. | `viewportSemanticMapMaxBytes` | 48000 | emitted map byte budget | | `sinkFactories` | empty | extra `TugboatCaptureSinkFactory` adapters | | `outbox` | disabled | durable HTTP outbox configuration | -| `screenshotBudget` | defaults | degraded-capture skip window / budget | +| `screenshotBudget` | 60ms / 5s window | degraded-capture skip window / budget | ### Legacy gesture projection deprecation @@ -433,9 +433,10 @@ cancellations, evidence, legacy projections, and diagnostics. Frames can be triggered by initial startup, taps, scrolls, routes, lifecycle, or explicit controller calls. Capture requests are serialized and coalesced. -The SDK first skips repeated state signatures, then uses a small dHash to avoid -PNG encoding for visually unchanged content, and finally deduplicates encoded -frames by content hash. +When the capture boundary has not painted since the last accepted frame, the +SDK reuses that frame without GPU readback. Otherwise it uses a small dHash +(Hamming distance ≤ 2) to avoid JPEG encoding for near-identical content, and +finally deduplicates encoded frames by content hash. Pointer coordinates in event data (`x`, `y`, and swipe `startX`/`startY`) are Flutter global logical-pixel coordinates from the pointer event. The SDK @@ -515,7 +516,7 @@ to an inferred event. The closed outcome vocabulary is: | `fresh_accepted` | A fresh frame was accepted. | | `exact_content_reused` | An exact content hash reused a compatible frame. | | `perceptual_hash_coalesced` | A perceptual hash reused a compatible frame. | -| `state_signature_short_circuit` | Compatible semantic state made capture unnecessary. | +| `paint_generation_unchanged` | The capture subtree had not painted since the last accepted frame. | | `screenshot_budget_skip` | Degraded screenshot budget skipped eligible work. | | `superseded_route_epoch` | Navigation superseded the request's route epoch. | | `paint_readiness_timeout` | A fresh paint did not become available in time. | @@ -547,7 +548,8 @@ sink registration API has not been published. - Platform views, maps, video textures, and native overlays may be absent or incomplete in repaint-boundary screenshots and structural walks. -- Screenshot readback and PNG encoding perform UI-thread work at checkpoints. +- Screenshot readback and JPEG encoding perform UI-thread and background-isolate + work at checkpoints. - Runtime activation/deactivation requires a host rebuild, and activation IDs are not yet the emitted session IDs. - There is no automatic Android intent-extra/deep-link bridge, offline file diff --git a/packages/tugboat/benchmark/screenshot_budget_baseline.dart b/packages/tugboat/benchmark/screenshot_budget_baseline.dart index 09732a1..e04c2ce 100644 --- a/packages/tugboat/benchmark/screenshot_budget_baseline.dart +++ b/packages/tugboat/benchmark/screenshot_budget_baseline.dart @@ -5,7 +5,7 @@ /// enabling aggressive degradation in production profiles. class ScreenshotBudgetBaseline { static const window = Duration(seconds: 5); - static const budgetMicros = 80 * 1000; + static const budgetMicros = 60 * 1000; static const maxAvgEncodeMicros = 50 * 1000; static const maxAvgReadbackMicros = 40 * 1000; } diff --git a/packages/tugboat/example/pubspec.yaml b/packages/tugboat/example/pubspec.yaml index 04b104a..d75a6e1 100644 --- a/packages/tugboat/example/pubspec.yaml +++ b/packages/tugboat/example/pubspec.yaml @@ -32,7 +32,7 @@ resolution: workspace dependencies: flutter: sdk: flutter - tugboat: ^0.7.0 + tugboat: ^0.7.1 # The following adds the Cupertino Icons font to your application. # Use with the CupertinoIcons class for iOS style icons. diff --git a/packages/tugboat/lib/src/capture_boundary.dart b/packages/tugboat/lib/src/capture_boundary.dart index b221d76..486453f 100644 --- a/packages/tugboat/lib/src/capture_boundary.dart +++ b/packages/tugboat/lib/src/capture_boundary.dart @@ -18,9 +18,77 @@ class TugboatCaptureRenderBoundary extends RenderRepaintBoundary { int get paintGeneration => _paintGeneration; + /// Subtree paint signature for this boundary, including nested + /// [RepaintBoundary] retained layers. Prefer this over [paintGeneration] + /// alone when deciding whether a capture would observe new pixels. + int get subtreePaintSignature => tugboatSubtreePaintSignature(this); + @override void paint(PaintingContext context, Offset offset) { super.paint(context, offset); _paintGeneration++; } } + +/// Paint-activity signature for [root] and every descendant [RenderRepaintBoundary]. +/// +/// Nested repaint boundaries can rasterize without invoking [root]'s [paint], +/// so an outer paint-generation counter alone cannot decide whether a capture +/// would observe new pixels. This mixes: +/// - outer [TugboatCaptureRenderBoundary.paintGeneration] when present; +/// - render-object identity for each nested boundary; +/// - retained [PictureLayer] picture identity under each boundary, which +/// changes when that boundary paints. +int tugboatSubtreePaintSignature(RenderObject root) { + var signature = 0; + + void visitLayer(Layer? layer) { + if (layer == null) { + return; + } + signature = Object.hash(signature, identityHashCode(layer)); + if (layer is PictureLayer) { + signature = Object.hash(signature, identityHashCode(layer.picture)); + } + if (layer is TransformLayer) { + final matrix = layer.transform; + if (matrix != null) { + signature = Object.hash(signature, matrix.storage.hashCode); + } + } + if (layer is OpacityLayer) { + signature = Object.hash(signature, layer.alpha); + } + if (layer is OffsetLayer) { + signature = Object.hash( + signature, + layer.offset.dx, + layer.offset.dy, + ); + } + if (layer is ContainerLayer) { + var child = layer.firstChild; + while (child != null) { + visitLayer(child); + child = child.nextSibling; + } + } + } + + void visit(RenderObject node) { + if (node is RenderRepaintBoundary) { + signature = Object.hash(signature, identityHashCode(node)); + if (node is TugboatCaptureRenderBoundary) { + signature = Object.hash(signature, node.paintGeneration); + } + // RenderObject.layer is protected; nested-boundary paint detection needs + // the retained layer tree under each RepaintBoundary. + // ignore: invalid_use_of_protected_member + visitLayer(node.layer); + } + node.visitChildren(visit); + } + + visit(root); + return signature; +} diff --git a/packages/tugboat/lib/src/collector_http_sink.dart b/packages/tugboat/lib/src/collector_http_sink.dart index 17b66b0..4afb450 100644 --- a/packages/tugboat/lib/src/collector_http_sink.dart +++ b/packages/tugboat/lib/src/collector_http_sink.dart @@ -170,7 +170,12 @@ class CollectorHttpSink implements TugboatCaptureSink { ); return; } - _pendingFrames.add(_PendingFrameUpload(frameNo: frameNo, bytes: bytes)); + _pendingFrames.add( + _PendingFrameUpload( + frameNo: frameNo, + bytes: bytes, + ), + ); _trimPendingFrames(); // While a frame upload is retrying, rely on the periodic flush timer. if (!_framesNeedRetry) { @@ -586,17 +591,21 @@ class CollectorHttpSink implements TugboatCaptureSink { final result = _classifyResponse(response.statusCode); _framesNeedRetry = result == _SendResult.retry; if (_framesNeedRetry) { - _pendingFrames.insertAll(0, uploads); - _trimPendingFrames(); + _requeueFailedUploads(uploads); } } catch (_) { if (!_isCurrentEpoch(epoch)) return; _framesNeedRetry = true; - _pendingFrames.insertAll(0, uploads); - _trimPendingFrames(); + _requeueFailedUploads(uploads); } } + void _requeueFailedUploads(List<_PendingFrameUpload> uploads) { + // Events reference exact frame IDs; never drop uploads on retry. + _pendingFrames.insertAll(0, uploads); + _trimPendingFrames(); + } + void _trimPendingEvents() { if (_pendingEvents.length <= _config.maxPendingEvents) return; final dropped = _pendingEvents.length - _config.maxPendingEvents; @@ -674,7 +683,10 @@ class _PendingSessionLifecycle { } class _PendingFrameUpload { - const _PendingFrameUpload({required this.frameNo, required this.bytes}); + const _PendingFrameUpload({ + required this.frameNo, + required this.bytes, + }); final int frameNo; final Uint8List bytes; diff --git a/packages/tugboat/lib/src/controller.dart b/packages/tugboat/lib/src/controller.dart index 2037aa2..e05f380 100644 --- a/packages/tugboat/lib/src/controller.dart +++ b/packages/tugboat/lib/src/controller.dart @@ -135,7 +135,7 @@ enum _CaptureOutcome { freshAccepted, exactContentReused, perceptualHashCoalesced, - stateSignatureShortCircuit, + paintGenerationUnchanged, screenshotBudgetSkip, noFrameAvailable, noCompatibleFrame, @@ -151,8 +151,7 @@ extension on _CaptureOutcome { _CaptureOutcome.freshAccepted => 'fresh_accepted', _CaptureOutcome.exactContentReused => 'exact_content_reused', _CaptureOutcome.perceptualHashCoalesced => 'perceptual_hash_coalesced', - _CaptureOutcome.stateSignatureShortCircuit => - 'state_signature_short_circuit', + _CaptureOutcome.paintGenerationUnchanged => 'paint_generation_unchanged', _CaptureOutcome.screenshotBudgetSkip => 'screenshot_budget_skip', _CaptureOutcome.noFrameAvailable => 'no_frame_available', _CaptureOutcome.noCompatibleFrame => 'no_compatible_frame', @@ -843,10 +842,8 @@ class TugboatReplayController extends ChangeNotifier { static String _routeCaptureKey(String? navigatorId) => navigatorId ?? ''; final Map _scrollTrackers = {}; - String? _lastCapturedStateSignature; final Set _emittedInventories = {}; SemanticsHandle? _semanticsHandle; - String? _lastDHash; late final ViewportSemanticSession _viewportSemantics = ViewportSemanticSession( config: config, @@ -1089,6 +1086,7 @@ class TugboatReplayController extends ChangeNotifier { completedAtMs: atMs, completionStateAnchor: _snapshotStateAnchor(_currentStateAnchor), ); + _capturer?.rememberAcceptedPaintGeneration(); _trim(); return frameId; } @@ -1342,6 +1340,11 @@ class TugboatReplayController extends ChangeNotifier { } _explorationSink = null; _collectorHttpSink = null; + final capturer = _capturer; + _capturer = null; + if (capturer != null) { + unawaited(capturer.dispose()); + } super.dispose(); } @@ -1430,11 +1433,10 @@ class TugboatReplayController extends ChangeNotifier { _hashToFrameId.clear(); _frameProvenance.clear(); _frameReuseObservations.clear(); - _lastCapturedStateSignature = null; _lastCaptureFailure = null; _emittedInventories.clear(); _viewportSemantics.clear(); - _lastDHash = null; + _capturer?.resetCoalesceState(); _captureDiagnosticOutcomes.clear(); _captureDiagnosticTotal = 0; _lastCaptureDiagnosticOutcome = null; @@ -1647,6 +1649,37 @@ class TugboatReplayController extends ChangeNotifier { return frameId; } + _CaptureExecution _reuseWithoutCapture({ + required _CaptureRequestContext context, + required _CaptureOutcome outcome, + required String reuseReason, + int queueWaitMicros = 0, + bool recordBudget = false, + String? budgetDropReason, + }) { + final compatible = _compatibleFrameFor(context); + if (compatible == null) { + return const _CaptureExecution(outcome: _CaptureOutcome.noCompatibleFrame); + } + if (recordBudget) { + _screenshotBudget.record( + queueWaitMicros: queueWaitMicros, + readbackMicros: 0, + encodeMicros: 0, + encodedBytes: 0, + dropReason: budgetDropReason ?? reuseReason, + ); + } + _reuseCompatibleFrame(compatible, context, reuseReason); + _refreshStateAnchor(); + _maybeEmitSceneInventory(); + return _CaptureExecution( + outcome: outcome, + frameId: compatible, + reuseReason: reuseReason, + ); + } + /// Emits exactly one bounded, sanitized resolution record for a logical /// request. This deliberately records a taxonomy value rather than the /// underlying exception so replay telemetry never contains app data. @@ -2060,8 +2093,8 @@ class TugboatReplayController extends ChangeNotifier { ? 'content_hash' : outcome == _CaptureOutcome.perceptualHashCoalesced ? 'dhash' - : outcome == _CaptureOutcome.stateSignatureShortCircuit - ? 'state_signature' + : outcome == _CaptureOutcome.paintGenerationUnchanged + ? 'paint_generation' : null, ); } @@ -2080,28 +2113,23 @@ class TugboatReplayController extends ChangeNotifier { return const _CaptureExecution(outcome: _CaptureOutcome.noFrameAvailable); } + final compatibleFrame = _compatibleFrameFor(context); + final hasCompatibleFrame = compatibleFrame != null; + final eligibleToSkip = freshness == _CaptureFreshness.reusable && trigger != TugboatFrameTrigger.initial && trigger != TugboatFrameTrigger.lifecycle && config.screenshotBudget.skipEligibleWhenDegraded && _screenshotBudget.shouldSkipEligible; - final compatibleSkipFrame = eligibleToSkip - ? _compatibleFrameFor(context) - : null; - if (compatibleSkipFrame != null) { - _screenshotBudget.record( - queueWaitMicros: queueWaitMicros, - readbackMicros: 0, - encodeMicros: 0, - encodedBytes: 0, - dropReason: 'budget', - ); - _refreshStateAnchor(); - _maybeEmitSceneInventory(); - return _CaptureExecution( + if (eligibleToSkip && hasCompatibleFrame) { + return _reuseWithoutCapture( + context: context, outcome: _CaptureOutcome.screenshotBudgetSkip, - frameId: compatibleSkipFrame, + reuseReason: 'budget', + queueWaitMicros: queueWaitMicros, + recordBudget: true, + budgetDropReason: 'budget', ); } @@ -2148,77 +2176,89 @@ class TugboatReplayController extends ChangeNotifier { _beginCapture(); try { - final attempt = await capturer.captureAttempt( - lastDHash: _lastDHash, - // A freshness-sensitive request needs a new logical observation even - // when its pixels match. Reusing the old frame would also reuse its - // old completion-state provenance. - force: force || requiresFreshPaint, - waitForFrame: true, - requireFreshPaint: requiresFreshPaint, - cancelled: captureCancellation, - isCurrent: () => - _captureContextStillCurrent( - context, - captureGeneration, - captureSession, - ) && - !_capturePaused && - !_skipCapture, - ); - final result = attempt.result; - if (result == null || - _disposed || - !_captureContextStillCurrent(context, captureGeneration, session)) { - _lastCaptureFailure = attempt.failure; - if (attempt.failure != ScreenshotCaptureFailure.cancelled && - _captureContextStillCurrent( - context, - captureGeneration, - captureSession, - )) { - _screenshotBudget.record( - queueWaitMicros: queueWaitMicros, - frameWaitMicros: attempt.frameWaitMicros, - readbackMicros: 0, - encodeMicros: 0, - encodedBytes: 0, - dropReason: attempt.failure?.name ?? 'capture_failed', - ); - } - return _CaptureExecution( - outcome: - !_captureContextStillCurrent( + var allowPaintSkip = + trigger != TugboatFrameTrigger.initial && hasCompatibleFrame; + var captureForce = force || requiresFreshPaint || !hasCompatibleFrame; + late ScreenshotCaptureAttempt attempt; + late ScreenshotCaptureResult result; + + for (var paintRetry = 0; paintRetry < 2; paintRetry++) { + attempt = await capturer.captureAttempt( + // A freshness-sensitive request needs a new logical observation even + // when its pixels match. Reusing the old frame would also reuse its + // old completion-state provenance. + force: captureForce, + waitForFrame: true, + requireFreshPaint: requiresFreshPaint, + allowPaintGenerationSkip: allowPaintSkip, + cancelled: captureCancellation, + isCurrent: () => + _captureContextStillCurrent( context, captureGeneration, captureSession, - ) - ? _CaptureOutcome.supersededRoute - : _diagnosticOutcomeForFailure(attempt.failure), - failure: attempt.failure, - cancellationReason: - attempt.failure == ScreenshotCaptureFailure.cancelled - ? 'superseded_route' - : null, + ) && + !_capturePaused && + !_skipCapture, ); + final attemptResult = attempt.result; + if (attemptResult == null || + _disposed || + !_captureContextStillCurrent(context, captureGeneration, session)) { + _lastCaptureFailure = attempt.failure; + if (attempt.failure != ScreenshotCaptureFailure.cancelled && + _captureContextStillCurrent( + context, + captureGeneration, + captureSession, + )) { + _screenshotBudget.record( + queueWaitMicros: queueWaitMicros, + frameWaitMicros: attempt.frameWaitMicros, + readbackMicros: 0, + encodeMicros: 0, + encodedBytes: 0, + dropReason: attempt.failure?.name ?? 'capture_failed', + ); + } + return _CaptureExecution( + outcome: + !_captureContextStillCurrent( + context, + captureGeneration, + captureSession, + ) + ? _CaptureOutcome.supersededRoute + : _diagnosticOutcomeForFailure(attempt.failure), + failure: attempt.failure, + cancellationReason: + attempt.failure == ScreenshotCaptureFailure.cancelled + ? 'superseded_route' + : null, + ); + } + _lastCaptureFailure = null; + + if (attemptResult.skippedByPaintGeneration) { + final reuseExecution = _reuseWithoutCapture( + context: context, + outcome: _CaptureOutcome.paintGenerationUnchanged, + reuseReason: 'paint_generation', + ); + if (reuseExecution.outcome == _CaptureOutcome.noCompatibleFrame && + paintRetry == 0) { + allowPaintSkip = false; + captureForce = true; + continue; + } + return reuseExecution; + } + + result = attemptResult; + break; } - _lastCaptureFailure = null; + _refreshStateAnchor(); - final signature = _currentStateAnchor?.signature ?? ''; - if (!force && - !requiresFreshPaint && - trigger != TugboatFrameTrigger.initial && - signature.isNotEmpty && - signature == _lastCapturedStateSignature) { - final compatible = _compatibleFrameFor(context); - return _CaptureExecution( - outcome: compatible == null - ? _CaptureOutcome.noCompatibleFrame - : _CaptureOutcome.stateSignatureShortCircuit, - frameId: compatible, - reuseReason: compatible == null ? null : 'state_signature', - ); - } final activeSession = session; final completionStateAnchor = _snapshotStateAnchor(_refreshStateAnchor()); @@ -2233,13 +2273,14 @@ class TugboatReplayController extends ChangeNotifier { ); if (result.skippedByDHash) { - if (result.dHash != null) { - _lastDHash = result.dHash; - } final compatible = _compatibleFrameFor(context); final reused = compatible == null ? null : _reuseCompatibleFrame(compatible, context, 'dhash'); + if (reused != null) { + capturer.commitAcceptedPaintGeneration(result.paintGeneration); + capturer.commitAcceptedDHash(result.dHash); + } return _CaptureExecution( outcome: reused == null ? _CaptureOutcome.noCompatibleFrame @@ -2255,12 +2296,8 @@ class TugboatReplayController extends ChangeNotifier { existingId != null && _isFrameCompatible(existingId, context)) { _reuseCompatibleFrame(existingId, context, 'content_hash'); - if (result.dHash != null) { - _lastDHash = result.dHash; - } - if (signature.isNotEmpty) { - _lastCapturedStateSignature = signature; - } + capturer.commitAcceptedPaintGeneration(result.paintGeneration); + capturer.commitAcceptedDHash(result.dHash); _maybeEmitSceneInventory(); return _CaptureExecution( outcome: _CaptureOutcome.exactContentReused, @@ -2300,12 +2337,8 @@ class TugboatReplayController extends ChangeNotifier { completedAtMs: atMs, completionStateAnchor: completionStateAnchor, ); - if (result.dHash != null) { - _lastDHash = result.dHash; - } - if (signature.isNotEmpty) { - _lastCapturedStateSignature = signature; - } + capturer.commitAcceptedPaintGeneration(result.paintGeneration); + capturer.commitAcceptedDHash(result.dHash); _maybeEmitSceneInventory(); _sinkHub?.recordFrame( frame, diff --git a/packages/tugboat/lib/src/health.dart b/packages/tugboat/lib/src/health.dart index 7d490e6..7948d31 100644 --- a/packages/tugboat/lib/src/health.dart +++ b/packages/tugboat/lib/src/health.dart @@ -193,7 +193,7 @@ class TugboatSanitizedFailure { class TugboatScreenshotBudgetTracker { TugboatScreenshotBudgetTracker({ this.window = const Duration(seconds: 5), - this.budgetMicros = 80 * 1000, // 80ms per window default + this.budgetMicros = 60 * 1000, // 60ms per window default }); Duration window; diff --git a/packages/tugboat/lib/src/perceptual_hash.dart b/packages/tugboat/lib/src/perceptual_hash.dart index 14f9fef..5af212b 100644 --- a/packages/tugboat/lib/src/perceptual_hash.dart +++ b/packages/tugboat/lib/src/perceptual_hash.dart @@ -11,15 +11,25 @@ String computeDHashFromRgba(Uint8List rgba, int width, int height) { final pixels = List.filled(hashWidth * hashHeight, 0); for (var y = 0; y < hashHeight; y++) { - final srcY = ((y + 0.5) * height / hashHeight).floor().clamp(0, height - 1); + final y0 = (y * height / hashHeight).floor(); + final y1 = ((y + 1) * height / hashHeight).floor().clamp(y0 + 1, height); for (var x = 0; x < hashWidth; x++) { - final srcX = ((x + 0.5) * width / hashWidth).floor().clamp(0, width - 1); - final offset = (srcY * width + srcX) * 4; - if (offset + 2 >= rgba.length) continue; - final r = rgba[offset]; - final g = rgba[offset + 1]; - final b = rgba[offset + 2]; - pixels[y * hashWidth + x] = ((r * 299 + g * 587 + b * 114) ~/ 1000); + final x0 = (x * width / hashWidth).floor(); + final x1 = ((x + 1) * width / hashWidth).floor().clamp(x0 + 1, width); + var sum = 0; + var count = 0; + for (var sy = y0; sy < y1; sy++) { + for (var sx = x0; sx < x1; sx++) { + final offset = (sy * width + sx) * 4; + if (offset + 2 >= rgba.length) continue; + final r = rgba[offset]; + final g = rgba[offset + 1]; + final b = rgba[offset + 2]; + sum += ((r * 299 + g * 587 + b * 114) ~/ 1000); + count++; + } + } + pixels[y * hashWidth + x] = count == 0 ? 0 : sum ~/ count; } } @@ -33,3 +43,30 @@ String computeDHashFromRgba(Uint8List rgba, int width, int height) { } return bits.toString(); } + +/// Hamming distance between two equal-length bit strings. +/// +/// Returns a large sentinel when either input is empty or lengths differ so +/// callers can treat malformed hashes as non-matching. +int dHashHammingDistance(String a, String b) { + if (a.isEmpty || b.isEmpty || a.length != b.length) { + return 0x7fffffff; + } + var distance = 0; + for (var i = 0; i < a.length; i++) { + if (a.codeUnitAt(i) != b.codeUnitAt(i)) distance++; + } + return distance; +} + +/// Maximum Hamming distance treated as visually unchanged for coalesce. +/// +/// A couple of flipped bits typically cover single-pixel anti-alias shimmer +/// without swallowing meaningful UI changes. +const int dHashMatchDistance = 2; + +/// Whether [candidate] is close enough to [previous] to skip JPEG encoding. +bool dHashVisuallyMatches(String? previous, String candidate) { + if (previous == null || previous.isEmpty || candidate.isEmpty) return false; + return dHashHammingDistance(previous, candidate) <= dHashMatchDistance; +} diff --git a/packages/tugboat/lib/src/replay_config.dart b/packages/tugboat/lib/src/replay_config.dart index 1d7eb19..0919f76 100644 --- a/packages/tugboat/lib/src/replay_config.dart +++ b/packages/tugboat/lib/src/replay_config.dart @@ -67,7 +67,9 @@ TugboatViewportSemanticPolicy resolveViewportSemanticPolicy({ class TugboatScreenshotBudgetConfig { const TugboatScreenshotBudgetConfig({ this.window = const Duration(seconds: 5), - this.budgetMicros = 80 * 1000, + // 60ms / 5s: engage eligible-capture skipping sooner under load now that + // post-capture state-signature short circuit no longer filters work. + this.budgetMicros = 60 * 1000, this.skipEligibleWhenDegraded = true, }); diff --git a/packages/tugboat/lib/src/screenshot_capturer.dart b/packages/tugboat/lib/src/screenshot_capturer.dart index 410f5a7..64882ba 100644 --- a/packages/tugboat/lib/src/screenshot_capturer.dart +++ b/packages/tugboat/lib/src/screenshot_capturer.dart @@ -1,22 +1,17 @@ import 'dart:async'; import 'dart:ui' as ui; -import 'package:crypto/crypto.dart'; import 'package:flutter/foundation.dart'; import 'package:flutter/material.dart'; import 'package:flutter/rendering.dart'; import 'package:flutter/scheduler.dart'; -import 'package:image/image.dart' as img; import 'anchors.dart'; import 'capture_boundary.dart'; -import 'perceptual_hash.dart'; +import 'screenshot_encode.dart'; +import 'screenshot_encode_isolate.dart'; import 'screenshot_mask_level.dart'; -/// JPEG quality for emitted frames. Screenshots are photo-heavy once masking -/// is relaxed, where JPEG is ~5x smaller than PNG at comparable legibility. -const int _jpegQuality = 80; - /// Why a screenshot request could not produce a fresh rendered observation. /// /// This remains an internal capture detail: callers continue to receive a @@ -56,24 +51,6 @@ class ScreenshotCaptureAttempt { final int frameWaitMicros; } -class _JpegEncodeRequest { - const _JpegEncodeRequest(this.rgba, this.width, this.height); - - final Uint8List rgba; - final int width; - final int height; -} - -Uint8List _encodeJpeg(_JpegEncodeRequest request) { - final image = img.Image.fromBytes( - width: request.width, - height: request.height, - bytes: request.rgba.buffer, - order: img.ChannelOrder.rgba, - ); - return Uint8List.fromList(img.encodeJpg(image, quality: _jpegQuality)); -} - class MaskRect { const MaskRect(this.rect); @@ -93,6 +70,8 @@ class ScreenshotCaptureResult { required this.encodeMicros, this.maskMicros = 0, this.skippedByDHash = false, + this.skippedByPaintGeneration = false, + this.paintGeneration, }); final Uint8List bytes; @@ -106,6 +85,15 @@ class ScreenshotCaptureResult { final int encodeMicros; final int maskMicros; final bool skippedByDHash; + final bool skippedByPaintGeneration; + + /// Subtree paint signature observed at gate time for this attempt. + /// + /// Covers the capture root and nested [RepaintBoundary] activity (see + /// [tugboatSubtreePaintSignature]). The controller commits this via + /// [ScreenshotCapturer.commitAcceptedPaintGeneration] only after accepting + /// a new frame or successfully reusing a compatible one. + final int? paintGeneration; } class ScreenshotCapturer { @@ -115,18 +103,61 @@ class ScreenshotCapturer { required this.anchorResolver, this.pixelRatio = 0.75, @visibleForTesting Future Function()? frameWaiter, + @visibleForTesting ScreenshotEncoder? encoder, }) : _frameWaiter = - frameWaiter ?? (() => SchedulerBinding.instance.endOfFrame); + frameWaiter ?? (() => SchedulerBinding.instance.endOfFrame), + _encoder = encoder ?? IsolateScreenshotEncoder(); final GlobalKey boundaryKey; final double pixelRatio; final TugboatScreenshotMaskLevel maskLevel; final Future Function() _frameWaiter; + final ScreenshotEncoder _encoder; /// Shared resolver used for frame-scoped element maps when masking. final AnchorResolver anchorResolver; String? _lastDHash; + int? _lastAcceptedPaintSignature; + TugboatCaptureRenderBoundary? _lastAcceptedBoundary; + + /// Clears perceptual-hash and paint-signature coalesce state. + void resetCoalesceState() { + _lastDHash = null; + _lastAcceptedPaintSignature = null; + _lastAcceptedBoundary = null; + } + + /// Records the current subtree paint signature as accepted without a + /// GPU readback (for example [TugboatReplayController.debugSeedFrame]). + void rememberAcceptedPaintGeneration() { + final renderObject = boundaryKey.currentContext?.findRenderObject(); + if (renderObject is TugboatCaptureRenderBoundary) { + _lastAcceptedBoundary = renderObject; + _lastAcceptedPaintSignature = renderObject.subtreePaintSignature; + } + } + + /// Commits a pre-capture [paintSignature] after accept or reuse. + /// + /// Callers must pass the signature from the capture result (gate-time), not + /// a freshly recomputed value, so paints during encode cannot poison the + /// next skip decision. + void commitAcceptedPaintGeneration(int? paintSignature) { + final renderObject = boundaryKey.currentContext?.findRenderObject(); + if (paintSignature == null || + renderObject is! TugboatCaptureRenderBoundary) { + return; + } + _lastAcceptedBoundary = renderObject; + _lastAcceptedPaintSignature = paintSignature; + } + + /// Commits [dHash] after the controller accepts or reuses a frame. + void commitAcceptedDHash(String? dHash) { + if (dHash == null || dHash.isEmpty) return; + _lastDHash = dHash; + } /// Wait for one bounded compositor opportunity. The timeout does not try /// to cancel Flutter's frame future (which is shared by the binding); it @@ -194,10 +225,10 @@ class ScreenshotCapturer { /// snapshot the exact boundary before waiting; a remount during that wait is /// a failure, never permission to read a previous layer or a replacement. Future captureAttempt({ - String? lastDHash, bool force = false, bool waitForFrame = true, bool requireFreshPaint = false, + bool allowPaintGenerationSkip = true, Duration frameTimeout = const Duration(seconds: 2), bool Function()? isCurrent, Future? cancelled, @@ -307,8 +338,8 @@ class ScreenshotCapturer { // ignore: use_build_context_synchronously currentContext, currentBoundary, - lastDHash: lastDHash, force: force, + allowPaintGenerationSkip: allowPaintGenerationSkip, ); if (isCurrent != null && !isCurrent()) { return ScreenshotCaptureAttempt( @@ -340,11 +371,9 @@ class ScreenshotCapturer { } Future capture({ - String? lastDHash, bool force = false, bool waitForFrame = true, }) async => (await captureAttempt( - lastDHash: lastDHash, force: force, waitForFrame: waitForFrame, )).result; @@ -352,12 +381,38 @@ class ScreenshotCapturer { Future _captureReadyBoundary( Element context, RenderRepaintBoundary boundary, { - required String? lastDHash, required bool force, + required bool allowPaintGenerationSkip, }) async { final rootRender = boundary; final boundaryOrigin = boundary.localToGlobal(Offset.zero); final boundaryLogicalRect = boundaryOrigin & boundary.size; + // Gate-time subtree signature: nested RepaintBoundary paints are included + // even when this outer boundary's paintGeneration is unchanged. + final paintSignature = boundary is TugboatCaptureRenderBoundary + ? boundary.subtreePaintSignature + : null; + final scaledWidth = (boundary.size.width * pixelRatio).ceil().clamp(1, 1 << 20); + final scaledHeight = (boundary.size.height * pixelRatio).ceil().clamp(1, 1 << 20); + + if (allowPaintGenerationSkip && + !force && + paintSignature != null && + identical(boundary, _lastAcceptedBoundary) && + paintSignature == _lastAcceptedPaintSignature) { + return ScreenshotCaptureResult( + bytes: Uint8List(0), + contentHash: '', + width: scaledWidth, + height: scaledHeight, + boundaryLogicalRect: boundaryLogicalRect, + masked: false, + captureMicros: 0, + encodeMicros: 0, + skippedByPaintGeneration: true, + paintGeneration: paintSignature, + ); + } final List maskRects; final maskClock = Stopwatch()..start(); @@ -383,155 +438,73 @@ class ScreenshotCapturer { readbackClock.stop(); } try { - final scaledWidth = image.width; - final scaledHeight = image.height; - ui.Image rasterImage = image; - - var maskMicros = maskClock.elapsedMicroseconds; - if (maskRects.isNotEmpty) { - maskClock - ..reset() - ..start(); - try { - final recorder = ui.PictureRecorder(); - final canvas = Canvas(recorder); - canvas.drawImage(image, Offset.zero, Paint()); - - final maskPaint = Paint()..color = const Color(0xFF1A1A1A); - for (final mask in maskRects) { - final scaled = Rect.fromLTWH( - mask.rect.left * pixelRatio, - mask.rect.top * pixelRatio, - mask.rect.width * pixelRatio, - mask.rect.height * pixelRatio, - ); - if (scaled.width > 0 && scaled.height > 0) { - canvas.drawRect(scaled, maskPaint); - } - } - - final picture = recorder.endRecording(); - try { - rasterImage = await picture.toImage(scaledWidth, scaledHeight); - } finally { - picture.dispose(); - } - } catch (_) { - throw const _ScreenshotCaptureException( - ScreenshotCaptureFailure.maskFailed, - ); - } finally { - maskClock.stop(); - maskMicros += maskClock.elapsedMicroseconds; - } + final imageWidth = image.width; + final imageHeight = image.height; + + // Scale mask rects into capture pixel space once. Mask fills are applied + // in the encode worker so we avoid a second full-size picture.toImage. + final maskClockTotal = Stopwatch()..start(); + final scaledMasks = Float64List(maskRects.length * 4); + for (var i = 0; i < maskRects.length; i++) { + final rect = maskRects[i].rect; + final base = i * 4; + scaledMasks[base] = rect.left * pixelRatio; + scaledMasks[base + 1] = rect.top * pixelRatio; + scaledMasks[base + 2] = rect.right * pixelRatio; + scaledMasks[base + 3] = rect.bottom * pixelRatio; } + maskClockTotal.stop(); + final maskMicros = + maskClock.elapsedMicroseconds + maskClockTotal.elapsedMicroseconds; + final encodeClock = Stopwatch()..start(); try { - final encodeClock = Stopwatch()..start(); - try { - final quickDHash = await _dHashFromThumbnail(rasterImage); - final compareDHash = lastDHash ?? _lastDHash; - if (!force && - quickDHash != null && - compareDHash != null && - quickDHash == compareDHash) { - return ScreenshotCaptureResult( - bytes: Uint8List(0), - contentHash: '', - dHash: quickDHash, - width: scaledWidth, - height: scaledHeight, - boundaryLogicalRect: boundaryLogicalRect, - masked: maskRects.isNotEmpty, - captureMicros: readbackClock.elapsedMicroseconds, - encodeMicros: encodeClock.elapsedMicroseconds, - maskMicros: maskMicros, - skippedByDHash: true, - ); - } - - final byteData = await rasterImage.toByteData( - format: ui.ImageByteFormat.rawRgba, - ); - if (byteData == null) { - throw const _ScreenshotCaptureException( - ScreenshotCaptureFailure.encodingFailed, - ); - } - final jpeg = await compute( - _encodeJpeg, - _JpegEncodeRequest( - byteData.buffer.asUint8List(), - scaledWidth, - scaledHeight, - ), - ); - final contentHash = sha256.convert(jpeg).toString(); - if (quickDHash != null) { - _lastDHash = quickDHash; - } - return ScreenshotCaptureResult( - bytes: jpeg, - contentHash: contentHash, - dHash: quickDHash, - width: scaledWidth, - height: scaledHeight, - boundaryLogicalRect: boundaryLogicalRect, - masked: maskRects.isNotEmpty, - captureMicros: readbackClock.elapsedMicroseconds, - encodeMicros: encodeClock.elapsedMicroseconds, - maskMicros: maskMicros, - ); - } on _ScreenshotCaptureException { - rethrow; - } catch (_) { + final byteData = await image.toByteData( + format: ui.ImageByteFormat.rawRgba, + ); + if (byteData == null) { throw const _ScreenshotCaptureException( ScreenshotCaptureFailure.encodingFailed, ); - } finally { - encodeClock.stop(); } + final encoded = await _encoder.encode( + ScreenshotEncodeInput( + rgba: byteData.buffer.asUint8List(), + width: imageWidth, + height: imageHeight, + maskRects: scaledMasks, + lastDHash: _lastDHash, + force: force, + ), + ); + return ScreenshotCaptureResult( + bytes: encoded.bytes, + contentHash: encoded.contentHash, + dHash: encoded.dHash, + width: imageWidth, + height: imageHeight, + boundaryLogicalRect: boundaryLogicalRect, + masked: maskRects.isNotEmpty, + captureMicros: readbackClock.elapsedMicroseconds, + encodeMicros: encodeClock.elapsedMicroseconds, + maskMicros: maskMicros, + skippedByDHash: encoded.skippedByDHash, + paintGeneration: paintSignature, + ); + } on _ScreenshotCaptureException { + rethrow; + } catch (_) { + throw const _ScreenshotCaptureException( + ScreenshotCaptureFailure.encodingFailed, + ); } finally { - if (!identical(rasterImage, image)) { - rasterImage.dispose(); - } + encodeClock.stop(); } } finally { image.dispose(); } } - Future _dHashFromThumbnail(ui.Image source) async { - const hashWidth = 9; - const hashHeight = 8; - final recorder = ui.PictureRecorder(); - final canvas = Canvas(recorder); - canvas.drawImageRect( - source, - Rect.fromLTWH(0, 0, source.width.toDouble(), source.height.toDouble()), - Rect.fromLTWH(0, 0, hashWidth.toDouble(), hashHeight.toDouble()), - Paint()..filterQuality = FilterQuality.low, - ); - final picture = recorder.endRecording(); - final ui.Image thumb; - try { - thumb = await picture.toImage(hashWidth, hashHeight); - } finally { - picture.dispose(); - } - try { - final bytes = await thumb.toByteData(format: ui.ImageByteFormat.rawRgba); - if (bytes == null) return null; - return computeDHashFromRgba( - bytes.buffer.asUint8List(), - hashWidth, - hashHeight, - ); - } finally { - thumb.dispose(); - } - } - List _collectMaskRects(Element root, RenderBox ancestor) { return anchorResolver .collectMaskRects(rootRender: ancestor, shouldMask: _shouldMask) @@ -603,4 +576,6 @@ class ScreenshotCapturer { } return provider is AssetBundleImageProvider; } + + Future dispose() => _encoder.dispose(); } diff --git a/packages/tugboat/lib/src/screenshot_encode.dart b/packages/tugboat/lib/src/screenshot_encode.dart new file mode 100644 index 0000000..3858367 --- /dev/null +++ b/packages/tugboat/lib/src/screenshot_encode.dart @@ -0,0 +1,138 @@ +import 'dart:typed_data'; + +import 'package:crypto/crypto.dart'; +import 'package:flutter/foundation.dart'; +import 'package:image/image.dart' as img; + +import 'perceptual_hash.dart'; + +/// JPEG quality for emitted frames. Screenshots are photo-heavy once masking +/// is relaxed, where JPEG is ~5x smaller than PNG at comparable legibility. +const int screenshotJpegQuality = 80; + +/// Encoded JPEG bytes plus hashes used for coalesce / session dedup. +class ScreenshotEncodeResult { + const ScreenshotEncodeResult({ + required this.bytes, + required this.contentHash, + this.dHash, + this.skippedByDHash = false, + }); + + final Uint8List bytes; + final String contentHash; + final String? dHash; + final bool skippedByDHash; +} + +/// RGBA frame plus encode options passed to [ScreenshotEncoder]. +class ScreenshotEncodeInput { + const ScreenshotEncodeInput({ + required this.rgba, + required this.width, + required this.height, + this.maskRects, + this.lastDHash, + this.force = false, + }); + + final Uint8List rgba; + final int width; + final int height; + + /// Pixel-space mask rectangles as flat `[left, top, right, bottom, ...]`. + final Float64List? maskRects; + final String? lastDHash; + final bool force; + + Float64List get maskRectsOrEmpty => maskRects ?? Float64List(0); +} + +/// Dark fill used for masked regions (matches the previous canvas mask color). +const int _maskFillR = 0x1a; +const int _maskFillG = 0x1a; +const int _maskFillB = 0x1a; +const int _maskFillA = 0xff; + +void applyMaskRectsInPlace({ + required Uint8List rgba, + required int width, + required int height, + required Float64List maskRects, +}) { + if (maskRects.isEmpty) return; + for (var i = 0; i + 3 < maskRects.length; i += 4) { + final left = maskRects[i].floor().clamp(0, width); + final top = maskRects[i + 1].floor().clamp(0, height); + final right = maskRects[i + 2].ceil().clamp(0, width); + final bottom = maskRects[i + 3].ceil().clamp(0, height); + if (right <= left || bottom <= top) continue; + for (var y = top; y < bottom; y++) { + var offset = (y * width + left) * 4; + for (var x = left; x < right; x++) { + rgba[offset] = _maskFillR; + rgba[offset + 1] = _maskFillG; + rgba[offset + 2] = _maskFillB; + rgba[offset + 3] = _maskFillA; + offset += 4; + } + } + } +} + +/// Pure encode path: mask fills → dHash (Hamming≤2) → optional JPEG → SHA-256. +ScreenshotEncodeResult encodeScreenshotRgba(ScreenshotEncodeInput input) { + final rgba = input.rgba; + final maskRects = input.maskRects; + if (maskRects != null && maskRects.isNotEmpty) { + applyMaskRectsInPlace( + rgba: rgba, + width: input.width, + height: input.height, + maskRects: maskRects, + ); + } + final dHash = computeDHashFromRgba(rgba, input.width, input.height); + if (!input.force && dHashVisuallyMatches(input.lastDHash, dHash)) { + return ScreenshotEncodeResult( + bytes: Uint8List(0), + contentHash: '', + dHash: dHash, + skippedByDHash: true, + ); + } + final image = img.Image.fromBytes( + width: input.width, + height: input.height, + bytes: rgba.buffer, + bytesOffset: rgba.offsetInBytes, + rowStride: input.width * 4, + order: img.ChannelOrder.rgba, + ); + final jpeg = Uint8List.fromList( + img.encodeJpg(image, quality: screenshotJpegQuality), + ); + return ScreenshotEncodeResult( + bytes: jpeg, + contentHash: sha256.convert(jpeg).toString(), + dHash: dHash.isEmpty ? null : dHash, + ); +} + +/// Encodes screenshot RGBA off the UI thread or inline for tests. +abstract class ScreenshotEncoder { + Future encode(ScreenshotEncodeInput input); + + Future dispose(); +} + +/// Runs [encodeScreenshotRgba] on the calling isolate (for widget tests). +class InlineScreenshotEncoder implements ScreenshotEncoder { + @override + Future encode(ScreenshotEncodeInput input) async { + return encodeScreenshotRgba(input); + } + + @override + Future dispose() => Future.value(); +} diff --git a/packages/tugboat/lib/src/screenshot_encode_isolate.dart b/packages/tugboat/lib/src/screenshot_encode_isolate.dart new file mode 100644 index 0000000..979deef --- /dev/null +++ b/packages/tugboat/lib/src/screenshot_encode_isolate.dart @@ -0,0 +1,250 @@ +import 'dart:async'; +import 'dart:isolate'; + +import 'package:flutter/foundation.dart'; + +import 'screenshot_encode.dart'; + +class ScreenshotEncodeIsolateCommand { + const ScreenshotEncodeIsolateCommand({ + required this.jobId, + required this.rgba, + required this.width, + required this.height, + required this.maskRects, + this.lastDHash, + required this.force, + }); + + final int jobId; + final TransferableTypedData rgba; + final int width; + final int height; + final Float64List maskRects; + final String? lastDHash; + final bool force; +} + +class ScreenshotEncodeIsolateReply { + const ScreenshotEncodeIsolateReply._({ + required this.jobId, + this.result, + this.message, + }); + + const ScreenshotEncodeIsolateReply.success({ + required int jobId, + required ScreenshotEncodeResult result, + }) : this._(jobId: jobId, result: result); + + const ScreenshotEncodeIsolateReply.failure({ + required int jobId, + required String message, + }) : this._(jobId: jobId, message: message); + + final int jobId; + final ScreenshotEncodeResult? result; + final String? message; + + bool get isSuccess => result != null; +} + +ScreenshotEncodeResult _encodeInCompute(ScreenshotEncodeInput input) { + return encodeScreenshotRgba(input); +} + +/// [compute]-based encoder for tests that need async completion without a +/// persistent isolate (for example under FakeAsync). +class ComputeScreenshotEncoder implements ScreenshotEncoder { + @override + Future encode(ScreenshotEncodeInput input) { + return compute(_encodeInCompute, input); + } + + @override + Future dispose() => Future.value(); +} + +/// Production encoder backed by a persistent isolate with +/// [TransferableTypedData] transport. +class IsolateScreenshotEncoder implements ScreenshotEncoder { + Isolate? _isolate; + SendPort? _commands; + ReceivePort? _responses; + StreamSubscription? _subscription; + Completer? _starting; + var _nextJobId = 0; + final Map> _pending = + >{}; + var _disposed = false; + + /// Ensures the worker isolate is running. Safe to call repeatedly. + Future ensureStarted() async { + if (_disposed) { + throw StateError('IsolateScreenshotEncoder is disposed'); + } + if (_commands != null) return; + final inFlight = _starting; + if (inFlight != null) { + await inFlight.future; + if (_disposed) { + throw StateError('IsolateScreenshotEncoder is disposed'); + } + return; + } + final starting = Completer(); + _starting = starting; + try { + final handshake = Completer(); + final responses = ReceivePort(); + _responses = responses; + _subscription = responses.listen((message) { + if (!handshake.isCompleted && message is SendPort) { + handshake.complete(message); + return; + } + _onReply(message); + }); + _isolate = await Isolate.spawn( + _screenshotEncodeIsolateMain, + responses.sendPort, + debugName: 'tugboat-screenshot-encode', + ); + final commands = await handshake.future.timeout( + const Duration(seconds: 5), + ); + if (_disposed) { + await _tearDown(); + throw StateError('IsolateScreenshotEncoder is disposed'); + } + _commands = commands; + starting.complete(); + } catch (error, stack) { + if (!starting.isCompleted) { + starting.completeError(error, stack); + } + await _tearDown(); + rethrow; + } finally { + if (identical(_starting, starting)) { + _starting = null; + } + } + } + + void _onReply(dynamic message) { + if (message is! ScreenshotEncodeIsolateReply) { + _failAllPending( + StateError('encode reply had unexpected type: ${message.runtimeType}'), + ); + return; + } + final pending = _pending.remove(message.jobId); + if (pending == null || pending.isCompleted) return; + if (message.isSuccess) { + pending.complete(message.result!); + return; + } + pending.completeError( + StateError(message.message ?? 'encode reply missing error detail'), + ); + } + + void _failAllPending(Object error) { + final pending = Map>.from(_pending); + _pending.clear(); + for (final completer in pending.values) { + if (!completer.isCompleted) { + completer.completeError(error); + } + } + } + + @override + Future encode(ScreenshotEncodeInput input) async { + if (_disposed) { + throw StateError('IsolateScreenshotEncoder is disposed'); + } + await ensureStarted(); + if (_disposed) { + throw StateError('IsolateScreenshotEncoder is disposed'); + } + final commands = _commands; + if (commands == null) { + throw StateError('IsolateScreenshotEncoder failed to start'); + } + final jobId = _nextJobId++; + final completer = Completer(); + if (_disposed) { + throw StateError('IsolateScreenshotEncoder is disposed'); + } + _pending[jobId] = completer; + // fromList copies once into a transferable buffer on this isolate; the + // worker then materializes without a second full-frame copy. This still + // pays one sender-side memcpy versus keeping pixels in shared storage. + commands.send( + ScreenshotEncodeIsolateCommand( + jobId: jobId, + rgba: TransferableTypedData.fromList([input.rgba]), + width: input.width, + height: input.height, + maskRects: input.maskRectsOrEmpty, + lastDHash: input.lastDHash, + force: input.force, + ), + ); + return completer.future; + } + + @override + Future dispose() async { + if (_disposed) return; + _disposed = true; + _failAllPending(StateError('IsolateScreenshotEncoder disposed')); + await _tearDown(); + } + + Future _tearDown() async { + await _subscription?.cancel(); + _subscription = null; + _responses?.close(); + _responses = null; + _commands = null; + _isolate?.kill(priority: Isolate.immediate); + _isolate = null; + } +} + +@pragma('vm:entry-point') +void _screenshotEncodeIsolateMain(SendPort replyTo) { + final commands = ReceivePort(); + replyTo.send(commands.sendPort); + commands.listen((message) { + if (message is! ScreenshotEncodeIsolateCommand) { + return; + } + try { + final rgba = message.rgba.materialize().asUint8List(); + final encoded = encodeScreenshotRgba( + ScreenshotEncodeInput( + rgba: rgba, + width: message.width, + height: message.height, + maskRects: message.maskRects, + lastDHash: message.lastDHash, + force: message.force, + ), + ); + replyTo.send( + ScreenshotEncodeIsolateReply.success(jobId: message.jobId, result: encoded), + ); + } catch (error) { + replyTo.send( + ScreenshotEncodeIsolateReply.failure( + jobId: message.jobId, + message: error.toString(), + ), + ); + } + }); +} diff --git a/packages/tugboat/lib/src/sdk_version.dart b/packages/tugboat/lib/src/sdk_version.dart index 1b05b3c..67f8467 100644 --- a/packages/tugboat/lib/src/sdk_version.dart +++ b/packages/tugboat/lib/src/sdk_version.dart @@ -1,3 +1,3 @@ // Keep this in sync with packages/tugboat/pubspec.yaml. The SDK version test // reads pubspec.yaml directly so release bumps fail fast if this drifts. -const tugboatSdkVersion = '0.7.0'; +const tugboatSdkVersion = '0.7.1'; diff --git a/packages/tugboat/pubspec.yaml b/packages/tugboat/pubspec.yaml index 628b74f..c160584 100644 --- a/packages/tugboat/pubspec.yaml +++ b/packages/tugboat/pubspec.yaml @@ -1,7 +1,7 @@ name: tugboat description: >- Screenshot-based session replay with compact interaction anchors for Tugboat. -version: 0.7.0 +version: 0.7.1 repository: https://github.com/blendto/tugboat-flutter issue_tracker: https://github.com/blendto/tugboat-flutter/issues homepage: https://github.com/blendto/tugboat-flutter diff --git a/packages/tugboat/test/collector_http_sink_test.dart b/packages/tugboat/test/collector_http_sink_test.dart index 358ae9a..0504a42 100644 --- a/packages/tugboat/test/collector_http_sink_test.dart +++ b/packages/tugboat/test/collector_http_sink_test.dart @@ -403,6 +403,111 @@ void main() { expect(framePosts, isEmpty); sink.dispose(); }); + + test('uploads pending scroll and tap frames without superseding', () async { + frameStatus = 503; + final sink = CollectorHttpSink(config: configForServer()); + final session = createSession(); + sink.startSession(session); + await Future.delayed(const Duration(milliseconds: 50)); + + sink.recordFrame( + const TugboatFrame( + id: 'frame-0', + atMs: 0, + width: 1, + height: 1, + contentHash: 'scroll-a', + trigger: TugboatFrameTrigger.scroll, + ), + Uint8List.fromList([0]), + sessionId: session.id, + ); + await Future.delayed(const Duration(milliseconds: 30)); + sink.recordFrame( + const TugboatFrame( + id: 'frame-1', + atMs: 1, + width: 1, + height: 1, + contentHash: 'scroll-b', + trigger: TugboatFrameTrigger.scroll, + ), + Uint8List.fromList([1]), + sessionId: session.id, + ); + sink.recordFrame( + const TugboatFrame( + id: 'frame-2', + atMs: 2, + width: 1, + height: 1, + contentHash: 'tap-final', + trigger: TugboatFrameTrigger.tap, + ), + Uint8List.fromList([2]), + sessionId: session.id, + ); + + framePosts.clear(); + frameStatus = 202; + await sink.flush(); + await Future.delayed(const Duration(milliseconds: 50)); + + expect(framePosts, isNotEmpty); + final uploaded = framePosts.last['frameNos'] as List; + expect(uploaded, ['0', '1', '2']); + sink.dispose(); + }); + + test('retries failed frame uploads without dropping earlier frames', () async { + frameStatus = 503; + final sink = CollectorHttpSink(config: configForServer()); + final session = createSession(); + sink.startSession(session); + await Future.delayed(const Duration(milliseconds: 50)); + + sink.recordFrame( + const TugboatFrame( + id: 'frame-0', + atMs: 0, + width: 1, + height: 1, + contentHash: 'scroll-old', + trigger: TugboatFrameTrigger.scroll, + ), + Uint8List.fromList([0]), + sessionId: session.id, + ); + await Future.delayed(const Duration(milliseconds: 50)); + + sink.recordFrame( + const TugboatFrame( + id: 'frame-1', + atMs: 1, + width: 1, + height: 1, + contentHash: 'tap-new', + trigger: TugboatFrameTrigger.tap, + ), + Uint8List.fromList([1]), + sessionId: session.id, + ); + + framePosts.clear(); + frameStatus = 202; + await sink.flush(); + await Future.delayed(const Duration(milliseconds: 50)); + + expect(framePosts, isNotEmpty); + final uploadedNos = framePosts + .expand((post) => (post['frameNos'] as List).cast()) + .toSet(); + expect(uploadedNos.contains('0'), isTrue); + expect(uploadedNos.contains('1'), isTrue); + sink.dispose(); + }); + test('skips duplicate session_start events in the event batch', () async { final sink = CollectorHttpSink(config: configForServer()); final session = createSession(); diff --git a/packages/tugboat/test/replay/capture_diagnostics_test.dart b/packages/tugboat/test/replay/capture_diagnostics_test.dart index 5586399..a94dde0 100644 --- a/packages/tugboat/test/replay/capture_diagnostics_test.dart +++ b/packages/tugboat/test/replay/capture_diagnostics_test.dart @@ -104,7 +104,7 @@ void main() { 'fresh_accepted', 'exact_content_reused', 'perceptual_hash_coalesced', - 'state_signature_short_circuit', + 'paint_generation_unchanged', 'screenshot_budget_skip', 'no_frame_available', 'no_compatible_frame', diff --git a/packages/tugboat/test/replay/screenshot_encode_isolate_test.dart b/packages/tugboat/test/replay/screenshot_encode_isolate_test.dart new file mode 100644 index 0000000..1efe849 --- /dev/null +++ b/packages/tugboat/test/replay/screenshot_encode_isolate_test.dart @@ -0,0 +1,77 @@ +import 'dart:typed_data'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:image/image.dart' as img; +import 'package:tugboat/src/screenshot_encode.dart'; +import 'package:tugboat/src/screenshot_encode_isolate.dart'; + +Uint8List _solidRed() { + final rgba = Uint8List(4 * 8 * 8); + for (var i = 0; i < rgba.length; i += 4) { + rgba[i] = 255; + rgba[i + 3] = 255; + } + return rgba; +} + +ScreenshotEncodeInput _solidRedInput({String? lastDHash, bool force = false}) { + return ScreenshotEncodeInput( + rgba: _solidRed(), + width: 8, + height: 8, + lastDHash: lastDHash, + force: force, + ); +} + +void main() { + test('persistent encode isolate returns jpeg bytes and content hash', () async { + final worker = IsolateScreenshotEncoder(); + addTearDown(worker.dispose); + final first = await worker + .encode(_solidRedInput()) + .timeout(const Duration(seconds: 10)); + final second = await worker + .encode(_solidRedInput()) + .timeout(const Duration(seconds: 10)); + expect(first.bytes, isNotEmpty); + expect(first.contentHash, isNotEmpty); + expect(second.contentHash, first.contentHash); + }); + + test('encode isolate applies mask fills before jpeg encoding', () async { + final worker = IsolateScreenshotEncoder(); + addTearDown(worker.dispose); + final masked = await worker + .encode( + ScreenshotEncodeInput( + rgba: _solidRed(), + width: 8, + height: 8, + maskRects: Float64List.fromList([0, 0, 4, 4]), + ), + ) + .timeout(const Duration(seconds: 10)); + final decoded = img.decodeJpg(masked.bytes)!; + final maskedPixel = decoded.getPixel(1, 1); + final unmaskedPixel = decoded.getPixel(6, 6); + expect(maskedPixel.r.toInt(), lessThan(80)); + expect(unmaskedPixel.r.toInt(), greaterThan(200)); + }); + + test('encode isolate skips jpeg when masked dHash matches', () async { + final worker = IsolateScreenshotEncoder(); + addTearDown(worker.dispose); + final first = await worker + .encode(_solidRedInput()) + .timeout(const Duration(seconds: 10)); + final second = await worker + .encode(_solidRedInput(lastDHash: first.dHash)) + .timeout(const Duration(seconds: 10)); + expect(first.skippedByDHash, isFalse); + expect(first.dHash, isNotNull); + expect(second.skippedByDHash, isTrue); + expect(second.bytes, isEmpty); + expect(second.dHash, first.dHash); + }); +} diff --git a/packages/tugboat/test/replay/screenshot_fresh_paint_test.dart b/packages/tugboat/test/replay/screenshot_fresh_paint_test.dart index 219b825..f51f13e 100644 --- a/packages/tugboat/test/replay/screenshot_fresh_paint_test.dart +++ b/packages/tugboat/test/replay/screenshot_fresh_paint_test.dart @@ -7,6 +7,7 @@ import 'package:tugboat/src/anchors.dart'; import 'package:tugboat/src/capture_boundary.dart'; import 'package:tugboat/src/health.dart'; import 'package:tugboat/src/screenshot_capturer.dart'; +import 'package:tugboat/src/screenshot_encode.dart'; import 'package:tugboat/src/screenshot_mask_level.dart'; Widget _scene(GlobalKey boundaryKey, Color color) => Directionality( @@ -53,6 +54,7 @@ void main() { anchorResolver: AnchorResolver(rootKey: boundaryKey), pixelRatio: 1, frameWaiter: () => redFrame.future, + encoder: InlineScreenshotEncoder(), ); await tester.pumpWidget(_scene(boundaryKey, Colors.red)); @@ -64,6 +66,7 @@ void main() { ); await tester.pump(); final red = await redFuture; + addTearDown(capturer.dispose); await tester.pumpWidget(_scene(boundaryKey, Colors.blue)); final blueFrame = Completer(); @@ -73,7 +76,9 @@ void main() { anchorResolver: AnchorResolver(rootKey: boundaryKey), pixelRatio: 1, frameWaiter: () => blueFrame.future, + encoder: InlineScreenshotEncoder(), ); + addTearDown(blueCapturer.dispose); final blueFuture = blueCapturer.captureAttempt(requireFreshPaint: true); await tester.pump(); blueFrame.complete(); @@ -111,6 +116,7 @@ void main() { maskLevel: TugboatScreenshotMaskLevel.explicitOnly, anchorResolver: AnchorResolver(rootKey: boundaryKey), frameWaiter: () => frame.future, + encoder: InlineScreenshotEncoder(), ); await tester.pumpWidget(_scene(boundaryKey, Colors.red)); @@ -135,6 +141,7 @@ void main() { maskLevel: TugboatScreenshotMaskLevel.explicitOnly, anchorResolver: AnchorResolver(rootKey: boundaryKey), frameWaiter: () => Future.value(), + encoder: InlineScreenshotEncoder(), ); await tester.pumpWidget(_scene(boundaryKey, Colors.red)); @@ -152,6 +159,7 @@ void main() { maskLevel: TugboatScreenshotMaskLevel.explicitOnly, anchorResolver: AnchorResolver(rootKey: boundaryKey), frameWaiter: () => Future.value(), + encoder: InlineScreenshotEncoder(), ); await tester.pumpWidget(_plainRepaintScene(boundaryKey)); @@ -168,6 +176,7 @@ void main() { maskLevel: TugboatScreenshotMaskLevel.explicitOnly, anchorResolver: AnchorResolver(rootKey: boundaryKey), frameWaiter: () => frame.future, + encoder: InlineScreenshotEncoder(), ); await tester.pumpWidget(_scene(boundaryKey, Colors.red)); @@ -192,6 +201,7 @@ void main() { maskLevel: TugboatScreenshotMaskLevel.explicitOnly, anchorResolver: AnchorResolver(rootKey: boundaryKey), frameWaiter: () => frame.future, + encoder: InlineScreenshotEncoder(), ); await tester.pumpWidget(_scene(boundaryKey, Colors.red)); @@ -214,6 +224,7 @@ void main() { maskLevel: TugboatScreenshotMaskLevel.explicitOnly, anchorResolver: AnchorResolver(rootKey: boundaryKey), frameWaiter: () => frame.future, + encoder: InlineScreenshotEncoder(), ); await tester.pumpWidget(_scene(boundaryKey, Colors.red)); @@ -234,6 +245,127 @@ void main() { expect(attempt.failure, ScreenshotCaptureFailure.boundaryReplaced); }); + testWidgets( + 'paint-generation gate skips when unchanged, captures after repaint, force bypasses', + (tester) async { + final boundaryKey = GlobalKey(); + final capturer = ScreenshotCapturer( + boundaryKey: boundaryKey, + maskLevel: TugboatScreenshotMaskLevel.explicitOnly, + anchorResolver: AnchorResolver(rootKey: boundaryKey), + pixelRatio: 1, + frameWaiter: () => Future.value(), + encoder: InlineScreenshotEncoder(), + ); + addTearDown(capturer.dispose); + await tester.pumpWidget(_scene(boundaryKey, Colors.red)); + + final first = await tester.runAsync( + () => capturer.captureAttempt(force: true), + ); + expect(first, isNotNull); + expect(first!.failure, isNull); + expect(first.result, isNotNull); + expect(first.result!.skippedByPaintGeneration, isFalse); + capturer.commitAcceptedPaintGeneration(first.result!.paintGeneration); + + final skipped = await capturer.captureAttempt(); + expect(skipped.failure, isNull); + expect(skipped.result, isNotNull); + expect(skipped.result!.skippedByPaintGeneration, isTrue); + expect(skipped.result!.bytes, isEmpty); + + final boundary = + boundaryKey.currentContext!.findRenderObject()! + as TugboatCaptureRenderBoundary; + final generationBeforeRepaint = boundary.paintGeneration; + boundary.markNeedsPaint(); + await tester.pump(); + expect(boundary.paintGeneration, greaterThan(generationBeforeRepaint)); + + final afterRepaint = await tester.runAsync( + () => capturer.captureAttempt(), + ); + expect(afterRepaint, isNotNull); + expect(afterRepaint!.failure, isNull); + expect(afterRepaint.result, isNotNull); + expect(afterRepaint.result!.skippedByPaintGeneration, isFalse); + + capturer.commitAcceptedPaintGeneration( + afterRepaint.result!.paintGeneration, + ); + final forced = await tester.runAsync( + () => capturer.captureAttempt(force: true), + ); + expect(forced, isNotNull); + expect(forced!.failure, isNull); + expect(forced.result, isNotNull); + expect(forced.result!.skippedByPaintGeneration, isFalse); + expect(forced.result!.bytes, isNotEmpty); + }, + ); + + testWidgets( + 'paint-generation gate does not skip when a nested RepaintBoundary paints', + (tester) async { + final boundaryKey = GlobalKey(); + var nestedColor = Colors.red; + final capturer = ScreenshotCapturer( + boundaryKey: boundaryKey, + maskLevel: TugboatScreenshotMaskLevel.explicitOnly, + anchorResolver: AnchorResolver(rootKey: boundaryKey), + pixelRatio: 1, + frameWaiter: () => Future.value(), + encoder: InlineScreenshotEncoder(), + ); + addTearDown(capturer.dispose); + + Widget scene() => Directionality( + textDirection: TextDirection.ltr, + child: Center( + child: TugboatCaptureBoundary( + key: boundaryKey, + child: SizedBox( + width: 80, + height: 80, + child: RepaintBoundary( + child: ColoredBox(color: nestedColor), + ), + ), + ), + ), + ); + + await tester.pumpWidget(scene()); + final first = await tester.runAsync( + () => capturer.captureAttempt(force: true), + ); + expect(first, isNotNull); + expect(first!.failure, isNull); + expect(first.result, isNotNull); + capturer.commitAcceptedPaintGeneration(first.result!.paintGeneration); + + final outer = + boundaryKey.currentContext!.findRenderObject()! + as TugboatCaptureRenderBoundary; + final outerGeneration = outer.paintGeneration; + final signatureBefore = tugboatSubtreePaintSignature(outer); + + nestedColor = Colors.blue; + await tester.pumpWidget(scene()); + // Nested boundary owns the paint; outer generation must stay put so this + // exercises the unsafe outer-only gate scenario. + expect(outer.paintGeneration, outerGeneration); + expect(tugboatSubtreePaintSignature(outer), isNot(signatureBefore)); + + final afterNested = await tester.runAsync(() => capturer.captureAttempt()); + expect(afterNested, isNotNull); + expect(afterNested!.failure, isNull); + expect(afterNested.result, isNotNull); + expect(afterNested.result!.skippedByPaintGeneration, isFalse); + }, + ); + test('screenshot budget reports independent capture-stage metrics', () { final tracker = TugboatScreenshotBudgetTracker(); tracker.record( diff --git a/packages/tugboat/test/tugboat_replay_test.dart b/packages/tugboat/test/tugboat_replay_test.dart index abcccc6..abb0564 100644 --- a/packages/tugboat/test/tugboat_replay_test.dart +++ b/packages/tugboat/test/tugboat_replay_test.dart @@ -8,7 +8,8 @@ import 'package:flutter_test/flutter_test.dart'; import 'package:image/image.dart' as img; import 'package:tugboat/tugboat.dart'; import 'package:tugboat/src/anchors.dart'; -import 'package:tugboat/src/perceptual_hash.dart' show computeDHashFromRgba; +import 'package:tugboat/src/perceptual_hash.dart' + show computeDHashFromRgba, dHashHammingDistance, dHashVisuallyMatches; import 'helpers/json_roundtrip.dart'; @@ -1422,6 +1423,41 @@ void main() { expect(first.length, 64); }); + test('perceptual hash match tolerates small hamming distance', () { + final base = '0' * 64; + final oneBit = '1${'0' * 63}'; + final twoBits = '11${'0' * 62}'; + final threeBits = '111${'0' * 61}'; + expect(dHashHammingDistance(base, oneBit), 1); + expect(dHashVisuallyMatches(base, oneBit), isTrue); + expect(dHashVisuallyMatches(base, twoBits), isTrue); + expect(dHashVisuallyMatches(base, threeBits), isFalse); + }); + + test('perceptual hash aggregates cell pixels on large buffers', () { + const width = 80; + const height = 80; + final rgba = Uint8List(width * height * 4); + for (var y = 0; y < height; y++) { + for (var x = 0; x < width; x++) { + final offset = (y * width + x) * 4; + final isCorner = x < width ~/ 8 && y < height ~/ 8; + final gray = isCorner ? 0 : 200; + rgba[offset] = gray; + rgba[offset + 1] = gray; + rgba[offset + 2] = gray; + rgba[offset + 3] = 255; + } + } + final uniform = computeDHashFromRgba( + Uint8List.fromList(List.filled(width * height * 4, 200)), + width, + height, + ); + final withCorner = computeDHashFromRgba(rgba, width, height); + expect(withCorner, isNot(uniform)); + }); + testWidgets('skips tap capture when route capture is pending', ( tester, ) async { diff --git a/packages/tugboat_dio/CHANGELOG.md b/packages/tugboat_dio/CHANGELOG.md index ec3baf7..d4f9591 100644 --- a/packages/tugboat_dio/CHANGELOG.md +++ b/packages/tugboat_dio/CHANGELOG.md @@ -1,3 +1,10 @@ +## 0.7.1 + +### Changed + +- Compatibility release for `tugboat` 0.7.1. The Dio adapter has no runtime + behavior changes. + ## 0.7.0 ### Changed diff --git a/packages/tugboat_dio/README.md b/packages/tugboat_dio/README.md index bc1f548..8c6cfd3 100644 --- a/packages/tugboat_dio/README.md +++ b/packages/tugboat_dio/README.md @@ -5,14 +5,14 @@ status, outcome, and duration into an active Tugboat session. HTTP error responses additionally retain bounded JSON/text bodies. Successful response bodies, headers, queries, raw transport errors, and stack traces are omitted. -Requires `tugboat` `0.7.0` (lockstep). +Requires `tugboat` `0.7.1` (lockstep). ## Install ```yaml dependencies: - tugboat: ^0.7.0 - tugboat_dio: ^0.7.0 + tugboat: ^0.7.1 + tugboat_dio: ^0.7.1 ``` ## Usage diff --git a/packages/tugboat_dio/pubspec.yaml b/packages/tugboat_dio/pubspec.yaml index 5170e1d..9317c56 100644 --- a/packages/tugboat_dio/pubspec.yaml +++ b/packages/tugboat_dio/pubspec.yaml @@ -2,7 +2,7 @@ name: tugboat_dio description: >- Dio interceptor that records safe, bounded network evidence into an active Tugboat capture session. -version: 0.7.0 +version: 0.7.1 repository: https://github.com/blendto/tugboat-flutter issue_tracker: https://github.com/blendto/tugboat-flutter/issues homepage: https://github.com/blendto/tugboat-flutter @@ -18,7 +18,7 @@ dependencies: dio: ^5.4.0 flutter: sdk: flutter - tugboat: ^0.7.0 + tugboat: ^0.7.1 dev_dependencies: flutter_lints: ^5.0.0