From 7dff6f032b1f8b5548fb8634af7f853ce505ba1b Mon Sep 17 00:00:00 2001 From: Chinmay-KB Date: Mon, 3 Aug 2026 19:13:59 +0530 Subject: [PATCH 1/2] feat(replay): remove control value tracking --- docs/README.md | 4 +- docs/design/capture-and-fingerprint.md | 45 +- packages/tugboat/CHANGELOG.md | 25 + packages/tugboat/README.md | 71 +- .../example/lib/screens/profile_screen.dart | 23 - packages/tugboat/example/pubspec.yaml | 2 +- .../tugboat/example/test/widget_test.dart | 51 +- packages/tugboat/lib/src/anchor_resolver.dart | 302 +---- packages/tugboat/lib/src/anchors.dart | 3 - packages/tugboat/lib/src/control_value.dart | 1018 --------------- packages/tugboat/lib/src/controller.dart | 128 +- .../lib/src/interaction_transaction.dart | 30 +- packages/tugboat/lib/src/models.dart | 6 +- packages/tugboat/lib/src/sdk_version.dart | 2 +- .../lib/src/semantics_flags_compat.dart | 35 - packages/tugboat/lib/src/widget_roles.dart | 6 +- packages/tugboat/lib/tugboat.dart | 15 - packages/tugboat/pubspec.yaml | 2 +- packages/tugboat/test/control_value_test.dart | 1125 ----------------- packages/tugboat/test/fingerprint_test.dart | 79 ++ .../tugboat/test/helpers/json_roundtrip.dart | 2 +- .../release_compatibility_matrix_test.dart | 41 +- .../test/semantics_flags_compat_test.dart | 67 +- .../tugboat/test/tugboat_replay_test.dart | 93 +- 24 files changed, 262 insertions(+), 2913 deletions(-) delete mode 100644 packages/tugboat/lib/src/control_value.dart delete mode 100644 packages/tugboat/test/control_value_test.dart diff --git a/docs/README.md b/docs/README.md index 933f394..befef09 100644 --- a/docs/README.md +++ b/docs/README.md @@ -28,8 +28,8 @@ verified in their own repositories. ## Current compatibility -- package version: `0.4.17`; -- session JSON schema: `8`; +- package version: `0.5.0`; +- session JSON schema: `9`; - fingerprint schema: `6`; - minimum Dart SDK: `3.9.2`; - minimum Flutter SDK: `3.35.0`. diff --git a/docs/design/capture-and-fingerprint.md b/docs/design/capture-and-fingerprint.md index 0142ebc..2acc96d 100644 --- a/docs/design/capture-and-fingerprint.md +++ b/docs/design/capture-and-fingerprint.md @@ -63,7 +63,7 @@ current controller and keeps future calls to `wrapApp` inert. Runtime requiring a host rebuild. `deactivate()` tears capture down through the same gate. Pause/hidden flush pending delivery; detach ends the session once. -Identity fields (session schema **v7**; readers accept v6): +Identity fields (session schema **v9**; compatibility readers accept v6–v9): - `activationRequestId` — host request correlation - `captureSessionId` — SDK-emitted session (`session.id`) @@ -76,7 +76,10 @@ emits exact build and fingerprint-schema provenance only. ## Session and event model The controller owns one bounded, in-memory `TugboatSession`. Serialized session -JSON is schema version `7`. Readers accept schema versions `6` and `7`. +JSON is schema version `9`. Compatibility readers accept schema versions +`6` through `9`. Schema v9 does not write `controlValue`, +`controlValueTransition`, or `semanticAnnotation` in event `data`; +those fields are optional historic data in older sessions only. The session stores: @@ -235,43 +238,9 @@ Developer-authored identity strings can still be emitted: - widget type names or configured `widgetNames` replacements; - canonical structural paths. -Interaction events may also carry a `controlValue` payload (schema version 4) for -valued controls (checkbox, switch, radio, slider, dropdown / menu item, chip) -and for hit targets that expose Flutter semantic annotations: - -- bools and numbers are emitted literally; -- enums and developer identifiers are emitted literally; -- arbitrary strings, including numeric strings and single-word values, are - emitted literally; -- explicit custom-control values can be supplied with - `TugboatControlValueScope`, including a stable `controlKey`, optional unit, - and numeric `min`, `max`, and `step` metadata. - -`tap` includes a `controlValue` snapshot sampled at pointer-down. -`tap_settled` uses the distinct `controlValueTransition` contract with -`before` / `after` snapshots. Its post-callback sample stays bound to the -original hit element, so later taps, route changes, or dismissed overlays -cannot donate unrelated control state. Slider drags that become `swipe` events -carry a `controlValue` snapshot sampled at pointer-up. - -When a typed widget value is unavailable (custom GestureDetector rows, bottom -sheets, etc.), the SDK still samples `SemanticsProperties` / live semantics -nodes under the pointer and records raw `semanticValue` / `semanticLabel`. -Standard controls may include both widget -state and semantic annotations under `sources: ["semantics","widget"]`. - -Independently, every interaction event (`tap`, `tap_settled`, `swipe`, -`scroll_start`, `scroll_end`) may carry a top-level `semanticAnnotation` -payload (schema version 2) whenever Flutter semantics expose an identifier, label, value, or -selection flag on the target. This covers ordinary buttons and scrollables as -well as valued controls. The field is named `semanticAnnotation` to avoid -colliding with `tap_settled.data.settleObservation.semantic` (state-signature -change evidence). - Bounds, pointer coordinates, scroll metrics, and masked screenshot pixels are -also capture data. Apps must treat tags, route names, subview labels, and -semantic value/label tokens as telemetry and avoid putting raw user PII in -them. +also capture data. Apps must treat tags, route names, and subview labels as +telemetry and avoid putting user data in them. ## Screenshot pipeline diff --git a/packages/tugboat/CHANGELOG.md b/packages/tugboat/CHANGELOG.md index d056b6e..57cd583 100644 --- a/packages/tugboat/CHANGELOG.md +++ b/packages/tugboat/CHANGELOG.md @@ -1,3 +1,28 @@ +## 0.5.0 + +### Breaking changes + +- **Control-value and semantic-annotation telemetry** — session JSON writers + now emit schema version 9 and no longer write `controlValue`, + `controlValueTransition`, or `semanticAnnotation` in event `data`. + Readers that support historic schemas should continue to tolerate versions + 6–8, where those fields may be present. +- **Removed public barrel exports** — + `TugboatEncodedControlScalar`, `TugboatVisibleControlValue`, + `TugboatControlValueScope`, `TugboatControlValue`, and + `TugboatSemanticAnnotation`. +- **Removed public schema constants** — + `tugboatControlValueSchemaVersion`, + `tugboatControlValueTransitionSchemaVersion`, and + `tugboatSemanticAnnotationSchemaVersion`. +- **Removed public extraction and merge helpers** — + `tugboatControlValueForWidget`, + `tugboatControlValueFromSemanticsProperties`, + `tugboatControlValueFromSemanticsNode`, + `tugboatSemanticAnnotationFromProperties`, + `tugboatSemanticAnnotationFromNode`, + `tugboatMergeSemanticAnnotations`, and `tugboatMergeControlValues`. + ## 0.4.18 ### Fixed diff --git a/packages/tugboat/README.md b/packages/tugboat/README.md index f3327a9..4522b6f 100644 --- a/packages/tugboat/README.md +++ b/packages/tugboat/README.md @@ -5,9 +5,9 @@ checkpoints around meaningful interactions, compact structural anchors, route transitions, scrolling evidence, and optional viewport semantic maps. Capture can be sent to the local exploration WebSocket, the HTTP collector, or both. -The current package version is `0.4.18`. Session JSON uses schema version `8` -(readers still accept `6`), and structural fingerprints use fingerprint schema -version `6`. +The current package version is `0.5.0`. Session JSON writers emit schema +version `9`; compatibility readers should accept versions `6` through +`9`. Structural fingerprints use fingerprint schema version `6`. ## Install @@ -19,6 +19,29 @@ import 'package:tugboat/tugboat.dart'; The package requires Dart 3.9.2 or newer and Flutter 3.35.0 or newer. +## Migrating to 0.5.0 + +This is a breaking release. Session JSON written by 0.5.0 uses schema version +`9` and no longer includes `controlValue`, `controlValueTransition`, or +`semanticAnnotation` in event `data`. Consumers reading historical schemas +`6`–`8` should treat those fields as optional historic data; new captures do +not provide them. + +The public `package:tugboat/tugboat.dart` barrel no longer exports: + +- `TugboatEncodedControlScalar`, `TugboatVisibleControlValue`, + `TugboatControlValueScope`, `TugboatControlValue`, and + `TugboatSemanticAnnotation`; +- `tugboatControlValueSchemaVersion`, + `tugboatControlValueTransitionSchemaVersion`, and + `tugboatSemanticAnnotationSchemaVersion`; +- `tugboatControlValueForWidget`, + `tugboatControlValueFromSemanticsProperties`, + `tugboatControlValueFromSemanticsNode`, + `tugboatSemanticAnnotationFromProperties`, + `tugboatSemanticAnnotationFromNode`, + `tugboatMergeSemanticAnnotations`, and `tugboatMergeControlValues`. + ## Minimal integration Install both the app wrapper and navigator observer. Capture is dormant by @@ -209,11 +232,10 @@ Available mask levels are `explicitOnly`, `allTextAndMedia`, `allText`, stay visible; other custom-painted or decorated image surfaces are not classified by this mode, so wrap them in `TugboatSensitive` when needed). -Control values and semantic strings are retained verbatim so session summaries -and aggregate analysis can use values such as slider positions, video duration, -and selected templates. Dynamic list discriminators remain hashed before they -enter canonical paths. Telemetry also includes developer-authored routing and -identity strings where applicable: +The structural telemetry does not retain arbitrary `Text`, accessibility, +tooltip, or icon label strings. Dynamic list discriminators are hashed before +they enter canonical paths. Telemetry does include developer-authored routing +and identity strings where applicable: - route names in `route_change.data` and anchor `routeKey` fields; - `TugboatSubView.label` in state/scroll context; @@ -223,35 +245,10 @@ identity strings where applicable: - normalized bounds, pointer coordinates, scroll metrics, and screenshot pixels after the configured masking policy is applied. -Screenshots and telemetry can contain rendered or semantic user content. Choose -an explicit production masking policy and test custom widgets, platform views, -overlays, and semantic labels before enabling production capture. - -### Explicit custom-control values - -Standard Flutter controls expose their typed state automatically. Wrap custom -controls when the app knows a more useful stable key, unit, or range: - -```dart -TugboatControlValueScope( - controlKey: 'video_duration', - role: 'slider', - unit: 'milliseconds', - min: 1_000, - max: 60_000, - step: 1_000, - value: TugboatVisibleControlValue.duration( - const Duration(seconds: 15), - ), - child: MyDurationSlider(), -) -``` - -For template or preset selection, use a stable enum identifier: - -```dart -value: TugboatVisibleControlValue.enumId('modern_minimal'), -``` +Screenshots are the only captured surface that can contain rendered user +content. Choose an explicit production masking policy and test custom widgets, +platform views, and overlays in the target app before enabling production +capture. ## Event and frame model diff --git a/packages/tugboat/example/lib/screens/profile_screen.dart b/packages/tugboat/example/lib/screens/profile_screen.dart index 8efe0c7..6a52fd2 100644 --- a/packages/tugboat/example/lib/screens/profile_screen.dart +++ b/packages/tugboat/example/lib/screens/profile_screen.dart @@ -16,7 +16,6 @@ class _ProfileScreenState extends State { bool _darkMode = false; double _notificationVolume = 0.6; String _language = 'English'; - int _generationCount = 1; @override Widget build(BuildContext context) { @@ -144,28 +143,6 @@ class _ProfileScreenState extends State { ), ], ), - DemoSection( - title: 'Generation settings', - children: [ - Wrap( - spacing: 8, - children: [ - for (final count in [1, 2, 3, 4]) - Semantics( - label: 'Number of generations', - value: count.toString(), - selected: _generationCount == count, - child: FilledButton( - key: Key('generation-count-$count'), - onPressed: () => - setState(() => _generationCount = count), - child: Text(count.toString()), - ), - ), - ], - ), - ], - ), DemoSection( title: 'Account actions', children: [ diff --git a/packages/tugboat/example/pubspec.yaml b/packages/tugboat/example/pubspec.yaml index 7a3bd08..621d096 100644 --- a/packages/tugboat/example/pubspec.yaml +++ b/packages/tugboat/example/pubspec.yaml @@ -32,7 +32,7 @@ resolution: workspace dependencies: flutter: sdk: flutter - tugboat: ^0.4.0 + tugboat: ^0.5.0 # The following adds the Cupertino Icons font to your application. # Use with the CupertinoIcons class for iOS style icons. diff --git a/packages/tugboat/example/test/widget_test.dart b/packages/tugboat/example/test/widget_test.dart index 1745c97..343b11f 100644 --- a/packages/tugboat/example/test/widget_test.dart +++ b/packages/tugboat/example/test/widget_test.dart @@ -1,23 +1,7 @@ -import 'package:flutter/material.dart'; -import 'package:flutter_test/flutter_test.dart'; -import 'package:tugboat/tugboat.dart'; import 'package:tugboat_example/main.dart'; - -Future _waitForTugboatEvents(WidgetTester tester) async { - for (var attempt = 0; attempt < 12; attempt++) { - await tester.pump(const Duration(milliseconds: 50)); - } -} - -Map _semanticAnnotation(TugboatEvent event) { - final raw = event.data['semanticAnnotation']; - return Map.from(raw! as Map); -} +import 'package:flutter_test/flutter_test.dart'; void main() { - setUp(TugboatReplay.resetForTest); - tearDown(TugboatReplay.resetForTest); - testWidgets('demo app loads home screen', (tester) async { await tester.pumpWidget(const ReplayDemoApp()); await tester.pump(); @@ -28,37 +12,4 @@ void main() { await tester.pump(const Duration(milliseconds: 300)); } }); - - testWidgets( - 'generation count tap emits its semantic parameter label and value', - (tester) async { - await tester.binding.setSurfaceSize(const Size(800, 1200)); - addTearDown(() => tester.binding.setSurfaceSize(null)); - TugboatReplay.activate( - activationRequestId: 'example-semantic-parameter-test', - profile: TugboatCaptureProfile.productionLean, - ); - await tester.pumpWidget(const ReplayDemoApp()); - await _waitForTugboatEvents(tester); - - await tester.tap(find.text('Profile & settings')); - await tester.pumpAndSettle(); - await _waitForTugboatEvents(tester); - - await tester.tap(find.byKey(const Key('generation-count-3'))); - await _waitForTugboatEvents(tester); - - final tap = TugboatReplay.controller!.session!.events.lastWhere( - (event) => event.type == 'tap', - ); - final annotation = _semanticAnnotation(tap); - - expect(annotation['label'], { - 'kind': 'string', - 'value': 'Number of generations', - }); - expect(annotation['value'], {'kind': 'string', 'value': '3'}); - expect(annotation['selected'], isFalse); - }, - ); } diff --git a/packages/tugboat/lib/src/anchor_resolver.dart b/packages/tugboat/lib/src/anchor_resolver.dart index 58e0c42..915389b 100644 --- a/packages/tugboat/lib/src/anchor_resolver.dart +++ b/packages/tugboat/lib/src/anchor_resolver.dart @@ -54,38 +54,12 @@ class _VisitAcc { final bool hasTokenizedActionableDescendant; } -/// Metadata sampled from one interaction target. -/// -/// The resolver retains the concrete hit element privately so a post-callback -/// sample can stay bound to the original target instead of re-hit-testing a -/// coordinate that may now belong to another route or overlay. -class TugboatInteractionMetadata { - const TugboatInteractionMetadata._({ - required Element? element, - this.controlValue, - this.semanticAnnotation, - }) : _element = element; - - final Element? _element; - final TugboatControlValue? controlValue; - final TugboatSemanticAnnotation? semanticAnnotation; - - Object? get resampleTargetIdentity => _element; - - TugboatInteractionMetadata detached() => TugboatInteractionMetadata._( - element: null, - controlValue: controlValue, - semanticAnnotation: semanticAnnotation, - ); -} - /// Builds target anchors from hit-test results. class AnchorResolver { AnchorResolver({required this.rootKey, this.widgetNames = const {}}); final GlobalKey rootKey; final Map widgetNames; - List _controlValueHashKey = _newControlValueHashKey(); _TokenMap? _cachedTokenMap; int? _cachedFrameId; @@ -95,10 +69,6 @@ class AnchorResolver { int _frameEpoch = 0; bool _frameCallbackScheduled = false; - void rotateControlValueHashKey() { - _controlValueHashKey = _newControlValueHashKey(); - } - void invalidateTokenMapCache() { _cachedTokenMap = null; _cachedFrameId = null; @@ -199,240 +169,8 @@ class AnchorResolver { ); } - /// Samples control and semantic metadata with one hit test and semantics - /// session. - /// - /// The returned sample can be passed to [resampleInteractionMetadata] after - /// the host callback runs to read updated state from the same target. - TugboatInteractionMetadata? interactionMetadataAt(Offset globalPosition) { - final rootContext = rootKey.currentContext; - final rootRender = rootContext?.findRenderObject(); - if (rootRender is! RenderBox || rootContext is! Element) return null; - - final tokenMap = _tokenMapFor(rootContext, rootRender); - if (tokenMap == null) return null; - - return _withControlValueHashKey( - _controlValueHashKey, - () => _withSemanticsEnabled(rootRender, () { - final result = BoxHitTestResult(); - final localPosition = rootRender.globalToLocal(globalPosition); - rootRender.hitTest(result, position: localPosition); - return _interactionMetadataFromHitTest( - globalPosition: globalPosition, - result: result, - tokenMap: tokenMap, - rootContext: rootContext, - rootRender: rootRender, - ); - }), - ); - } - - TugboatInteractionMetadata _interactionMetadataFromHitTest({ - required Offset globalPosition, - required BoxHitTestResult result, - required _TokenMap tokenMap, - required Element rootContext, - required RenderBox rootRender, - }) { - Element? sampledElement; - TugboatControlValue? controlValue; - TugboatSemanticAnnotation? semanticAnnotation; - for (final entry in result.path) { - if (entry.target is! RenderObject) continue; - final element = tokenMap.renderElements[entry.target as RenderObject]; - if (element == null || tugboatIsCaptureChrome(element.widget)) continue; - final nextControl = controlValue == null - ? tugboatControlValueForElement(element) - : null; - final nextSemantic = semanticAnnotation == null - ? tugboatSemanticAnnotationForElement(element) - : null; - if (nextControl != null || nextSemantic != null) { - sampledElement ??= element; - controlValue ??= nextControl; - semanticAnnotation ??= nextSemantic; - } - if (controlValue != null && semanticAnnotation != null) break; - } - - final localSemanticPair = - semanticAnnotation?.label != null && semanticAnnotation?.value != null; - if (controlValue == null || !localSemanticPair) { - // An overlay can sit outside this capture boundary while the global - // semantics tree still contains the actual control at the tap point. - // Only inspect that tree when local hit-test metadata is incomplete: - // flushing and walking it is comparatively expensive for every tap. - final hits = _semanticsNodesAt( - globalPosition: globalPosition, - rootContext: rootContext, - rootRender: rootRender, - ); - final semanticFromHits = _semanticAnnotationFromHits(hits); - final semanticPair = - semanticFromHits?.label != null && semanticFromHits?.value != null; - if (semanticAnnotation == null || (!localSemanticPair && semanticPair)) { - semanticAnnotation = semanticFromHits ?? semanticAnnotation; - } - - final controlFromHits = _controlValueFromSemanticsHits(hits); - if (controlValue == null || - (!localSemanticPair && - semanticPair && - controlValue.sources.contains('semantics'))) { - controlValue = controlFromHits ?? controlValue; - } - } - - return TugboatInteractionMetadata._( - element: sampledElement, - controlValue: controlValue, - semanticAnnotation: semanticAnnotation, - ); - } - - /// Re-samples state from the exact element captured by - /// [interactionMetadataAt]. - TugboatInteractionMetadata? resampleInteractionMetadata( - TugboatInteractionMetadata sample, - ) { - final element = sample._element; - if (element == null || !element.mounted) return null; - final rootContext = rootKey.currentContext; - final rootRender = rootContext?.findRenderObject(); - - TugboatInteractionMetadata readElement() => TugboatInteractionMetadata._( - element: null, - controlValue: tugboatControlValueForElement(element), - semanticAnnotation: tugboatSemanticAnnotationForElement(element), - ); - - return _withControlValueHashKey( - _controlValueHashKey, - () => rootRender is RenderBox - ? _withSemanticsEnabled(rootRender, readElement) - : readElement(), - ); - } - - /// Semantic annotation for an [element] already in the tree. - TugboatSemanticAnnotation? semanticAnnotationForElement(Element element) { - final rootContext = rootKey.currentContext; - final rootRender = rootContext?.findRenderObject(); - if (rootRender is! RenderBox) { - return tugboatSemanticAnnotationForElement(element); - } - return _withControlValueHashKey( - _controlValueHashKey, - () => _withSemanticsEnabled( - rootRender, - () => tugboatSemanticAnnotationForElement(element), - ), - ); - } - - T _withSemanticsEnabled(RenderBox rootRender, T Function() body) { - final pipelineOwner = - rootRender.owner ?? RendererBinding.instance.rootPipelineOwner; - final semanticsAlreadyEnabled = - pipelineOwner.semanticsOwner != null || - RendererBinding.instance.rootPipelineOwner.semanticsOwner != null; - final semanticsHandle = semanticsAlreadyEnabled - ? null - : SemanticsBinding.instance.ensureSemantics(); - try { - if (!semanticsAlreadyEnabled) { - pipelineOwner.flushSemantics(); - } - return body(); - } finally { - semanticsHandle?.dispose(); - } - } - - TugboatSemanticAnnotation? _semanticAnnotationFromHits( - List hits, - ) { - TugboatSemanticAnnotation? merged; - // hits are root→leaf; reverse so deeper nodes win, ancestors fill gaps. - for (final node in hits.reversed) { - final next = tugboatSemanticAnnotationFromNode(node); - if (next == null) continue; - merged = merged == null - ? next - : tugboatMergeSemanticAnnotations(merged, next); - } - return merged; - } - - TugboatControlValue? _controlValueFromSemanticsHits( - List hits, - ) { - TugboatControlValue? fallback; - for (final node in hits.reversed) { - final value = tugboatControlValueFromSemanticsNode(node); - if (value == null) continue; - fallback ??= value; - final annotation = tugboatSemanticAnnotationFromNode(node); - if (annotation?.label != null && annotation?.value != null) { - return value; - } - } - return fallback; - } - - List _semanticsNodesAt({ - required Offset globalPosition, - required Element rootContext, - required RenderBox rootRender, - }) { - final pipelineOwner = - rootRender.owner ?? RendererBinding.instance.rootPipelineOwner; - final semanticsOwner = - pipelineOwner.semanticsOwner ?? - RendererBinding.instance.rootPipelineOwner.semanticsOwner; - if (semanticsOwner == null) return const []; - pipelineOwner.flushSemantics(); - final rootNode = semanticsOwner.rootSemanticsNode; - if (rootNode == null) return const []; - - final devicePixelRatio = View.maybeOf(rootContext)?.devicePixelRatio ?? 1.0; - final physical = globalPosition * devicePixelRatio; - final hits = []; - _collectSemanticsHits(rootNode, physical, hits, Matrix4.identity()); - return hits; - } - - void _collectSemanticsHits( - SemanticsNode node, - Offset physicalGlobal, - List hits, - Matrix4 transformToRoot, - ) { - final transform = node.transform; - final nextTransform = transform == null - ? transformToRoot - : (transformToRoot.clone()..multiply(transform)); - final inverted = Matrix4.tryInvert(nextTransform); - if (inverted != null) { - final local = MatrixUtils.transformPoint(inverted, physicalGlobal); - if (node.rect.contains(local)) { - hits.add(node); - } - } - node.visitChildren((child) { - _collectSemanticsHits(child, physicalGlobal, hits, nextTransform); - return true; - }); - } - /// Builds inventory and resolves a tap target from one token-map walk. - ({ - TugboatSceneInventory? inventory, - TugboatTargetAnchor? target, - TugboatInteractionMetadata? metadata, - }) + ({TugboatSceneInventory? inventory, TugboatTargetAnchor? target}) buildTapContext({ required Offset tapPosition, required String? route, @@ -442,31 +180,11 @@ class AnchorResolver { final rootContext = rootKey.currentContext; final rootRender = rootContext?.findRenderObject(); if (rootRender is! RenderBox || rootContext is! Element) { - return (inventory: null, target: null, metadata: null); + return (inventory: null, target: null); } final tokenMap = _tokenMapFor(rootContext, rootRender); - if (tokenMap == null) { - return (inventory: null, target: null, metadata: null); - } - final hitTest = BoxHitTestResult(); - rootRender.hitTest( - hitTest, - position: rootRender.globalToLocal(tapPosition), - ); - final metadata = _withControlValueHashKey( - _controlValueHashKey, - () => _withSemanticsEnabled( - rootRender, - () => _interactionMetadataFromHitTest( - globalPosition: tapPosition, - result: hitTest, - tokenMap: tokenMap, - rootContext: rootContext, - rootRender: rootRender, - ), - ), - ); + if (tokenMap == null) return (inventory: null, target: null); final stateAnchor = _stateAnchorFromTokenMap( tokenMap: tokenMap, route: route, @@ -474,7 +192,7 @@ class AnchorResolver { modalOpen: modalOpen, ); if (stateAnchor.signature.isEmpty) { - return (inventory: null, target: null, metadata: metadata); + return (inventory: null, target: null); } var target = _targetAtWithTokenMap( @@ -482,7 +200,6 @@ class AnchorResolver { route: route, tokenMap: tokenMap, rootRender: rootRender, - hitTest: hitTest, ); var inventory = _buildSceneInventoryFromTokenMap( tokenMap: tokenMap, @@ -505,7 +222,7 @@ class AnchorResolver { tokenMap: tokenMap, rootRender: rootRender, ); - return (inventory: inventory, target: target, metadata: metadata); + return (inventory: inventory, target: target); } /// Resolves a [TugboatTargetAnchor] for the [Scrollable] element that emitted @@ -552,14 +269,11 @@ class AnchorResolver { required String? route, required _TokenMap tokenMap, required RenderBox rootRender, - BoxHitTestResult? hitTest, }) { final viewport = rootRender.size; - final result = hitTest ?? BoxHitTestResult(); - if (hitTest == null) { - final localPosition = rootRender.globalToLocal(globalPosition); - rootRender.hitTest(result, position: localPosition); - } + final result = BoxHitTestResult(); + final localPosition = rootRender.globalToLocal(globalPosition); + rootRender.hitTest(result, position: localPosition); TugboatTargetAnchor? roleOnly; TugboatTargetAnchor? fallback; diff --git a/packages/tugboat/lib/src/anchors.dart b/packages/tugboat/lib/src/anchors.dart index e4109d8..f4051c0 100644 --- a/packages/tugboat/lib/src/anchors.dart +++ b/packages/tugboat/lib/src/anchors.dart @@ -1,6 +1,4 @@ -import 'dart:async'; import 'dart:convert'; -import 'dart:math'; import 'package:crypto/crypto.dart'; import 'package:flutter/cupertino.dart'; @@ -14,7 +12,6 @@ import 'semantics_flags_compat.dart'; part 'anchor_fingerprint.dart'; part 'anchor_models.dart'; part 'widget_roles.dart'; -part 'control_value.dart'; part 'anchor_resolver.dart'; part 'anchor_scene_inventory.dart'; part 'anchor_viewport_semantics.dart'; diff --git a/packages/tugboat/lib/src/control_value.dart b/packages/tugboat/lib/src/control_value.dart deleted file mode 100644 index 0bc4a65..0000000 --- a/packages/tugboat/lib/src/control_value.dart +++ /dev/null @@ -1,1018 +0,0 @@ -part of 'anchors.dart'; - -/// Schema version for raw control value payloads. -const int tugboatControlValueSchemaVersion = 4; - -/// Schema version for `tap_settled.controlValueTransition`. -const int tugboatControlValueTransitionSchemaVersion = 1; - -/// Schema version for per-interaction semantic annotations. -const int tugboatSemanticAnnotationSchemaVersion = 2; - -final RegExp _developerTokenPattern = RegExp(r'^[A-Za-z0-9_./:-]{1,64}$'); -const String _developerTokenPrefix = 'tugboat:'; -// AnchorResolver owns a per-controller key and still uses this zone to isolate -// its capture work. Raw control values no longer depend on the key. -const Symbol _controlValueHashKeyZoneKey = #tugboatControlValueHashKey; - -List _newControlValueHashKey() { - final random = Random.secure(); - return List.generate(32, (_) => random.nextInt(256), growable: false); -} - -T _withControlValueHashKey(List key, T Function() body) { - return runZoned(body, zoneValues: {_controlValueHashKeyZoneKey: key}); -} - -/// Encodes a single control scalar for analytics payloads. -class TugboatEncodedControlScalar { - const TugboatEncodedControlScalar._({required this.kind, this.value}); - - /// `null`, `bool`, `number`, `string`, `enum`, or a typed explicit value. - final String kind; - - /// Raw scalar value. - final Object? value; - - factory TugboatEncodedControlScalar.encode(Object? raw) { - if (raw == null) { - return const TugboatEncodedControlScalar._(kind: 'null'); - } - if (raw is bool) { - return TugboatEncodedControlScalar._(kind: 'bool', value: raw); - } - if (raw is num && raw.isFinite) { - return TugboatEncodedControlScalar._(kind: 'number', value: raw); - } - if (raw is num) { - return TugboatEncodedControlScalar._( - kind: 'string', - value: raw.toString(), - ); - } - if (raw is Enum) { - return TugboatEncodedControlScalar._( - kind: 'enum', - value: '${raw.runtimeType}.${raw.name}', - ); - } - if (raw is String) { - return TugboatEncodedControlScalar._(kind: 'string', value: raw); - } - return TugboatEncodedControlScalar._(kind: 'string', value: raw.toString()); - } - - /// Retains developer identifiers without the `tugboat:` namespace prefix. - static TugboatEncodedControlScalar encodeDeveloperToken(String raw) { - if (raw.startsWith(_developerTokenPrefix)) { - final token = raw.substring(_developerTokenPrefix.length); - if (_developerTokenPattern.hasMatch(token)) { - return TugboatEncodedControlScalar._(kind: 'enum', value: token); - } - } - return TugboatEncodedControlScalar.encode(raw); - } - - Map toJson() => { - 'kind': kind, - if (kind != 'null') 'value': value, - }; - - @override - bool operator ==(Object other) => - other is TugboatEncodedControlScalar && - kind == other.kind && - value == other.value; - - @override - int get hashCode => Object.hash(kind, value); -} - -/// A developer-declared typed control value. -class TugboatVisibleControlValue { - const TugboatVisibleControlValue._(this._encoded); - - final TugboatEncodedControlScalar _encoded; - - /// A finite numeric value, such as a slider position or percentage. - factory TugboatVisibleControlValue.number(num value) { - if (!value.isFinite) { - throw ArgumentError.value(value, 'value', 'must be finite'); - } - return TugboatVisibleControlValue._( - TugboatEncodedControlScalar._(kind: 'number', value: value), - ); - } - - /// A boolean control state. - TugboatVisibleControlValue.boolean(bool value) - : _encoded = TugboatEncodedControlScalar._(kind: 'bool', value: value); - - /// A duration represented as an exact non-negative number of milliseconds. - factory TugboatVisibleControlValue.duration(Duration value) { - if (value.isNegative) { - throw ArgumentError.value(value, 'value', 'must not be negative'); - } - return TugboatVisibleControlValue._( - TugboatEncodedControlScalar._( - kind: 'duration_ms', - value: value.inMilliseconds, - ), - ); - } - - /// A stable, developer-authored enum or template identifier. - factory TugboatVisibleControlValue.enumId(String value) { - final trimmed = value.trim(); - if (!_developerTokenPattern.hasMatch(trimmed)) { - throw ArgumentError.value( - value, - 'value', - 'must be 1-64 ASCII identifier characters', - ); - } - return TugboatVisibleControlValue._( - TugboatEncodedControlScalar._(kind: 'enum', value: trimmed), - ); - } - - Map toJson() => _encoded.toJson(); -} - -/// Declares a typed analytics value for a custom interactive control. -/// -/// Wrap controls whose actual state is not available from a standard Flutter -/// widget. [controlKey] should be a stable developer-owned identifier, for -/// example `video_duration`, `text_curve`, or `template`. -class TugboatControlValueScope extends StatelessWidget { - const TugboatControlValueScope({ - super.key, - required this.controlKey, - required this.value, - required this.child, - this.role, - this.unit, - this.min, - this.max, - this.step, - }); - - final String controlKey; - final TugboatVisibleControlValue value; - final Widget child; - - /// Optional role override, such as `slider`, `dropdown`, or `chip`. - final String? role; - - /// Optional stable unit, such as `ratio`, `percent`, or `milliseconds`. - final String? unit; - - /// Optional inclusive lower bound for [value]. - final num? min; - - /// Optional inclusive upper bound for [value]. - final num? max; - - /// Optional increment for [value]. - final num? step; - - @override - Widget build(BuildContext context) => child; - - TugboatControlValue? _toControlValue({required String fallbackRole}) { - if (!_hasValidNumericMetadata()) return null; - final normalizedKey = controlKey.trim(); - if (!_developerTokenPattern.hasMatch(normalizedKey)) return null; - final normalizedRole = role?.trim(); - final normalizedUnit = unit?.trim(); - return TugboatControlValue( - role: normalizedRole != null && normalizedRole.isNotEmpty - ? normalizedRole - : fallbackRole, - sources: const ['developer'], - controlKey: normalizedKey, - unit: - normalizedUnit != null && - _developerTokenPattern.hasMatch(normalizedUnit) - ? normalizedUnit - : null, - value: value._encoded, - min: _finiteNumber(min), - max: _finiteNumber(max), - step: _finiteNumber(step), - ); - } - - TugboatEncodedControlScalar? _finiteNumber(num? value) { - if (value == null || !value.isFinite) return null; - return TugboatEncodedControlScalar.encode(value); - } - - bool _hasValidNumericMetadata() { - if ((min != null && !min!.isFinite) || - (max != null && !max!.isFinite) || - (step != null && !step!.isFinite) || - (min != null && max != null && min! > max!) || - (step != null && step! <= 0)) { - return false; - } - if (min == null && max == null && step == null) return true; - final encoded = value._encoded; - if (encoded.kind != 'number' && encoded.kind != 'duration_ms') return false; - final numericValue = encoded.value as num; - return (min == null || numericValue >= min!) && - (max == null || numericValue <= max!); - } -} - -/// Semantic annotation for any interaction target. -/// -/// Attached to taps, settles, swipes, and scrolls whenever Flutter semantics -/// expose an identifier, label, value, or selection flag under the target. -class TugboatSemanticAnnotation { - const TugboatSemanticAnnotation({ - this.role, - this.identifier, - this.label, - this.value, - this.selected, - this.checked, - this.toggled, - this.schemaVersion = tugboatSemanticAnnotationSchemaVersion, - }); - - final int schemaVersion; - final String? role; - - /// Developer-authored semantics identifier when set. - final TugboatEncodedControlScalar? identifier; - - /// Raw semantics label when present. - final TugboatEncodedControlScalar? label; - - /// Raw semantics value when present. - final TugboatEncodedControlScalar? value; - - final bool? selected; - final bool? checked; - final bool? toggled; - - bool get hasPayload => - (role != null && role!.isNotEmpty) || - identifier != null || - label != null || - value != null || - selected != null || - checked != null || - toggled != null; - - Map toJson() => { - 'schemaVersion': schemaVersion, - if (role != null && role!.isNotEmpty) 'role': role, - if (identifier != null) 'identifier': identifier!.toJson(), - if (label != null) 'label': label!.toJson(), - if (value != null) 'value': value!.toJson(), - if (selected != null) 'selected': selected, - if (checked != null) 'checked': checked, - if (toggled != null) 'toggled': toggled, - }; - - @override - bool operator ==(Object other) => - other is TugboatSemanticAnnotation && - schemaVersion == other.schemaVersion && - role == other.role && - identifier == other.identifier && - label == other.label && - value == other.value && - selected == other.selected && - checked == other.checked && - toggled == other.toggled; - - @override - int get hashCode => Object.hash( - schemaVersion, - role, - identifier, - label, - value, - selected, - checked, - toggled, - ); -} - -/// Builds a semantic annotation from explicit [SemanticsProperties]. -TugboatSemanticAnnotation? tugboatSemanticAnnotationFromProperties( - SemanticsProperties properties, { - String? roleHint, -}) { - final identifierText = properties.identifier; - final labelText = properties.label; - final valueText = properties.value; - final selected = properties.selected; - final checked = properties.checked; - final toggled = properties.toggled; - - final identifier = - (identifierText != null && identifierText.trim().isNotEmpty) - ? TugboatEncodedControlScalar.encodeDeveloperToken(identifierText) - : null; - final label = (labelText != null && labelText.trim().isNotEmpty) - ? TugboatEncodedControlScalar.encode(labelText) - : null; - final value = (valueText != null && valueText.trim().isNotEmpty) - ? TugboatEncodedControlScalar.encode(valueText) - : null; - - final role = - roleHint ?? - (properties.slider == true - ? 'slider' - : properties.button == true - ? 'button' - : properties.link == true - ? 'link' - : properties.textField == true - ? 'textField' - : properties.header == true - ? 'header' - : checked != null - ? 'checkbox' - : toggled != null - ? 'switch' - : null); - - final annotation = TugboatSemanticAnnotation( - role: role, - identifier: identifier, - label: label, - value: value, - selected: selected, - checked: checked, - toggled: toggled, - ); - return annotation.hasPayload ? annotation : null; -} - -/// Builds a semantic annotation from a live [SemanticsNode]. -TugboatSemanticAnnotation? tugboatSemanticAnnotationFromNode( - SemanticsNode node, { - String? roleHint, -}) { - final data = node.getSemanticsData(); - final flags = data.flagsCollection; - final checked = semanticsCheckedFromFlags(flags); - final toggled = semanticsToggledFromFlags(flags); - final selected = semanticsSelectedFromFlags(flags); - - final identifier = data.identifier.trim().isNotEmpty - ? TugboatEncodedControlScalar.encodeDeveloperToken(data.identifier) - : null; - final label = data.label.trim().isNotEmpty - ? TugboatEncodedControlScalar.encode(data.label) - : null; - final value = data.value.trim().isNotEmpty - ? TugboatEncodedControlScalar.encode(data.value) - : null; - - final role = - roleHint ?? - (flags.isButton - ? 'button' - : flags.isLink - ? 'link' - : flags.isTextField - ? 'textField' - : flags.isHeader - ? 'header' - : checked != null - ? 'checkbox' - : toggled != null - ? 'switch' - : data.role != SemanticsRole.none - ? data.role.name - : null); - - final annotation = TugboatSemanticAnnotation( - role: role, - identifier: identifier, - label: label, - value: value, - selected: selected, - checked: checked, - toggled: toggled, - ); - return annotation.hasPayload ? annotation : null; -} - -/// Merges two annotations, preferring [primary] fields and filling gaps. -/// -/// An ancestor that supplies both a label and a value semantically describes a -/// parameter/value pair. Keep that label as the parameter identity instead of -/// replacing it with the descendant button's visible value text. -TugboatSemanticAnnotation tugboatMergeSemanticAnnotations( - TugboatSemanticAnnotation primary, - TugboatSemanticAnnotation fallback, -) { - final fallbackDescribesParameter = - fallback.label != null && fallback.value != null; - return TugboatSemanticAnnotation( - role: (primary.role != null && primary.role!.isNotEmpty) - ? primary.role - : fallback.role, - identifier: primary.identifier ?? fallback.identifier, - label: fallbackDescribesParameter - ? fallback.label - : primary.label ?? fallback.label, - value: primary.value ?? fallback.value, - selected: primary.selected ?? fallback.selected, - checked: primary.checked ?? fallback.checked, - toggled: primary.toggled ?? fallback.toggled, - ); -} - -/// Walks [hitElement] and ancestors, merging semantic fields. -/// -/// Child/deeper nodes win for concrete fields; ancestors fill gaps so a -/// Material button role can combine with a child Text label. -TugboatSemanticAnnotation? tugboatSemanticAnnotationForElement( - Element hitElement, -) { - TugboatSemanticAnnotation? merged; - - void consider(Element element) { - TugboatSemanticAnnotation? next; - if (element.widget is Semantics) { - next = tugboatSemanticAnnotationFromProperties( - (element.widget as Semantics).properties, - ); - } - next ??= () { - final node = element.renderObject?.debugSemantics; - return node == null ? null : tugboatSemanticAnnotationFromNode(node); - }(); - if (next == null) return; - merged = merged == null - ? next - : tugboatMergeSemanticAnnotations(merged!, next); - } - - consider(hitElement); - hitElement.visitAncestorElements((ancestor) { - consider(ancestor); - return true; - }); - return merged; -} - -/// Privacy-safe snapshot of an interactive control's value at sample time. -/// -/// Prefer typed widget state for standard Material/Cupertino controls. When -/// the hit target exposes Flutter semantics, [semanticValue] / [semanticLabel] -/// are attached as well so custom rows (e.g. GestureDetector lists) can still -/// report developer-authored semantic values. -/// -/// Bools, finite numbers, enums, and strings are retained as raw values. -class TugboatControlValue { - const TugboatControlValue({ - required this.role, - this.widgetType, - this.sources = const ['widget'], - this.controlKey, - this.unit, - this.min, - this.max, - this.step, - this.value, - this.groupValue, - this.selected, - this.index, - this.start, - this.end, - this.semanticValue, - this.semanticLabel, - this.schemaVersion = tugboatControlValueSchemaVersion, - }); - - final int schemaVersion; - - /// Control role (`checkbox`, `switch`, `radio`, `slider`, `dropdown`, - /// `dropdownItem`, `menuItem`, `chip`, `button`, `semantic`, …). - final String role; - final String? widgetType; - - /// Provenance markers such as `widget` and/or `semantics`. - final List sources; - - /// Stable developer-owned key for an explicitly visible custom value. - final String? controlKey; - - /// Optional unit for [value], such as `ratio` or `milliseconds`. - final String? unit; - - /// Optional inclusive lower bound for an explicitly declared value. - final TugboatEncodedControlScalar? min; - - /// Optional inclusive upper bound for an explicitly declared value. - final TugboatEncodedControlScalar? max; - - /// Optional increment for an explicitly declared value. - final TugboatEncodedControlScalar? step; - - /// Primary sampled value (option identity, toggle state, slider position, - /// or best-effort semantic value when no typed widget value exists). - final TugboatEncodedControlScalar? value; - - /// Current group selection for radio controls. - final TugboatEncodedControlScalar? groupValue; - - /// Whether this option is the active selection (radios/chips/semantics). - final bool? selected; - - /// Zero-based index among sibling options when known. - final int? index; - - /// Range slider start (inclusive). - final TugboatEncodedControlScalar? start; - - /// Range slider end (inclusive). - final TugboatEncodedControlScalar? end; - - /// Encoded [SemanticsData.value] / [SemanticsProperties.value] when present. - final TugboatEncodedControlScalar? semanticValue; - - /// Encoded [SemanticsData.label] / [SemanticsProperties.label] when present. - final TugboatEncodedControlScalar? semanticLabel; - - bool get hasPayload => - value != null || - groupValue != null || - selected != null || - start != null || - end != null || - semanticValue != null || - semanticLabel != null; - - TugboatControlValue copyWith({ - String? role, - String? widgetType, - List? sources, - String? controlKey, - String? unit, - TugboatEncodedControlScalar? min, - TugboatEncodedControlScalar? max, - TugboatEncodedControlScalar? step, - TugboatEncodedControlScalar? value, - TugboatEncodedControlScalar? groupValue, - bool? selected, - int? index, - TugboatEncodedControlScalar? start, - TugboatEncodedControlScalar? end, - TugboatEncodedControlScalar? semanticValue, - TugboatEncodedControlScalar? semanticLabel, - }) { - return TugboatControlValue( - schemaVersion: schemaVersion, - role: role ?? this.role, - widgetType: widgetType ?? this.widgetType, - sources: sources ?? this.sources, - controlKey: controlKey ?? this.controlKey, - unit: unit ?? this.unit, - min: min ?? this.min, - max: max ?? this.max, - step: step ?? this.step, - value: value ?? this.value, - groupValue: groupValue ?? this.groupValue, - selected: selected ?? this.selected, - index: index ?? this.index, - start: start ?? this.start, - end: end ?? this.end, - semanticValue: semanticValue ?? this.semanticValue, - semanticLabel: semanticLabel ?? this.semanticLabel, - ); - } - - Map toJson() => { - 'schemaVersion': schemaVersion, - 'role': role, - if (widgetType != null && widgetType!.isNotEmpty) 'widgetType': widgetType, - if (sources.isNotEmpty) 'sources': sources, - if (controlKey != null && controlKey!.isNotEmpty) 'controlKey': controlKey, - if (unit != null && unit!.isNotEmpty) 'unit': unit, - if (min != null) 'min': min!.toJson(), - if (max != null) 'max': max!.toJson(), - if (step != null) 'step': step!.toJson(), - if (value != null) 'value': value!.toJson(), - if (groupValue != null) 'groupValue': groupValue!.toJson(), - if (selected != null) 'selected': selected, - if (index != null) 'index': index, - if (start != null) 'start': start!.toJson(), - if (end != null) 'end': end!.toJson(), - if (semanticValue != null) 'semanticValue': semanticValue!.toJson(), - if (semanticLabel != null) 'semanticLabel': semanticLabel!.toJson(), - }; - - @override - bool operator ==(Object other) => - other is TugboatControlValue && - schemaVersion == other.schemaVersion && - role == other.role && - widgetType == other.widgetType && - _listEquals(sources, other.sources) && - controlKey == other.controlKey && - unit == other.unit && - min == other.min && - max == other.max && - step == other.step && - value == other.value && - groupValue == other.groupValue && - selected == other.selected && - index == other.index && - start == other.start && - end == other.end && - semanticValue == other.semanticValue && - semanticLabel == other.semanticLabel; - - @override - int get hashCode => Object.hash( - schemaVersion, - role, - widgetType, - Object.hashAll(sources), - controlKey, - unit, - min, - max, - step, - value, - groupValue, - selected, - index, - start, - end, - semanticValue, - semanticLabel, - ); -} - -/// Reads a control value from [widget], or null when unsupported. -TugboatControlValue? tugboatControlValueForWidget(Widget widget, {int? index}) { - final widgetType = widget.runtimeType.toString(); - - if (widget is Checkbox) { - return TugboatControlValue( - role: 'checkbox', - widgetType: widgetType, - value: TugboatEncodedControlScalar.encode(widget.value), - selected: widget.value == true, - index: index, - ); - } - if (widget is CheckboxListTile) { - return TugboatControlValue( - role: 'checkbox', - widgetType: widgetType, - value: TugboatEncodedControlScalar.encode(widget.value), - selected: widget.value == true, - index: index, - ); - } - if (widget is Switch) { - return TugboatControlValue( - role: 'switch', - widgetType: widgetType, - value: TugboatEncodedControlScalar.encode(widget.value), - selected: widget.value, - index: index, - ); - } - if (widget is CupertinoSwitch) { - return TugboatControlValue( - role: 'switch', - widgetType: widgetType, - value: TugboatEncodedControlScalar.encode(widget.value), - selected: widget.value, - index: index, - ); - } - if (widget is SwitchListTile) { - return TugboatControlValue( - role: 'switch', - widgetType: widgetType, - value: TugboatEncodedControlScalar.encode(widget.value), - selected: widget.value, - index: index, - ); - } - if (widget is Radio || widget is RadioListTile) { - // Typed Radio / RadioListTile cannot be read through a promoted - // Radio view; keep access dynamic like role detection. - final dynamic radio = widget; - final option = radio.value; - final group = radio.groupValue; - return TugboatControlValue( - role: 'radio', - widgetType: widgetType, - value: TugboatEncodedControlScalar.encode(option), - groupValue: TugboatEncodedControlScalar.encode(group), - selected: option == group, - index: index, - ); - } - if (widget is Slider) { - return TugboatControlValue( - role: 'slider', - widgetType: widgetType, - value: TugboatEncodedControlScalar.encode(widget.value), - index: index, - ); - } - if (widget is CupertinoSlider) { - return TugboatControlValue( - role: 'slider', - widgetType: widgetType, - value: TugboatEncodedControlScalar.encode(widget.value), - index: index, - ); - } - if (widget is RangeSlider) { - return TugboatControlValue( - role: 'slider', - widgetType: widgetType, - start: TugboatEncodedControlScalar.encode(widget.values.start), - end: TugboatEncodedControlScalar.encode(widget.values.end), - index: index, - ); - } - if (widget is DropdownButton) { - final dynamic dropdown = widget; - return TugboatControlValue( - role: 'dropdown', - widgetType: widgetType, - value: TugboatEncodedControlScalar.encode(dropdown.value), - index: index, - ); - } - if (widget is DropdownMenuItem) { - final dynamic item = widget; - return TugboatControlValue( - role: 'dropdownItem', - widgetType: widgetType, - value: TugboatEncodedControlScalar.encode(item.value), - index: index, - ); - } - if (widget is PopupMenuItem) { - final dynamic item = widget; - return TugboatControlValue( - role: 'menuItem', - widgetType: widgetType, - value: TugboatEncodedControlScalar.encode(item.value), - index: index, - ); - } - if (widget is FilterChip) { - return TugboatControlValue( - role: 'chip', - widgetType: widgetType, - value: TugboatEncodedControlScalar.encode(widget.selected), - selected: widget.selected, - index: index, - ); - } - if (widget is ChoiceChip) { - return TugboatControlValue( - role: 'chip', - widgetType: widgetType, - value: TugboatEncodedControlScalar.encode(widget.selected), - selected: widget.selected, - index: index, - ); - } - return null; -} - -/// Builds a control-value snapshot from explicit [SemanticsProperties]. -TugboatControlValue? tugboatControlValueFromSemanticsProperties( - SemanticsProperties properties, { - String? widgetType, - int? index, - String? roleHint, -}) { - final label = properties.label; - final valueText = properties.value; - final selected = properties.selected; - final checked = properties.checked; - final toggled = properties.toggled; - - final semanticValue = (valueText != null && valueText.trim().isNotEmpty) - ? TugboatEncodedControlScalar.encode(valueText) - : null; - final semanticLabel = (label != null && label.trim().isNotEmpty) - ? TugboatEncodedControlScalar.encode(label) - : null; - - if (semanticValue == null && - semanticLabel == null && - selected == null && - checked == null && - toggled == null) { - return null; - } - - final role = - roleHint ?? - (properties.slider == true - ? 'slider' - : properties.button == true - ? 'button' - : checked != null - ? 'checkbox' - : toggled != null - ? 'switch' - : 'semantic'); - - final semanticState = role == 'checkbox' || role == 'switch' - ? (checked ?? toggled ?? selected) - : null; - - return TugboatControlValue( - role: role, - widgetType: widgetType, - sources: const ['semantics'], - value: - semanticValue ?? - (semanticState != null - ? TugboatEncodedControlScalar.encode(semanticState) - : null), - selected: selected ?? checked ?? toggled, - index: index, - semanticValue: semanticValue, - semanticLabel: semanticLabel, - ); -} - -/// Builds a control-value snapshot from a live [SemanticsNode]. -TugboatControlValue? tugboatControlValueFromSemanticsNode( - SemanticsNode node, { - String? roleHint, -}) { - final data = node.getSemanticsData(); - final flags = data.flagsCollection; - final checked = semanticsCheckedFromFlags(flags); - final toggled = semanticsToggledFromFlags(flags); - final selected = semanticsSelectedFromFlags(flags); - - final semanticValue = data.value.trim().isNotEmpty - ? TugboatEncodedControlScalar.encode(data.value) - : null; - final semanticLabel = data.label.trim().isNotEmpty - ? TugboatEncodedControlScalar.encode(data.label) - : null; - - if (semanticValue == null && - semanticLabel == null && - checked == null && - toggled == null && - selected == null) { - return null; - } - - final role = - roleHint ?? - (flags.isButton - ? 'button' - : checked != null - ? 'checkbox' - : toggled != null - ? 'switch' - : data.role != SemanticsRole.none - ? data.role.name - : 'semantic'); - - final semanticState = role == 'checkbox' || role == 'switch' - ? (checked ?? toggled ?? selected) - : null; - - return TugboatControlValue( - role: role, - sources: const ['semantics'], - value: - semanticValue ?? - (semanticState != null - ? TugboatEncodedControlScalar.encode(semanticState) - : null), - selected: selected ?? checked ?? toggled, - semanticValue: semanticValue, - semanticLabel: semanticLabel, - ); -} - -/// Merges typed widget state with semantic annotations. -TugboatControlValue? tugboatMergeControlValues( - TugboatControlValue? widgetValue, - TugboatControlValue? semanticsValue, -) { - if (widgetValue == null) return semanticsValue; - if (semanticsValue == null) return widgetValue; - - final sources = { - ...widgetValue.sources, - ...semanticsValue.sources, - }.toList()..sort(); - - return widgetValue.copyWith( - sources: sources, - controlKey: widgetValue.controlKey ?? semanticsValue.controlKey, - unit: widgetValue.unit ?? semanticsValue.unit, - value: widgetValue.value ?? semanticsValue.value, - selected: widgetValue.selected ?? semanticsValue.selected, - semanticValue: semanticsValue.semanticValue ?? widgetValue.semanticValue, - semanticLabel: semanticsValue.semanticLabel ?? widgetValue.semanticLabel, - ); -} - -/// Walks [hitElement] and its ancestors for widget + semantic control values. -TugboatControlValue? tugboatControlValueForElement(Element hitElement) { - TugboatControlValue? widgetValue; - TugboatControlValue? semanticsValue; - TugboatControlValueScope? developerScope; - - void consider(Element element) { - developerScope ??= element.widget is TugboatControlValueScope - ? element.widget as TugboatControlValueScope - : null; - final index = widgetValue == null - ? _optionIndexAmongSiblings(element) - : null; - widgetValue ??= tugboatControlValueForWidget(element.widget, index: index); - // Explicit Semantics widgets are preferred over live nodes for labels. - if (semanticsValue == null && element.widget is Semantics) { - semanticsValue = tugboatControlValueFromSemanticsProperties( - (element.widget as Semantics).properties, - widgetType: element.widget.runtimeType.toString(), - ); - } - if (semanticsValue == null) { - final node = element.renderObject?.debugSemantics; - if (node != null) { - semanticsValue = tugboatControlValueFromSemanticsNode(node); - } - } - } - - consider(hitElement); - hitElement.visitAncestorElements((ancestor) { - consider(ancestor); - return true; - }); - - final merged = tugboatMergeControlValues(widgetValue, semanticsValue); - final developerValue = developerScope?._toControlValue( - fallbackRole: merged?.role ?? 'semantic', - ); - final value = tugboatMergeControlValues(developerValue, merged); - if (value == null || !value.hasPayload) return null; - return value; -} - -int? _optionIndexAmongSiblings(Element element) { - final self = tugboatControlValueForWidget(element.widget); - if (self == null) return null; - const optionRoles = {'radio', 'dropdownItem', 'menuItem', 'chip'}; - if (!optionRoles.contains(self.role)) return null; - - // Walk up until a parent exposes multiple same-role options among its - // descendants, then return this element's ordinal among those options. - Element? parent; - element.visitAncestorElements((ancestor) { - parent = ancestor; - return false; - }); - while (parent != null) { - final options = []; - void collect(Element node) { - final value = tugboatControlValueForWidget(node.widget); - if (value != null && value.role == self.role) { - options.add(node); - return; - } - node.visitChildElements(collect); - } - - parent!.visitChildElements(collect); - if (options.length > 1) { - final index = options.indexWhere((option) => identical(option, element)); - return index >= 0 ? index : null; - } - - Element? next; - parent!.visitAncestorElements((ancestor) { - next = ancestor; - return false; - }); - parent = next; - } - return null; -} diff --git a/packages/tugboat/lib/src/controller.dart b/packages/tugboat/lib/src/controller.dart index b386c6f..a3f244c 100644 --- a/packages/tugboat/lib/src/controller.dart +++ b/packages/tugboat/lib/src/controller.dart @@ -1,5 +1,4 @@ import 'dart:async'; -import 'dart:collection'; import 'dart:typed_data'; import 'package:flutter/semantics.dart'; @@ -29,20 +28,6 @@ export 'replay_config.dart' TugboatViewportSemanticPolicy, resolveViewportSemanticPolicy; -class _PostCallbackMetadataCapture { - TugboatInteractionMetadata? value; - bool _ambiguous = false; - - void markAmbiguous() { - _ambiguous = true; - value = null; - } - - void complete(TugboatInteractionMetadata? metadata) { - if (!_ambiguous) value = metadata; - } -} - class _ScrollTracker { _ScrollTracker({ required this.scrollableElement, @@ -58,7 +43,6 @@ class _ScrollTracker { required this.depth, required this.maxScrollExtent, this.pageStart, - this.semantic, }); final Element scrollableElement; @@ -74,7 +58,6 @@ class _ScrollTracker { final int depth; final double maxScrollExtent; final double? pageStart; - final TugboatSemanticAnnotation? semantic; int overscrollCount = 0; DateTime? lastSampleAt; } @@ -823,8 +806,6 @@ class TugboatReplayController extends ChangeNotifier { {}; String? _latestRouteCaptureKey; final Set<_TapSettleWork> _activeTapSettles = <_TapSettleWork>{}; - final Map _pendingPostCallbackMetadata = - HashMap.identity(); /// Most recently started route-capture work (any Navigator). _RouteCaptureWork? get _activeRouteCapture { @@ -1340,7 +1321,6 @@ class TugboatReplayController extends ChangeNotifier { _clock ..reset() ..start(); - _anchorResolver?.rotateControlValueHashKey(); _session = TugboatSession( id: 'session-${DateTime.now().microsecondsSinceEpoch}', startedAt: DateTime.now(), @@ -1362,7 +1342,6 @@ class TugboatReplayController extends ChangeNotifier { _latestFrameId = null; _clearReleasedInteractions(); _interactions.clearAll(); - _pendingPostCallbackMetadata.clear(); _scrollTrackers.clear(); _hashToFrameId.clear(); _frameProvenance.clear(); @@ -2287,9 +2266,6 @@ class TugboatReplayController extends ChangeNotifier { TugboatTargetAnchor? target; TugboatStateAnchor? tapState = _currentStateAnchor; TugboatSceneInventory? tapInventory; - TugboatInteractionMetadata? metadata; - TugboatControlValue? controlValue; - TugboatSemanticAnnotation? semantic; if (resolver != null && config.profile != TugboatCaptureProfile.dormant) { final tapContext = resolver.buildTapContext( @@ -2300,9 +2276,6 @@ class TugboatReplayController extends ChangeNotifier { ); target = tapContext.target; tapInventory = tapContext.inventory; - metadata = tapContext.metadata; - controlValue = metadata?.controlValue; - semantic = metadata?.semanticAnnotation; if (tapInventory != null) { _currentStateAnchor = tapInventory.stateAnchor; tapState = tapInventory.stateAnchor; @@ -2310,9 +2283,6 @@ class TugboatReplayController extends ChangeNotifier { } } else { target = resolver?.targetAt(position, route: _currentRoute); - metadata = resolver?.interactionMetadataAt(position); - controlValue = metadata?.controlValue; - semantic = metadata?.semanticAnnotation; } // Resolve after the tap context so a stale settled map can be refreshed @@ -2345,8 +2315,6 @@ class TugboatReplayController extends ChangeNotifier { }, if (viewportResolution != null) 'viewportSemanticResolution': viewportResolution.toJson(), - if (controlValue != null) 'controlValue': controlValue.toJson(), - if (semantic != null) 'semanticAnnotation': semantic.toJson(), }; final beforeState = tapState; @@ -2365,15 +2333,8 @@ class TugboatReplayController extends ChangeNotifier { startPosition: position, pointerGeneration: ++_pointerGeneration, captureSessionId: _session?.id, - controlValue: controlValue, - semantic: semantic, - ); - final tx = InteractionTransaction( - origin: origin, - pointerId: pointer, - metadata: metadata?.detached(), - resampleTarget: metadata, ); + final tx = InteractionTransaction(origin: origin, pointerId: pointer); final legacyStream = config.legacyGestureStream; tx.bufferedOutside = target == null ? TugboatEvent( @@ -2781,7 +2742,6 @@ class TugboatReplayController extends ChangeNotifier { if (!_acceptsPointerInput) return; final pending = _interactions.removePending(pointer); if (pending == null) return; - final resampleTarget = pending.takeResampleTarget(); if (pending.isSwipeOrScroll) { if (pending.claimed) { @@ -2804,13 +2764,6 @@ class TugboatReplayController extends ChangeNotifier { : null; final scrolled = scrollStartEventId != null; final tapWasEmitted = pending.tapEmitted; - final metadata = resampleTarget == null - ? null - : _anchorResolver?.resampleInteractionMetadata(resampleTarget); - final controlValue = - metadata?.controlValue ?? pending.metadata?.controlValue; - final semantic = - metadata?.semanticAnnotation ?? pending.metadata?.semanticAnnotation; pending.gesture = scrolled ? InteractionGesture.scroll : InteractionGesture.swipe; @@ -2818,8 +2771,6 @@ class TugboatReplayController extends ChangeNotifier { ? InteractionResultStatus.changed : InteractionResultStatus.unchanged; pending.resultObservedAtMs = atMs; - pending.resultControlValue = controlValue; - pending.resultSemanticAnnotation = semantic; if (scrollStartEventId != null) pending.addEvidence(scrollStartEventId); if (config.emitLegacyInteractionProjection) { _addEvent( @@ -2852,8 +2803,6 @@ class TugboatReplayController extends ChangeNotifier { if (tapWasEmitted) 'invalidatesRelatedTap': true, if (scrollStartEventId != null) 'scrollStartEventId': scrollStartEventId, - if (controlValue != null) 'controlValue': controlValue.toJson(), - if (semantic != null) 'semanticAnnotation': semantic.toJson(), 'interactionId': pending.id, }, ), @@ -2882,38 +2831,7 @@ class TugboatReplayController extends ChangeNotifier { final work = _TapSettleWork(session: _session); _activeTapSettles.add(work); - final postCallbackMetadata = _capturePostCallbackMetadata(resampleTarget); - unawaited( - _resolveTapSettle( - work, - pending, - position, - _activeRouteCapture, - postCallbackMetadata, - ), - ); - } - - _PostCallbackMetadataCapture _capturePostCallbackMetadata( - TugboatInteractionMetadata? before, - ) { - final capture = _PostCallbackMetadataCapture(); - final targetIdentity = before?.resampleTargetIdentity; - if (before == null || targetIdentity == null) return capture; - final overlapping = _pendingPostCallbackMetadata[targetIdentity]; - if (overlapping != null) { - overlapping.markAmbiguous(); - capture.markAmbiguous(); - } - _pendingPostCallbackMetadata[targetIdentity] = capture; - WidgetsBinding.instance.addPostFrameCallback((_) { - if (identical(_pendingPostCallbackMetadata[targetIdentity], capture)) { - _pendingPostCallbackMetadata.remove(targetIdentity); - } - if (_disposed) return; - capture.complete(_anchorResolver?.resampleInteractionMetadata(before)); - }); - return capture; + unawaited(_resolveTapSettle(work, pending, position, _activeRouteCapture)); } Future _resolveTapSettle( @@ -2921,7 +2839,6 @@ class TugboatReplayController extends ChangeNotifier { InteractionTransaction pending, Offset position, _RouteCaptureWork? routeCaptureAtPointerUp, - _PostCallbackMetadataCapture postCallbackMetadata, ) async { try { final initialRouteCapture = @@ -3065,17 +2982,6 @@ class TugboatReplayController extends ChangeNotifier { final visualChanged = visualAvailable ? beforeContentHash != afterContentHash : null; - // The first post-callback frame samples the pointer-down target. - // Publication can happen much later without borrowing another - // interaction's state. If no frame ran, omit the after value. - final afterMetadata = postCallbackMetadata.value; - final controlValueTransition = _controlValueTransitionPayload( - before: pending.metadata?.controlValue, - after: afterMetadata?.controlValue, - ); - final semanticAnnotation = - afterMetadata?.semanticAnnotation ?? - pending.metadata?.semanticAnnotation; if (config.emitLegacyInteractionProjection) { _addEvent( @@ -3136,10 +3042,6 @@ class TugboatReplayController extends ChangeNotifier { observation.captureFailure ?? observation.captureOutcome, }, - if (controlValueTransition != null) - 'controlValueTransition': controlValueTransition, - if (semanticAnnotation != null) - 'semanticAnnotation': semanticAnnotation.toJson(), }, ), ); @@ -3162,8 +3064,6 @@ class TugboatReplayController extends ChangeNotifier { pending.resultStateAnchor = afterState; pending.resultRoute = observation.route; pending.resultObservedAtMs = atMs; - pending.controlValueTransition = controlValueTransition; - pending.resultSemanticAnnotation = semanticAnnotation; if (observation.routeEventId != null) { pending.addEvidence(observation.routeEventId!); } @@ -3239,23 +3139,6 @@ class TugboatReplayController extends ChangeNotifier { _activeTapSettles.clear(); } - /// Builds a before/after control-value transition for `tap_settled`. - Map? _controlValueTransitionPayload({ - required TugboatControlValue? before, - required TugboatControlValue? after, - }) { - if (before == null && after == null) return null; - final role = after?.role ?? before!.role; - final widgetType = after?.widgetType ?? before?.widgetType; - return { - 'schemaVersion': tugboatControlValueTransitionSchemaVersion, - 'role': role, - if (widgetType != null && widgetType.isNotEmpty) 'widgetType': widgetType, - if (before != null) 'before': before.toJson(), - if (after != null) 'after': after.toJson(), - }; - } - TugboatInteractionResult _computeTapSettleResult({ required TugboatStateAnchor? beforeState, required TugboatStateAnchor? afterState, @@ -3394,9 +3277,6 @@ class TugboatReplayController extends ChangeNotifier { if (tracker.sectionLabel != null) { data['sectionLabel'] = tracker.sectionLabel; } - if (tracker.semantic != null) { - data['semanticAnnotation'] = tracker.semantic!.toJson(); - } if (overscrollCount != null && overscrollCount > 0) { data['overscrollCount'] = overscrollCount; } @@ -3446,9 +3326,6 @@ class TugboatReplayController extends ChangeNotifier { _refreshStateAnchor(); final targetAnchor = _resolveScrollableAnchor(scrollableElement); final sectionLabel = _sectionLabelFor(scrollableElement); - final semantic = _anchorResolver?.semanticAnnotationForElement( - scrollableElement, - ); final attachmentContext = _captureContext(TugboatFrameTrigger.scroll); final beforeFrame = _compatibleFrameFor(attachmentContext); final unavailableReason = _unavailableAttachmentReason(attachmentContext); @@ -3469,7 +3346,6 @@ class TugboatReplayController extends ChangeNotifier { depth: depth, maxScrollExtent: metrics.maxScrollExtent, pageStart: pageStart, - semantic: semantic, ); _scrollTrackers[scrollableElement] = tracker; diff --git a/packages/tugboat/lib/src/interaction_transaction.dart b/packages/tugboat/lib/src/interaction_transaction.dart index 14f77bf..34096a7 100644 --- a/packages/tugboat/lib/src/interaction_transaction.dart +++ b/packages/tugboat/lib/src/interaction_transaction.dart @@ -29,8 +29,6 @@ class InteractionOrigin { required this.startPosition, required this.pointerGeneration, required this.captureSessionId, - this.controlValue, - this.semantic, }); final String interactionId; @@ -45,8 +43,6 @@ class InteractionOrigin { final Offset startPosition; final int pointerGeneration; final String? captureSessionId; - final TugboatControlValue? controlValue; - final TugboatSemanticAnnotation? semantic; Map toJson() => { 'interactionId': interactionId, @@ -61,8 +57,6 @@ class InteractionOrigin { 'startPosition': {'x': startPosition.dx, 'y': startPosition.dy}, 'pointerGeneration': pointerGeneration, if (captureSessionId != null) 'captureSessionId': captureSessionId, - if (controlValue != null) 'controlValue': controlValue!.toJson(), - if (semantic != null) 'semanticAnnotation': semantic!.toJson(), }; } @@ -135,17 +129,10 @@ enum InteractionRejectionReason { /// Bounded in-memory transaction for one pointer gesture. class InteractionTransaction { - InteractionTransaction({ - required this.origin, - required this.pointerId, - this.metadata, - TugboatInteractionMetadata? resampleTarget, - }) : _resampleTarget = resampleTarget; + InteractionTransaction({required this.origin, required this.pointerId}); final InteractionOrigin origin; final int pointerId; - final TugboatInteractionMetadata? metadata; - TugboatInteractionMetadata? _resampleTarget; InteractionGesture gesture = InteractionGesture.tap; bool claimed = false; @@ -171,9 +158,6 @@ class InteractionTransaction { String? afterFrame; int? resultObservedAtMs; TugboatStateAnchor? resultStateAnchor; - TugboatControlValue? resultControlValue; - Map? controlValueTransition; - TugboatSemanticAnnotation? resultSemanticAnnotation; Completer? _successorSignal; @@ -213,12 +197,6 @@ class InteractionTransaction { gesture = InteractionGesture.swipe; } - TugboatInteractionMetadata? takeResampleTarget() { - final target = _resampleTarget; - _resampleTarget = null; - return target; - } - Map resultToJson() => { 'status': (resultStatus ?? InteractionResultStatus.unknown).name, if (resultRoute != null) 'route': resultRoute, @@ -226,12 +204,6 @@ class InteractionTransaction { if (resultStateAnchor != null) 'stateAnchor': resultStateAnchor!.toJson(), if (afterFrame != null) 'afterFrame': afterFrame, if (resultObservedAtMs != null) 'observedAtMs': resultObservedAtMs, - if (resultControlValue != null) - 'controlValue': resultControlValue!.toJson(), - if (controlValueTransition != null) - 'controlValueTransition': controlValueTransition, - if (resultSemanticAnnotation != null) - 'semanticAnnotation': resultSemanticAnnotation!.toJson(), }; Map attributionToJson({int? windowMs}) => { diff --git a/packages/tugboat/lib/src/models.dart b/packages/tugboat/lib/src/models.dart index 4fd8854..e0560c3 100644 --- a/packages/tugboat/lib/src/models.dart +++ b/packages/tugboat/lib/src/models.dart @@ -6,8 +6,10 @@ import 'package:flutter/widgets.dart'; import 'anchors.dart'; import 'collector_config.dart'; -/// Current session JSON schema. Writers emit this; readers accept 6–8. -const int tugboatSessionSchemaVersion = 8; +/// Current session JSON schema. Writers emit this; readers accept 6–9. +/// +/// Schema 9 stops emitting value and semantic-annotation event data. +const int tugboatSessionSchemaVersion = 9; /// Event selection channel for enrichment / insight / replay consumers. enum TugboatEventStream { diff --git a/packages/tugboat/lib/src/sdk_version.dart b/packages/tugboat/lib/src/sdk_version.dart index 9f79b32..3544f9c 100644 --- a/packages/tugboat/lib/src/sdk_version.dart +++ b/packages/tugboat/lib/src/sdk_version.dart @@ -1,3 +1,3 @@ // Keep this in sync with packages/tugboat/pubspec.yaml. The SDK version test // reads pubspec.yaml directly so release bumps fail fast if this drifts. -const tugboatSdkVersion = '0.4.17'; +const tugboatSdkVersion = '0.5.0'; diff --git a/packages/tugboat/lib/src/semantics_flags_compat.dart b/packages/tugboat/lib/src/semantics_flags_compat.dart index 026d8ca..6365205 100644 --- a/packages/tugboat/lib/src/semantics_flags_compat.dart +++ b/packages/tugboat/lib/src/semantics_flags_compat.dart @@ -14,38 +14,3 @@ bool? semanticsEnabledFromFlags(SemanticsFlags flags) { } return enabled.toBoolOrNull() as bool?; } - -/// Reads checked state across Flutter SDK versions. -bool? semanticsCheckedFromFlags(SemanticsFlags flags) { - final dynamic state = flags; - final checked = state.isChecked; - if (checked is bool) { - if (state.hasCheckedState == true) return checked; - return null; - } - // Flutter 3.36+: CheckedState enum (none / isTrue / isFalse / mixed). - final name = checked.toString().split('.').last; - return switch (name) { - 'isTrue' => true, - 'isFalse' => false, - 'none' || 'mixed' => null, - _ => null, - }; -} - -/// Reads toggled state across Flutter SDK versions. -bool? semanticsToggledFromFlags(SemanticsFlags flags) { - final dynamic state = flags; - final toggled = state.isToggled; - if (toggled is bool) return toggled; - return toggled.toBoolOrNull() as bool?; -} - -/// Reads selected state across Flutter SDK versions. -bool? semanticsSelectedFromFlags(SemanticsFlags flags) { - final dynamic state = flags; - final selected = state.isSelected; - if (selected is bool) return selected; - // Flutter 3.36+: Tristate. - return selected.toBoolOrNull() as bool?; -} diff --git a/packages/tugboat/lib/src/widget_roles.dart b/packages/tugboat/lib/src/widget_roles.dart index 38aac71..c08eb09 100644 --- a/packages/tugboat/lib/src/widget_roles.dart +++ b/packages/tugboat/lib/src/widget_roles.dart @@ -188,8 +188,10 @@ WidgetRole? tugboatRoleForWidget(Widget widget) { ); } if (widget is Radio || widget is RadioListTile) { - // Same generic-callback cast hazard as DropdownButton: reading onChanged - // through RadioListTile / Radio can throw at runtime. + // Reading a generic Radio or RadioListTile callback through the promoted + // `` view can cast a typed callback to `void Function(dynamic)`, + // which is not a valid runtime cast. Keep this inspection dynamic so role + // detection works for typed radio controls. final enabled = (widget as dynamic).onChanged != null; return WidgetRole( 'radio', diff --git a/packages/tugboat/lib/tugboat.dart b/packages/tugboat/lib/tugboat.dart index bfd7708..8ed8f3e 100644 --- a/packages/tugboat/lib/tugboat.dart +++ b/packages/tugboat/lib/tugboat.dart @@ -5,21 +5,6 @@ export 'src/anchors.dart' TugboatNormalizedBounds, TugboatStateAnchor, TugboatTargetAnchor, - TugboatEncodedControlScalar, - TugboatVisibleControlValue, - TugboatControlValueScope, - TugboatControlValue, - TugboatSemanticAnnotation, - tugboatControlValueSchemaVersion, - tugboatControlValueTransitionSchemaVersion, - tugboatSemanticAnnotationSchemaVersion, - tugboatControlValueForWidget, - tugboatControlValueFromSemanticsProperties, - tugboatControlValueFromSemanticsNode, - tugboatSemanticAnnotationFromProperties, - tugboatSemanticAnnotationFromNode, - tugboatMergeSemanticAnnotations, - tugboatMergeControlValues, tugboatIconLabel, tugboatIconHash, tugboatLabelHash; diff --git a/packages/tugboat/pubspec.yaml b/packages/tugboat/pubspec.yaml index 2235e0d..58b6b1c 100644 --- a/packages/tugboat/pubspec.yaml +++ b/packages/tugboat/pubspec.yaml @@ -1,7 +1,7 @@ name: tugboat description: >- Screenshot-based session replay with compact interaction anchors for Tugboat. -version: 0.4.18 +version: 0.5.0 repository: https://github.com/blendto/tugboat-flutter issue_tracker: https://github.com/blendto/tugboat-flutter/issues homepage: https://github.com/blendto/tugboat-flutter diff --git a/packages/tugboat/test/control_value_test.dart b/packages/tugboat/test/control_value_test.dart deleted file mode 100644 index d965219..0000000 --- a/packages/tugboat/test/control_value_test.dart +++ /dev/null @@ -1,1125 +0,0 @@ -import 'package:flutter/cupertino.dart'; -import 'package:flutter/material.dart'; -import 'package:flutter/rendering.dart'; -import 'package:flutter_test/flutter_test.dart'; -import 'package:tugboat/tugboat.dart'; - -const _testConfig = TugboatReplayConfig( - profile: TugboatCaptureProfile.exploration, - settleDelay: Duration.zero, - interactionClaimWindow: Duration.zero, - enableGlobalPointerCapture: false, - capturePixelRatio: 1.0, -); - -const _canonicalTestConfig = TugboatReplayConfig( - profile: TugboatCaptureProfile.exploration, - settleDelay: Duration.zero, - interactionClaimWindow: Duration.zero, - interactionPublishMode: TugboatInteractionPublishMode.canonicalOnly, - enableGlobalPointerCapture: false, - capturePixelRatio: 1.0, -); - -class _SemanticsOnlyControl extends LeafRenderObjectWidget { - const _SemanticsOnlyControl({super.key, required this.value}); - - final String value; - - @override - _SemanticsOnlyRenderBox createRenderObject(BuildContext context) => - _SemanticsOnlyRenderBox(value); - - @override - void updateRenderObject( - BuildContext context, - _SemanticsOnlyRenderBox renderObject, - ) { - renderObject.value = value; - } -} - -class _SemanticsOnlyRenderBox extends RenderBox { - _SemanticsOnlyRenderBox(this._value); - - String _value; - - set value(String next) { - if (_value == next) return; - _value = next; - markNeedsSemanticsUpdate(); - } - - @override - bool get sizedByParent => true; - - @override - void performResize() { - size = constraints.constrain(const Size(120, 48)); - } - - @override - bool hitTestSelf(Offset position) => true; - - @override - void describeSemanticsConfiguration(SemanticsConfiguration config) { - super.describeSemanticsConfiguration(config); - config - ..isSemanticBoundary = true - ..isButton = true - ..textDirection = TextDirection.ltr - ..label = 'Semantics only control' - ..value = _value - ..onTap = () {}; - } -} - -Future _waitForCaptures(WidgetTester tester) async { - await tester.pump(); - await tester.runAsync(() async { - await Future.delayed(const Duration(milliseconds: 300)); - }); - await tester.pump(); -} - -Map? _controlValueFrom(TugboatEvent event) { - final raw = event.data['controlValue']; - if (raw is Map) return raw; - if (raw is Map) return Map.from(raw); - return null; -} - -Map? _controlValueTransitionFrom(TugboatEvent event) { - final raw = event.data['controlValueTransition']; - if (raw is Map) return raw; - if (raw is Map) return Map.from(raw); - return null; -} - -Map? _semanticAnnotationFrom(TugboatEvent event) { - final raw = event.data['semanticAnnotation']; - if (raw is Map) return raw; - if (raw is Map) return Map.from(raw); - return null; -} - -void main() { - setUp(TugboatReplay.resetForTest); - tearDown(TugboatReplay.resetForTest); - - group('tugboatControlValueForWidget', () { - test('encodes bool and number literals', () { - final checkbox = tugboatControlValueForWidget( - Checkbox(value: true, onChanged: (_) {}), - ); - expect(checkbox?.role, 'checkbox'); - expect(checkbox?.value?.kind, 'bool'); - expect(checkbox?.value?.value, isTrue); - - final slider = tugboatControlValueForWidget( - Slider(value: 0.4, onChanged: (_) {}), - ); - expect(slider?.role, 'slider'); - expect(slider?.value?.kind, 'number'); - expect(slider?.value?.value, 0.4); - }); - - test('keeps raw string scalars visible', () { - final freeText = TugboatEncodedControlScalar.encode('Secret Option Name'); - expect(freeText.kind, 'string'); - expect(freeText.value, 'Secret Option Name'); - - final oneWordName = TugboatEncodedControlScalar.encode('Alice'); - expect(oneWordName.value, 'Alice'); - - final numericPii = TugboatEncodedControlScalar.encode('123456'); - expect(numericPii.value, '123456'); - - final whitespace = TugboatEncodedControlScalar.encode(' value '); - expect(whitespace.value, ' value '); - final empty = TugboatEncodedControlScalar.encode(''); - expect(empty.toJson(), {'kind': 'string', 'value': ''}); - - final implicitIdentifier = - TugboatEncodedControlScalar.encodeDeveloperToken('123456'); - expect(implicitIdentifier.value, '123456'); - - final explicitIdentifier = - TugboatEncodedControlScalar.encodeDeveloperToken( - 'tugboat:duration-30', - ); - expect(explicitIdentifier.value, 'duration-30'); - }); - - test('keeps encoded numbers JSON-safe', () { - for (final value in [ - double.nan, - double.infinity, - double.negativeInfinity, - ]) { - final encoded = TugboatEncodedControlScalar.encode(value); - expect(encoded.kind, isNot('number')); - expect(() => encoded.toJson(), returnsNormally); - } - }); - - test('reads radio option identity and group selection', () { - final radio = tugboatControlValueForWidget( - // ignore: deprecated_member_use - Radio( - value: 2, - // ignore: deprecated_member_use - groupValue: 1, - // ignore: deprecated_member_use - onChanged: (_) {}, - ), - ); - expect(radio?.role, 'radio'); - expect(radio?.value?.value, 2); - expect(radio?.groupValue?.value, 1); - expect(radio?.selected, isFalse); - }); - - test('keeps explicitly declared safe values visible', () { - final duration = TugboatVisibleControlValue.duration( - const Duration(seconds: 15), - ); - final template = TugboatVisibleControlValue.enumId('modern-minimal'); - - expect(duration.toJson(), {'kind': 'duration_ms', 'value': 15000}); - expect(template.toJson(), {'kind': 'enum', 'value': 'modern-minimal'}); - }); - }); - - testWidgets('developer value scope preserves an explicit slider value', ( - tester, - ) async { - var value = 0.25; - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _testConfig, child: child!), - home: Scaffold( - body: StatefulBuilder( - builder: (context, setState) => TugboatControlValueScope( - controlKey: 'text_curve', - value: TugboatVisibleControlValue.number(value), - role: 'slider', - unit: 'ratio', - min: 0, - max: 1, - step: 0.01, - child: Slider( - key: const Key('visible-slider'), - value: value, - onChanged: (next) => setState(() => value = next), - ), - ), - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.tapAt( - tester.getCenter(find.byKey(const Key('visible-slider'))) + - const Offset(70, 0), - ); - await _waitForCaptures(tester); - - final settled = TugboatReplay.controller!.session!.events.firstWhere( - (event) => event.type == 'tap_settled', - ); - final transition = _controlValueTransitionFrom(settled)!; - final before = transition['before'] as Map; - final after = transition['after'] as Map; - - expect(before['controlKey'], 'text_curve'); - expect(before['unit'], 'ratio'); - expect((before['min'] as Map)['value'], 0); - expect((before['max'] as Map)['value'], 1); - expect((before['step'] as Map)['value'], 0.01); - expect((before['value'] as Map)['kind'], 'number'); - expect((before['value'] as Map)['value'], 0.25); - expect((after['value'] as Map)['kind'], 'number'); - expect((after['value'] as Map)['value'], isNot(0.25)); - }); - - testWidgets('invalid developer range metadata is not emitted', ( - tester, - ) async { - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _testConfig, child: child!), - home: Scaffold( - body: TugboatControlValueScope( - controlKey: 'text_curve', - value: TugboatVisibleControlValue.number(0.5), - min: 1, - max: 0, - step: 0, - child: Slider( - key: const Key('invalid-range-slider'), - value: 0.5, - onChanged: (_) {}, - ), - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.tap(find.byKey(const Key('invalid-range-slider'))); - await _waitForCaptures(tester); - - final tap = TugboatReplay.controller!.session!.events.firstWhere( - (event) => event.type == 'tap', - ); - final controlValue = _controlValueFrom(tap)!; - expect(controlValue, isNot(contains('controlKey'))); - expect(controlValue, isNot(contains('min'))); - }); - - testWidgets('retains raw semantic values across capture sessions', ( - tester, - ) async { - Future captureHash() async { - final targetKey = UniqueKey(); - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _testConfig, child: child!), - home: Scaffold( - body: Center( - child: Semantics( - value: 'Alice', - child: ElevatedButton( - key: targetKey, - onPressed: () {}, - child: const Text('Capture'), - ), - ), - ), - ), - ), - ); - await _waitForCaptures(tester); - await tester.tap(find.byKey(targetKey)); - await _waitForCaptures(tester); - final tap = TugboatReplay.controller!.session!.events.firstWhere( - (event) => event.type == 'tap', - ); - return ((_semanticAnnotationFrom(tap)!['value'] as Map)['value']); - } - - final first = await captureHash(); - await tester.pumpWidget(const SizedBox()); - TugboatReplay.resetForTest(); - final second = await captureHash(); - - expect(first, 'Alice'); - expect(second, 'Alice'); - }); - - testWidgets('controllers retain equivalent raw semantic values', ( - tester, - ) async { - final firstKey = GlobalKey(); - final secondKey = GlobalKey(); - await tester.pumpWidget( - MaterialApp( - home: Row( - children: [ - Expanded( - child: RepaintBoundary( - key: firstKey, - child: Semantics( - button: true, - value: 'Alice', - child: const SizedBox.expand(), - ), - ), - ), - Expanded( - child: RepaintBoundary( - key: secondKey, - child: Semantics( - button: true, - value: 'Alice', - child: const SizedBox.expand(), - ), - ), - ), - ], - ), - ), - ); - - final firstController = TugboatReplayController( - config: _testConfig, - boundaryKey: firstKey, - ); - final secondController = TugboatReplayController( - config: _testConfig, - boundaryKey: secondKey, - ); - await firstController.initialize(); - await secondController.initialize(); - - firstController.start(const Size(400, 600), 'test'); - await tester.pump(); - firstController.recordPointerDown( - tester.getCenter(find.byKey(firstKey)), - pointer: 1, - ); - firstController.recordPointerUp( - tester.getCenter(find.byKey(firstKey)), - pointer: 1, - ); - final firstTap = firstController.session!.events.lastWhere( - (event) => event.type == 'tap', - ); - final firstHash = - ((_semanticAnnotationFrom(firstTap)!['value'] as Map)['value']); - - secondController.start(const Size(400, 600), 'test'); - await tester.pump(); - firstController.recordPointerDown( - tester.getCenter(find.byKey(firstKey)), - pointer: 2, - ); - firstController.recordPointerUp( - tester.getCenter(find.byKey(firstKey)), - pointer: 2, - ); - final secondTap = firstController.session!.events.lastWhere( - (event) => event.type == 'tap', - ); - final secondHash = - ((_semanticAnnotationFrom(secondTap)!['value'] as Map)['value']); - - expect(firstHash, secondHash); - firstController.dispose(); - secondController.dispose(); - await tester.pumpWidget(const SizedBox()); - }); - - testWidgets('switch tap emits before/after control values', (tester) async { - var enabled = false; - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _testConfig, child: child!), - home: Scaffold( - body: StatefulBuilder( - builder: (context, setState) { - return Switch( - key: const Key('notify-switch'), - value: enabled, - onChanged: (next) => setState(() => enabled = next), - ); - }, - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.tap(find.byKey(const Key('notify-switch'))); - await _waitForCaptures(tester); - - final session = TugboatReplay.controller!.session!; - final tap = session.events.firstWhere((e) => e.type == 'tap'); - final settled = session.events.firstWhere((e) => e.type == 'tap_settled'); - - final tapValue = _controlValueFrom(tap); - expect(tapValue?['role'], 'switch'); - expect((tapValue?['value'] as Map)['value'], isFalse); - - final settledValue = _controlValueTransitionFrom(settled); - expect(settledValue?['role'], 'switch'); - expect( - settledValue?['schemaVersion'], - tugboatControlValueTransitionSchemaVersion, - ); - expect(settled.data, isNot(contains('controlValue'))); - expect((settledValue?['before'] as Map)['value'], isA()); - expect( - (settledValue?['before'] as Map)['schemaVersion'], - tugboatControlValueSchemaVersion, - ); - expect( - (settledValue?['after'] as Map)['schemaVersion'], - tugboatControlValueSchemaVersion, - ); - expect( - ((settledValue?['before'] as Map)['value'] as Map)['value'], - isFalse, - ); - expect(((settledValue?['after'] as Map)['value'] as Map)['value'], isTrue); - expect(enabled, isTrue); - }); - - testWidgets('canonical-only tap retains the control transition', ( - tester, - ) async { - var enabled = false; - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _canonicalTestConfig, child: child!), - home: Scaffold( - body: StatefulBuilder( - builder: (context, setState) => Switch( - key: const Key('canonical-switch'), - value: enabled, - onChanged: (next) => setState(() => enabled = next), - ), - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.tap(find.byKey(const Key('canonical-switch'))); - await _waitForCaptures(tester); - - final session = TugboatReplay.controller!.session!; - expect( - session.events.where((event) => event.type == 'tap_settled'), - isEmpty, - ); - final interaction = session.events.firstWhere( - (event) => event.type == 'interaction', - ); - final result = Map.from( - interaction.data['result']! as Map, - ); - final transition = Map.from( - result['controlValueTransition']! as Map, - ); - expect(transition['role'], 'switch'); - expect(((transition['before'] as Map)['value'] as Map)['value'], isFalse); - expect(((transition['after'] as Map)['value'] as Map)['value'], isTrue); - }); - - testWidgets('canonical-only swipe retains final control metadata', ( - tester, - ) async { - var value = 0.0; - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _canonicalTestConfig, child: child!), - home: Scaffold( - body: StatefulBuilder( - builder: (context, setState) => Slider( - key: const Key('canonical-slider'), - value: value, - onChanged: (next) => setState(() => value = next), - ), - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.drag( - find.byKey(const Key('canonical-slider')), - const Offset(80, 0), - ); - await _waitForCaptures(tester); - - final session = TugboatReplay.controller!.session!; - expect(session.events.where((event) => event.type == 'swipe'), isEmpty); - final interaction = session.events.firstWhere( - (event) => - event.type == 'interaction' && - (event.data['gesture'] == 'swipe' || - event.data['gesture'] == 'scroll'), - ); - final result = Map.from( - interaction.data['result']! as Map, - ); - final controlValue = Map.from( - result['controlValue']! as Map, - ); - expect(controlValue['role'], 'slider'); - expect((controlValue['value'] as Map)['value'], isA()); - }); - - testWidgets('radio tap records which option was selected', (tester) async { - int? selected = 1; - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _testConfig, child: child!), - home: Scaffold( - body: StatefulBuilder( - builder: (context, setState) { - return Column( - children: [ - // ignore: deprecated_member_use - RadioListTile( - key: const Key('radio-1'), - title: const Text('One'), - value: 1, - // ignore: deprecated_member_use - groupValue: selected, - // ignore: deprecated_member_use - onChanged: (next) => setState(() => selected = next), - ), - // ignore: deprecated_member_use - RadioListTile( - key: const Key('radio-2'), - title: const Text('Two'), - value: 2, - // ignore: deprecated_member_use - groupValue: selected, - // ignore: deprecated_member_use - onChanged: (next) => setState(() => selected = next), - ), - ], - ); - }, - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.tap(find.byKey(const Key('radio-2'))); - await _waitForCaptures(tester); - - final session = TugboatReplay.controller!.session!; - final tap = session.events.firstWhere((e) => e.type == 'tap'); - final tapValue = _controlValueFrom(tap)!; - expect(tapValue['role'], 'radio'); - expect((tapValue['value'] as Map)['value'], 2); - expect((tapValue['groupValue'] as Map)['value'], 1); - expect(tapValue['selected'], isFalse); - expect(tapValue['index'], 1); - - final settled = session.events.firstWhere((e) => e.type == 'tap_settled'); - final settledValue = _controlValueTransitionFrom(settled)!; - expect(((settledValue['after'] as Map)['value'] as Map)['value'], 2); - expect(((settledValue['after'] as Map)['groupValue'] as Map)['value'], 2); - expect((settledValue['after'] as Map)['selected'], isTrue); - expect(selected, 2); - }); - - testWidgets('dropdown item tap records the chosen option value', ( - tester, - ) async { - var selected = 1; - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _testConfig, child: child!), - home: Scaffold( - body: StatefulBuilder( - builder: (context, setState) { - return DropdownButton( - key: const Key('plan-dropdown'), - value: selected, - items: const [ - DropdownMenuItem(value: 1, child: Text('Starter')), - DropdownMenuItem(value: 2, child: Text('Pro')), - ], - onChanged: (next) { - if (next != null) setState(() => selected = next); - }, - ); - }, - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.tap(find.byKey(const Key('plan-dropdown'))); - await tester.pumpAndSettle(); - await tester.tap(find.text('Pro').last); - await _waitForCaptures(tester); - - final session = TugboatReplay.controller!.session!; - final itemTaps = session.events - .where((e) => e.type == 'tap') - .map(_controlValueFrom) - .where((value) => value?['role'] == 'dropdownItem') - .toList(); - expect(itemTaps, isNotEmpty); - expect((itemTaps.last!['value'] as Map)['value'], 2); - expect(selected, 2); - }); - - testWidgets('slider drag swipe records numeric value', (tester) async { - var value = 0.0; - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _testConfig, child: child!), - home: Scaffold( - body: StatefulBuilder( - builder: (context, setState) { - return Slider( - key: const Key('volume-slider'), - value: value, - onChanged: (next) => setState(() => value = next), - ); - }, - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.drag( - find.byKey(const Key('volume-slider')), - const Offset(80, 0), - ); - await _waitForCaptures(tester); - - final session = TugboatReplay.controller!.session!; - final swipes = session.events.where((e) => e.type == 'swipe').toList(); - expect(swipes, isNotEmpty); - final controlValue = _controlValueFrom(swipes.last); - expect(controlValue?['role'], 'slider'); - expect((controlValue?['value'] as Map)['kind'], 'number'); - expect((controlValue?['value'] as Map)['value'], isA()); - expect(value, greaterThan(0)); - }); - - testWidgets('cupertino switch values are captured', (tester) async { - var enabled = true; - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _testConfig, child: child!), - home: Scaffold( - body: StatefulBuilder( - builder: (context, setState) { - return CupertinoSwitch( - key: const Key('cupertino-switch'), - value: enabled, - onChanged: (next) => setState(() => enabled = next), - ); - }, - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.tap(find.byKey(const Key('cupertino-switch'))); - await _waitForCaptures(tester); - - final tap = TugboatReplay.controller!.session!.events.firstWhere( - (e) => e.type == 'tap', - ); - final tapValue = _controlValueFrom(tap); - expect(tapValue?['role'], 'switch'); - expect((tapValue?['value'] as Map)['value'], isTrue); - }); - - testWidgets('free-text dropdown values stay visible in session json', ( - tester, - ) async { - var selected = 'alpha-code'; - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _testConfig, child: child!), - home: Scaffold( - body: StatefulBuilder( - builder: (context, setState) { - return DropdownButton( - value: selected, - items: const [ - DropdownMenuItem(value: 'alpha-code', child: Text('Alpha')), - DropdownMenuItem( - value: 'Visible Secret City Name', - child: Text('Beta'), - ), - ], - onChanged: (next) { - if (next != null) setState(() => selected = next); - }, - ); - }, - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.tap(find.byType(DropdownButton)); - await tester.pumpAndSettle(); - await tester.tap(find.text('Beta').last); - await _waitForCaptures(tester); - - final json = TugboatReplay.controller!.session!.toJson().toString(); - expect(json, contains('Visible Secret City Name')); - expect(json, contains('alpha-code')); - }); - - test('semantic properties retain raw value and label strings', () { - final snapshot = tugboatControlValueFromSemanticsProperties( - const SemanticsProperties( - button: true, - value: '15', - label: 'Duration fifteen seconds', - selected: true, - ), - ); - expect(snapshot?.role, 'button'); - expect(snapshot?.sources, ['semantics']); - expect(snapshot?.value?.kind, 'string'); - expect(snapshot?.value?.value, '15'); - expect(snapshot?.semanticValue?.kind, 'string'); - expect(snapshot?.semanticValue?.value, '15'); - expect(snapshot?.semanticLabel?.value, 'Duration fifteen seconds'); - expect(snapshot?.selected, isTrue); - }); - - testWidgets('custom gesture detector list captures semantic value/label', ( - tester, - ) async { - String? selected; - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _testConfig, child: child!), - home: Scaffold( - body: StatefulBuilder( - builder: (context, setState) { - return Column( - children: [ - Semantics( - button: true, - identifier: 'tugboat:duration-15', - value: '15', - label: 'Duration 15 seconds', - selected: selected == '15', - child: GestureDetector( - key: const Key('duration-15'), - onTap: () => setState(() => selected = '15'), - child: const Text('15 seconds'), - ), - ), - Semantics( - button: true, - identifier: 'tugboat:duration-30', - value: '30', - label: 'Duration 30 seconds', - selected: selected == '30', - child: GestureDetector( - key: const Key('duration-30'), - onTap: () => setState(() => selected = '30'), - child: const Text('30 seconds'), - ), - ), - ], - ); - }, - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.tap(find.byKey(const Key('duration-30'))); - await _waitForCaptures(tester); - - final session = TugboatReplay.controller!.session!; - final tap = session.events.firstWhere((e) => e.type == 'tap'); - final tapValue = _controlValueFrom(tap)!; - expect(tapValue['sources'], contains('semantics')); - expect((tapValue['semanticValue'] as Map)['value'], '30'); - expect((tapValue['value'] as Map)['value'], '30'); - expect((tapValue['semanticLabel'] as Map)['value'], 'Duration 30 seconds'); - expect(tapValue.toString(), contains('Duration 30 seconds')); - - final annotation = _semanticAnnotationFrom(tap)!; - expect(annotation['schemaVersion'], tugboatSemanticAnnotationSchemaVersion); - expect(annotation['role'], 'button'); - expect((annotation['identifier'] as Map)['value'], 'duration-30'); - expect((annotation['value'] as Map)['value'], '30'); - expect((annotation['label'] as Map)['value'], 'Duration 30 seconds'); - expect(selected, '30'); - }); - - testWidgets('button taps emit semanticAnnotation labels', (tester) async { - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _testConfig, child: child!), - home: Scaffold( - body: FilledButton( - key: const Key('generate-cta'), - onPressed: () {}, - child: const Text('Generate'), - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.tap(find.byKey(const Key('generate-cta'))); - await _waitForCaptures(tester); - - final session = TugboatReplay.controller!.session!; - final tap = session.events.firstWhere((e) => e.type == 'tap'); - final settled = session.events.firstWhere((e) => e.type == 'tap_settled'); - final tapSemantic = _semanticAnnotationFrom(tap); - final settledSemantic = _semanticAnnotationFrom(settled); - - expect(tapSemantic, isNotNull); - expect(tapSemantic?['role'], 'button'); - expect((tapSemantic?['label'] as Map)['value'], 'Generate'); - expect(settledSemantic, isNotNull); - expect((settledSemantic?['label'] as Map)['value'], 'Generate'); - }); - - testWidgets('rapid taps retain per-interaction after values', (tester) async { - var enabled = false; - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => TugboatReplay.wrapApp( - config: const TugboatReplayConfig( - profile: TugboatCaptureProfile.exploration, - settleDelay: Duration(milliseconds: 120), - interactionClaimWindow: Duration.zero, - enableGlobalPointerCapture: false, - capturePixelRatio: 1.0, - ), - child: child!, - ), - home: Scaffold( - body: StatefulBuilder( - builder: (context, setState) { - return Switch( - key: const Key('rapid-switch'), - value: enabled, - onChanged: (next) => setState(() => enabled = next), - ); - }, - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.tap(find.byKey(const Key('rapid-switch'))); - await tester.pump(); - await tester.tap(find.byKey(const Key('rapid-switch'))); - await tester.pump(); - await tester.pump(const Duration(milliseconds: 150)); - for (var i = 0; i < 3; i += 1) { - await _waitForCaptures(tester); - } - - final session = TugboatReplay.controller!.session!; - final taps = session.events.where((event) => event.type == 'tap').toList(); - final settles = session.events - .where((event) => event.type == 'tap_settled') - .toList(); - expect(taps, hasLength(2)); - expect(settles, hasLength(2)); - - final first = _controlValueTransitionFrom( - settles.firstWhere((event) => event.relatedEventId == taps[0].id), - )!; - final second = _controlValueTransitionFrom( - settles.firstWhere((event) => event.relatedEventId == taps[1].id), - )!; - - expect(((first['before'] as Map)['value'] as Map)['value'], isFalse); - expect(((first['after'] as Map)['value'] as Map)['value'], isTrue); - expect(((second['before'] as Map)['value'] as Map)['value'], isTrue); - expect(((second['after'] as Map)['value'] as Map)['value'], isFalse); - }); - - testWidgets('same-frame taps omit ambiguous after values', (tester) async { - var enabled = false; - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _testConfig, child: child!), - home: Scaffold( - body: StatefulBuilder( - builder: (context, setState) { - return Switch( - key: const Key('same-frame-switch'), - value: enabled, - onChanged: (next) => setState(() => enabled = next), - ); - }, - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.tap(find.byKey(const Key('same-frame-switch'))); - await tester.tap(find.byKey(const Key('same-frame-switch'))); - await tester.pump(); - for (var i = 0; i < 3; i += 1) { - await _waitForCaptures(tester); - } - - final session = TugboatReplay.controller!.session!; - final settles = session.events - .where((event) => event.type == 'tap_settled') - .toList(); - expect(settles, hasLength(2)); - for (final settled in settles) { - final transition = _controlValueTransitionFrom(settled)!; - expect(transition, contains('before')); - expect(transition, isNot(contains('after'))); - } - }); - - testWidgets('semantics-only controls are resampled without accessibility', ( - tester, - ) async { - const renderKey = GlobalObjectKey('semantics-only-control'); - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => TugboatReplay.wrapApp( - config: const TugboatReplayConfig( - profile: TugboatCaptureProfile.productionLean, - settleDelay: Duration.zero, - interactionClaimWindow: Duration.zero, - enableGlobalPointerCapture: false, - capturePixelRatio: 1.0, - ), - child: child!, - ), - home: Scaffold( - body: GestureDetector( - onTap: () { - final renderObject = renderKey.currentContext!.findRenderObject(); - (renderObject! as _SemanticsOnlyRenderBox).value = 'on'; - }, - child: const _SemanticsOnlyControl(key: renderKey, value: 'off'), - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.tap(find.byKey(renderKey)); - await _waitForCaptures(tester); - - final settled = TugboatReplay.controller!.session!.events.firstWhere( - (event) => event.type == 'tap_settled', - ); - final transition = _controlValueTransitionFrom(settled)!; - expect(((transition['before'] as Map)['value'] as Map)['value'], 'off'); - expect(((transition['after'] as Map)['value'] as Map)['value'], 'on'); - expect( - ((transition['after'] as Map)['value'] as Map)['value'], - isNot(((transition['before'] as Map)['value'] as Map)['value']), - ); - }); - - testWidgets('settle never borrows a replacement at the old coordinate', ( - tester, - ) async { - var showOriginal = true; - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _testConfig, child: child!), - home: Scaffold( - body: StatefulBuilder( - builder: (context, setState) { - if (showOriginal) { - return Semantics( - identifier: 'tugboat:original-switch', - child: Switch( - key: const Key('original-switch'), - value: false, - onChanged: (_) => setState(() => showOriginal = false), - ), - ); - } - return Semantics( - identifier: 'tugboat:replacement-switch', - child: const Switch( - key: Key('replacement-switch'), - value: true, - onChanged: null, - ), - ); - }, - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.tap(find.byKey(const Key('original-switch'))); - await _waitForCaptures(tester); - - final settled = TugboatReplay.controller!.session!.events.firstWhere( - (event) => event.type == 'tap_settled', - ); - final transition = _controlValueTransitionFrom(settled)!; - final annotation = _semanticAnnotationFrom(settled)!; - - expect(((transition['before'] as Map)['value'] as Map)['value'], isFalse); - expect(transition, isNot(contains('after'))); - expect((annotation['identifier'] as Map)['value'], 'original-switch'); - expect( - (annotation['identifier'] as Map)['value'], - isNot('replacement-switch'), - ); - }); - - testWidgets('scroll events carry semanticAnnotation when present', ( - tester, - ) async { - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _testConfig, child: child!), - home: Scaffold( - body: Semantics( - identifier: 'tugboat:preset-list', - label: 'Preset options', - child: ListView( - key: const Key('preset-list'), - children: [ - for (var i = 0; i < 30; i++) ListTile(title: Text('Preset $i')), - ], - ), - ), - ), - ), - ); - await _waitForCaptures(tester); - - await tester.drag( - find.byKey(const Key('preset-list')), - const Offset(0, -200), - ); - await _waitForCaptures(tester); - - final session = TugboatReplay.controller!.session!; - final scrollEvents = session.events - .where((e) => e.type == 'scroll_start' || e.type == 'scroll_end') - .toList(); - final start = scrollEvents.firstWhere( - (event) => event.type == 'scroll_start', - ); - final end = scrollEvents.firstWhere((event) => event.type == 'scroll_end'); - for (final event in [start, end]) { - final annotation = _semanticAnnotationFrom(event); - expect(annotation, isNotNull); - expect((annotation?['identifier'] as Map)['value'], 'preset-list'); - } - }); -} diff --git a/packages/tugboat/test/fingerprint_test.dart b/packages/tugboat/test/fingerprint_test.dart index d071245..8d39066 100644 --- a/packages/tugboat/test/fingerprint_test.dart +++ b/packages/tugboat/test/fingerprint_test.dart @@ -608,6 +608,85 @@ void main() { selected.dispose(); }); + testWidgets('typed radio callbacks resolve roles and anchors', ( + tester, + ) async { + final rootKey = GlobalKey(); + final selected = ValueNotifier(1); + const radioKey = ValueKey('typed-radio'); + const radioTileKey = ValueKey('typed-radio-tile'); + + await tester.pumpWidget( + MaterialApp( + home: RepaintBoundary( + key: rootKey, + child: Scaffold( + body: Column( + children: [ + Radio( + key: radioKey, + value: 1, + // ignore: deprecated_member_use + groupValue: 1, + // ignore: deprecated_member_use + onChanged: (next) async { + if (next != null) selected.value = next; + }, + ), + RadioListTile( + key: radioTileKey, + value: 2, + // ignore: deprecated_member_use + groupValue: 1, + title: const Text('Second option'), + // ignore: deprecated_member_use + onChanged: (next) async { + if (next != null) selected.value = next; + }, + ), + ], + ), + ), + ), + ), + ); + await tester.pump(); + + final radioFinder = find.byKey(radioKey); + final radioTileFinder = find.byKey(radioTileKey); + final resolver = AnchorResolver(rootKey: rootKey); + + final radioRole = tugboatRoleForWidget( + tester.widget>(radioFinder), + ); + final radioTileRole = tugboatRoleForWidget( + tester.widget>(radioTileFinder), + ); + final radioAnchor = resolver.targetAt( + tester.getCenter(radioFinder), + route: '/radios', + ); + final radioTileAnchor = resolver.targetAt( + tester.getCenter(radioTileFinder), + route: '/radios', + ); + + expect(radioRole?.name, 'radio'); + expect(radioRole?.enabled, isTrue); + expect(radioTileRole?.name, 'radio'); + expect(radioTileRole?.enabled, isTrue); + // Radio controls are painted through InkResponse, so hit testing preserves + // the existing inner-button target role. The direct role checks above + // verify Radio classification; these assertions cover anchor resolution. + expect(radioAnchor, isNotNull); + expect(radioAnchor?.enabled, isTrue); + expect(radioAnchor?.actions, contains('tap')); + expect(radioTileAnchor, isNotNull); + expect(radioTileAnchor?.enabled, isTrue); + expect(radioTileAnchor?.actions, contains('tap')); + selected.dispose(); + }); + testWidgets('InkWell-based button yields non-empty canonical path', ( tester, ) async { diff --git a/packages/tugboat/test/helpers/json_roundtrip.dart b/packages/tugboat/test/helpers/json_roundtrip.dart index 9f7513a..33d0d3d 100644 --- a/packages/tugboat/test/helpers/json_roundtrip.dart +++ b/packages/tugboat/test/helpers/json_roundtrip.dart @@ -113,7 +113,7 @@ extension TugboatEventTestJson on TugboatEvent { extension TugboatSessionTestJson on TugboatSession { static TugboatSession fromJson(Map json) { final version = json['schemaVersion'] as int?; - if (version != 6 && version != 7 && version != 8) { + if (version != 6 && version != 7 && version != 8 && version != 9) { throw const FormatException( 'Unsupported Tugboat session schema version.', ); diff --git a/packages/tugboat/test/integration/release_compatibility_matrix_test.dart b/packages/tugboat/test/integration/release_compatibility_matrix_test.dart index 9148716..cd7640b 100644 --- a/packages/tugboat/test/integration/release_compatibility_matrix_test.dart +++ b/packages/tugboat/test/integration/release_compatibility_matrix_test.dart @@ -2,6 +2,8 @@ import 'package:flutter/material.dart'; import 'package:flutter_test/flutter_test.dart'; import 'package:tugboat/tugboat.dart'; +import '../helpers/json_roundtrip.dart'; + Future _waitForCaptures(WidgetTester tester) async { await tester.pump(); await tester.runAsync(() async { @@ -114,8 +116,43 @@ void main() { expect(routes, isNotEmpty); }); - test('v6 session JSON remains readable alongside v7 writers', () { - expect(tugboatSessionSchemaVersion, 8); + test('v6-v8 session JSON remains readable alongside v9 writers', () { + final session = TugboatSession( + id: 'legacy-session', + startedAt: DateTime.utc(2026, 8, 3), + platform: 'test', + viewport: const TugboatRect(0, 0, 100, 200), + ); + final writerJson = session.toJson(); + expect(writerJson['schemaVersion'], 9); + + for (final version in [6, 7, 8]) { + final legacyJson = Map.from(writerJson) + ..['schemaVersion'] = version + ..['events'] = [ + { + 'id': 'legacy-event-$version', + 'atMs': 0, + 'type': 'tap', + 'data': { + 'controlValue': {'kind': 'number', 'value': 0.5}, + 'controlValueTransition': { + 'before': {'kind': 'number', 'value': 0.4}, + 'after': {'kind': 'number', 'value': 0.5}, + }, + 'semanticAnnotation': { + 'label': {'kind': 'string', 'value': 'Legacy label'}, + }, + }, + }, + ]; + + final restored = TugboatSessionTestJson.fromJson(legacyJson); + expect(restored.id, 'legacy-session'); + expect(restored.events.single.data, contains('controlValue')); + expect(restored.events.single.data, contains('controlValueTransition')); + expect(restored.events.single.data, contains('semanticAnnotation')); + } }); test( diff --git a/packages/tugboat/test/semantics_flags_compat_test.dart b/packages/tugboat/test/semantics_flags_compat_test.dart index 3b519d8..6a83ac0 100644 --- a/packages/tugboat/test/semantics_flags_compat_test.dart +++ b/packages/tugboat/test/semantics_flags_compat_test.dart @@ -1,6 +1,5 @@ import 'dart:ui'; -import 'package:flutter/material.dart'; import 'package:flutter_test/flutter_test.dart'; import 'package:tugboat/src/semantics_flags_compat.dart'; @@ -12,66 +11,18 @@ void main() { }, ); - testWidgets('semanticsEnabledFromFlags reads explicit enabled state', ( - tester, - ) async { - for (final value in [true, false]) { - await tester.pumpWidget( - Directionality( - textDirection: TextDirection.ltr, - child: Semantics( - enabled: value, - child: const SizedBox(width: 20, height: 20), - ), - ), - ); - final flags = tester - .getSemantics(find.byType(Semantics)) - .getSemanticsData() - .flagsCollection; - expect(semanticsEnabledFromFlags(flags), value); - } - }); - - testWidgets('semanticsCheckedFromFlags reads true, false, and none', ( - tester, - ) async { - for (final value in [true, false, null]) { - await tester.pumpWidget( - Directionality( - textDirection: TextDirection.ltr, - child: Semantics( - checked: value, - child: const SizedBox(width: 20, height: 20), - ), - ), - ); - final flags = tester - .getSemantics(find.byType(Semantics)) - .getSemanticsData() - .flagsCollection; - expect(semanticsCheckedFromFlags(flags), value); - } - }); - - testWidgets('semanticsCheckedFromFlags treats mixed as unavailable', ( - tester, - ) async { - await tester.pumpWidget( - MaterialApp( - home: Scaffold( - body: Checkbox(tristate: true, value: null, onChanged: (_) {}), - ), + test('semanticsEnabledFromFlags reads explicit enabled state', () { + expect( + semanticsEnabledFromFlags( + SemanticsFlags.none.copyWith(hasEnabledState: true, isEnabled: true), ), + isTrue, ); - final flags = tester - .getSemantics(find.byType(Checkbox)) - .getSemanticsData() - .flagsCollection; - final dynamic runtimeFlags = flags; expect( - semanticsCheckedFromFlags(flags), - runtimeFlags.isChecked is bool ? isFalse : isNull, + semanticsEnabledFromFlags( + SemanticsFlags.none.copyWith(hasEnabledState: true, isEnabled: false), + ), + isFalse, ); }); } diff --git a/packages/tugboat/test/tugboat_replay_test.dart b/packages/tugboat/test/tugboat_replay_test.dart index 720fd0c..d852e1d 100644 --- a/packages/tugboat/test/tugboat_replay_test.dart +++ b/packages/tugboat/test/tugboat_replay_test.dart @@ -49,55 +49,6 @@ bool _containsLabelTelemetry(Object? value) { } void main() { - testWidgets('popup option emits its semantic parameter pair', (tester) async { - addTearDown(TugboatReplay.resetForTest); - - await tester.pumpWidget( - MaterialApp( - builder: (context, child) => - TugboatReplay.wrapApp(config: _testConfig, child: child!), - home: Builder( - builder: (context) => Scaffold( - body: FilledButton( - onPressed: () => showDialog( - context: context, - builder: (dialogContext) => AlertDialog( - content: Semantics( - container: true, - excludeSemantics: true, - label: 'Image quality', - value: '2K', - button: true, - onTap: () => Navigator.pop(dialogContext), - child: TextButton( - onPressed: () => Navigator.pop(dialogContext), - child: const Text('2K • Advanced'), - ), - ), - ), - ), - child: const Text('Choose quality'), - ), - ), - ), - ), - ); - await tester.pump(); - await tester.tap(find.text('Choose quality')); - await tester.pumpAndSettle(); - await tester.tap(find.text('2K • Advanced')); - await tester.pump(); - - final tap = TugboatReplay.controller!.session!.events - .where((event) => event.type == 'tap') - .last; - final semantic = Map.from( - tap.data['semanticAnnotation'] as Map, - ); - expect(semantic['label'], {'kind': 'string', 'value': 'Image quality'}); - expect(semantic['value'], {'kind': 'string', 'value': '2K'}); - }); - testWidgets('detached lifecycle emits session_end once', (tester) async { await tester.pumpWidget( MaterialApp( @@ -884,6 +835,48 @@ void main() { expect(_containsLabelTelemetry(anchor.toJson()), isFalse); }); + testWidgets('does not emit control or semantic value telemetry', ( + tester, + ) async { + var enabled = false; + await tester.pumpWidget( + MaterialApp( + builder: (context, child) => + TugboatReplay.wrapApp(config: _testConfig, child: child!), + home: Scaffold( + body: StatefulBuilder( + builder: (context, setState) => Switch( + value: enabled, + onChanged: (next) => setState(() => enabled = next), + ), + ), + ), + ), + ); + await _waitForCaptures(tester); + await tester.tap(find.byType(Switch)); + await _waitForCaptures(tester); + + final session = TugboatReplay.controller!.session!; + final tap = session.events.firstWhere((event) => event.type == 'tap'); + final settled = session.events.firstWhere( + (event) => event.type == 'tap_settled' && event.relatedEventId == tap.id, + ); + expect(enabled, isTrue); + expect(tap.targetAnchor, isNotNull); + expect(settled.targetAnchor, isNotNull); + for (final event in [tap, settled]) { + expect(event.data.containsKey('controlValue'), isFalse); + expect(event.data.containsKey('controlValueTransition'), isFalse); + expect(event.data.containsKey('semanticAnnotation'), isFalse); + } + + final sessionJson = jsonEncode(session.toJson()); + expect(sessionJson, isNot(contains('"controlValue"'))); + expect(sessionJson, isNot(contains('"controlValueTransition"'))); + expect(sessionJson, isNot(contains('"semanticAnnotation"'))); + }); + test('session round-trips through JSON', () { const appInfo = TugboatCollectorAppInfo( name: 'Example App', @@ -916,7 +909,7 @@ void main() { ); final json = jsonDecode(session.toPrettyJson()) as Map; - expect(json['schemaVersion'], 8); + expect(json['schemaVersion'], 9); expect(json.containsKey('routes'), isFalse); expect(json['events'], [isNot(contains('route'))]); expect(json['frames'], [containsPair('captureMicros', 12345)]); From 04154ca9294dbf785fa2cd66bdb6bad04fe15bf1 Mon Sep 17 00:00:00 2001 From: Chinmay-KB Date: Mon, 3 Aug 2026 19:30:28 +0530 Subject: [PATCH 2/2] docs: refresh production replay acceptance --- .../production-replay-acceptance.md | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/docs/integration/production-replay-acceptance.md b/docs/integration/production-replay-acceptance.md index f11f1a8..7efbe38 100644 --- a/docs/integration/production-replay-acceptance.md +++ b/docs/integration/production-replay-acceptance.md @@ -12,12 +12,16 @@ database receipt alone as proof that a replay is correct. ## Current acceptance status -Interaction consolidation shipped in SDK **0.4.15** (canonical `interaction` -events, 1,250 ms delayed claim window, diagnostic stream isolation). Use +The current SDK release candidate is **0.5.0**, which writes session schema +**v9**. It preserves structural interaction replay while no longer emitting +`controlValue`, `controlValueTransition`, or `semanticAnnotation` in event +data. Treat the absence of those fields as the expected privacy boundary, not +as missing capture evidence. + [`production-replay-acceptance-0.4.15.md`](./production-replay-acceptance-0.4.15.md) -for the Blend scoring gates. Collector/Context Graph migration onto -`stream: semantic` interactions remains a follow-up before legacy -`tap`/`tap_settled` projection can be removed. +remains a historical acceptance record for interaction consolidation. Do not +use its SDK version, legacy-projection assumptions, or scoring baseline as the +current release contract. Production acceptance #13/#14 remains open for rapid/nested modal chains and programmatic/automatic navigation gaps. Record those observations as SDK @@ -160,7 +164,7 @@ flows share a session, list the event IDs or timestamps that delimit each flow. Wait until the collector session has finalized and the replay is available in the production website. Filter to the recorded Blend build and SDK version -`0.4.10` (or the version under test), then open every recorded session. +under test (`0.5.0` for this release), then open every recorded session. For each interaction, inspect the actual replay UI and verify: @@ -189,7 +193,7 @@ Use one row per production session: | Session ID | UTC range | Blend build | SDK version / SHA | Flows | Frame availability | Route/action coherence | Verdict | Follow-up | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| `` | ` - ` | `` | `0.4.10 / ` | `` | pass/fail | pass/fail | accept/reject | `` | +| `` | ` - ` | `` | ` / ` | `` | pass/fail | pass/fail | accept/reject | `` | The cohort passes only when: