test: add replay coherence characterization harness (#5) - #15
Merged
Conversation
Add deterministic, advanceable scheduler/capture test seams and characterization coverage for known navigation/frame races without changing production capture behavior. Bumps tugboat to 0.4.1. Co-authored-by: Chinmay Kabi <chinmay@blend.to>
There was a problem hiding this comment.
Pull request overview
Adds a deterministic, advanceable test harness to reproduce and characterize known replay coherence races (navigation vs capture vs tap settlement), plus test-only controller seams to control time/delays/capture execution and seed frames. This supports the “SDK replay correctness” milestone by locking in today’s broken-but-observed sequences so follow-up fixes (#6–#10) can flip invariants without rewriting the harness.
Changes:
- Introduces a reusable replay coherence harness (
ControllableScheduler,ControllableCaptureExecutor,ReplayCoherenceHarness,CoherenceInvariants) and characterization scenarios covering known ordering/frame attribution races. - Adds test-only controller seams (
debugNow,debugDelay,debugExecuteCapture,debugSeedFrame,debugFreezeStateAnchor) to make controller timing/capture deterministic without wall-clock sleeps. - Bumps SDK/package version
0.4.0 → 0.4.1and updates changelog +tugboatSdkVersion.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/tugboat/test/replay_coherence_characterization_test.dart | Adds characterization tests for known replay coherence race sequences and invariant checks. |
| packages/tugboat/test/helpers/replay_coherence_harness.dart | Adds deterministic scheduler/capture executor harness and invariant helpers for replay coherence testing. |
| packages/tugboat/pubspec.yaml | Version bump to 0.4.1. |
| packages/tugboat/lib/src/sdk_version.dart | Keeps SDK version constant in sync with pubspec bump. |
| packages/tugboat/lib/src/controller.dart | Adds test-only seams for deterministic timing/capture and frame seeding; routes/capture waits now use overridable delay/clock. |
| packages/tugboat/CHANGELOG.md | Documents the new characterization harness/seams. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Keep zero-duration waits yielding via Future.delayed/debugDelay, and refresh state plus emit inventory on the debugExecuteCapture path so the test seam matches production capture semantics. Co-authored-by: Chinmay Kabi <chinmay@blend.to>
|
Addressed both Copilot review comments in eee3d26:
Tests still green on the characterization suite + |
Replace content-hash frame families with explicit harness route+epoch provenance so actionFrameMatchesRoute requires destination attribution. Co-authored-by: Cursor <cursoragent@cursor.com>
Require an exact destination route event id and tap-relative ordering in navigationTapHasNoEarlyNoVisibleChange harness checks. Co-authored-by: Cursor <cursoragent@cursor.com>
Mirror package widget tests with MaterialApp/Scaffold, RepaintBoundary, and a keyed FilledButton target mounted under the harness boundary key. Co-authored-by: Cursor <cursoragent@cursor.com>
Assert stable widget-backed target anchors and compare tap state signatures against scene_inventory rather than hard-coded synthetic values. Co-authored-by: Cursor <cursoragent@cursor.com>
Exercise the controller swipe classification seam and assert tap/swipe subsequence ordering without requiring an exact full event list. Co-authored-by: Cursor <cursoragent@cursor.com>
Require tap/settle before and after frames to share route+epoch provenance without forcing afterFrame to equal the origin capture id. Co-authored-by: Cursor <cursoragent@cursor.com>
Add a harness-only auto-release seam that cancels blocked captures with null instead of seeding a success frame, keeping manual completion separate. Co-authored-by: Cursor <cursoragent@cursor.com>
Apply dart format to the replay coherence harness and characterization tests. Co-authored-by: Cursor <cursoragent@cursor.com>
Resolve captured-frame route provenance from the frozen state anchor and dispose widget-backed harness sessions cleanly to avoid semantics leaks. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Chinmay-KB
marked this pull request as ready for review
July 25, 2026 08:09
This was referenced Jul 25, 2026
This was referenced Jul 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements #5 from the SDK replay correctness milestone: deterministic characterization coverage for known navigation/frame races, with no production capture behavior changes.
What’s included
debugNow,debugDelay,debugExecuteCapture,debugSeedFrame,debugFreezeStateAnchor, route/capture debug getters)ControllableScheduler,ControllableCaptureExecutor,ReplayCoherenceHarness,CoherenceInvariantsnoVisibleChangebefore delayedroute_changechangedwith unchanged frame0.4.0→0.4.1Review follow-ups
Addressed Copilot comments:
_delaypreservesFuture.delayed(Duration.zero)yielding (and always routes throughdebugDelaywhen set)debugExecuteCapturepath refreshes state + emits inventory like production captureTest plan
flutter test test/replay_coherence_characterization_test.dartflutter test test/sdk_version_test.dart test/replay_coherence_characterization_test.dart test/tugboat_replay_test.dartFollow-ups: #6–#10 can flip the
CoherenceInvariantsexpectations toisTrueas behavioral fixes land.Fixes #5.