Skip to content

Preserve migration state during multi-account catch-up - #382

Draft
piatoss3612 wants to merge 3 commits into
mainfrom
rowan/fix-migration-account-sync
Draft

Preserve migration state during multi-account catch-up#382
piatoss3612 wants to merge 3 commits into
mainfrom
rowan/fix-migration-account-sync

Conversation

@piatoss3612

@piatoss3612 piatoss3612 commented Jul 29, 2026

Copy link
Copy Markdown
Member

Problem

Vizor stores every Zcash account in one SQLite wallet DB and synchronizes all accounts through one wallet-wide sync loop.

That creates a race when an Ironwood migration is already running and the user imports or creates another account:

Account A migration is running
  → Account B is imported
  → wallet sync adds historical ranges and restarts from an old height
  → an already-running migration task finishes
  → it commits using the temporary catch-up height

The migration task may have passed its catch-up check before the import, then finish after catch-up has started. In the reproduced failure, Account B began syncing from height 0 while Account A's durable migration state regressed from nextActionHeight=1204 to 153.

This is a shared-state race, not an account-import failure: the new account syncs, but an in-flight migration operation can persist a result calculated from the wrong sync state. The same shared DB also allowed account mutations, sync, sends, and migration writers to contend for SQLite and retain a stale completed-sync snapshot after the account set grew.

Root causes

  • Adding an account did not invalidate every Dart and Rust wallet-wide completion marker at the account-growth boundary.
  • SQLite writers did not share one sufficiently narrow ownership boundary.
  • Migration paths checked account catch-up when an operation started, but an operation that was already running could still commit after the account set changed.
  • Internal Rust retries resumed the persisted scan frontier correctly, but their attempt-local progress scale could restart from zero.

Solution

Invalidate sync completion when the account set grows

  • Clear Dart's completed-sync snapshot when a new account appears.
  • Invalidate Rust's persisted completed-sync height inside the account mutation boundary.
  • Keep an account-growth catch-up marker active while the enlarged wallet scans its historical ranges.
  • Apply this to software, derived, and hardware account additions.

Protect migration state at the commit boundary

  • Coordinate wallet DB writers through a reentrant process-wide lease.
  • Hold the lease only around SQLite snapshot and commit work.
  • Keep encryption, transaction construction, signing, proof generation, and network work outside the lease.
  • Defer migration paths that start while account catch-up is active.
  • Crucially, re-check the catch-up marker inside the final locked transaction before promoting a signed migration child or advancing its retry height.
  • If catch-up began after the operation started, retain the signed child and return without changing durable migration state. The work can resume after catch-up completes.

This closes both sides of the race:

catch-up already active before migration work starts
  → entry guard defers the work

catch-up starts while migration work is already running
  → final transaction guard rejects the stale commit

Preserve progress across internal retries

  • Give each logical run_sync_inner session its own retry-progress mapper.
  • Start an internal retry from the last emitted progress floor instead of publishing a new 0% event.
  • Do not carry that floor into a genuinely new sync session, including the historical sync caused by a new account.

Expected behavior

When another account is imported during migration:

  • the new account performs a real historical catch-up;
  • existing migration schedules, retry heights, and confirmation state do not move backward because of the temporary low scan height;
  • already valid forward confirmation changes may still be persisted;
  • stale in-flight child promotion is retained for a later safe retry;
  • expensive migration and proof work does not hold the wallet-wide DB lease.

Regression coverage

  • Account-growth invalidation for Dart and Rust completion state.
  • Concurrent account mutation and migration DB writes.
  • Simultaneous migration starts using the shared DB.
  • Signed-child promotion that begins before account catch-up but reaches its commit afterward.
  • Internal retry progress and fresh-session progress isolation.
  • macOS regtest flow: start migration for Account A, import Account B, then rescan Account B from height 0 while continuously inspecting Account A's durable migration state.

Validation

Validation was repeated after rebasing onto origin/main at 8191dee08 (release/v0.0.43-rc.0).

  • Migration Rust tests: 155 passed.
  • Sync-engine Rust tests: 23 passed.
  • Focused Flutter provider and mutation-guard tests: 24 passed.
  • Changed Dart files: formatting and analysis passed.
  • git diff --check: passed.
  • macOS migration/account-import regtest E2E: passed.
    • Account B rescanned from rustScanned=0 through 1206.
    • Account A remained in the same migration run and broadcast_scheduled phase.
    • nextActionHeight remained 1204 throughout catch-up instead of regressing to a historical height.
    • All nine scheduled migration children remained accounted for; a valid child confirmation advanced forward during catch-up.

Scope boundaries

  • This does not add a runtime repair for the internal zcash_client_sqlite rc.1 to rc.4 test-DB incompatibility. That RC-only artifact is separate and is not reachable by normal users.
  • Retry progress hardening covers internal retry boundaries. It does not yet guarantee a monotonic percentage when the chain tip itself grows during one attempt.
  • This does not remove librustzcash's future seed-requiring migration limitation for multi-seed wallets.
  • The shared Rust behavior applies to desktop and mobile, but the new full end-to-end evidence is from macOS foreground sync, not the iOS native background-preparation lifecycle.

@piatoss3612 piatoss3612 self-assigned this Jul 29, 2026
@piatoss3612
piatoss3612 force-pushed the rowan/fix-migration-account-sync branch 3 times, most recently from c7a90d8 to 78bab6e Compare July 30, 2026 10:24
@piatoss3612 piatoss3612 changed the title Harden wallet DB locking during account sync and Ironwood migration Stabilize account catch-up, Ironwood migration, and sync retry progress Jul 31, 2026
@piatoss3612 piatoss3612 changed the title Stabilize account catch-up, Ironwood migration, and sync retry progress Preserve migration state during multi-account catch-up Jul 31, 2026
@piatoss3612
piatoss3612 force-pushed the rowan/fix-migration-account-sync branch from fed80c6 to 4dad094 Compare August 4, 2026 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant