Concurrent D1 migration executors can collide on migration DDL before bookkeeping race recovery

[yumam0815]

Summary

Two deployment migration executors started concurrently against the same completely empty remote D1 database can collide in migration DDL before either executor records the migration in _emdash_migrations.

This is distinct from #2755 (empty-D1 HTTP 400 error handling) and #954 (recovery from an already partial migration state).

Versions

• EmDash main baseline: 6c912b50159f37bcbfbd5eea9a34182ce609c36b
• Empty-D1 fix candidate used for the reproduction: 1bdf24f87337f8aa8f1ed4a6042949ac48fb18c3
• Core migration identity: EmDash 0.35.0, 69 migrations
• Remote database: a new, empty, disposable Cloudflare D1 database

Reproduction

1. Confirm the remote D1 database has no application tables and no migration history.
2. Start two independent deployment migration executors at the same time with the same account/database target and exact migration fingerprint.
3. Let both run the normal migration apply path.

Actual result

• One executor completed all 69 migrations.
• The other executor failed during 004_plugins with a sanitized D1 error equivalent to:

table _plugin_storage already exists (migration: 004_plugins)

The final ledger can appear complete because the successful executor finishes, but the losing executor may already have executed part of a multi-statement migration before the error.

The current race recovery recognizes a migration-history UNIQUE conflict after migration work. It does not prevent both executors from entering the migration body, so a DDL collision can occur before bookkeeping recovery is reached.

Kysely's SQLite migration-lock path assumes a connection-level serialization model and does not provide a cross-process lock for independent D1 REST executors.

Expected behavior

Concurrent deployment migration attempts for the same D1 target should not both enter migration DDL. Either:

• the adapter/runner provides a real cross-executor migration mutex; or
• the deployment interface requires and clearly documents an external single-owner coordinator and fails closed when ownership is unavailable.

A broad catch for table already exists is not safe because the losing executor may be observing a partial schema rather than a completed migration.

Current workaround

Use an external atomic lease scoped to a hash of Cloudflare account ID plus D1 UUID. Bind the lease to the exact migration artifact and fingerprint, allow only one migration owner, fail closed if the coordinator is unavailable, and require explicit status/partial-state checks before recovering an expired lease.

No production deployment or confirmed data loss is part of this report.

[danielmlr]

@ascorbic, which do you prefer for 1.0: a migration lock for D1, or the rule from #2418 that deployments serialize their own migration jobs? The CLI warning and the core migrations guide state that rule, but nothing enforces it: a local emdash migrate during a CI run can still collide.

The Worker binding and the REST executor both use D1Adapter, so a lock there covers both. The first run to claim Kysely's lock row holds it; the loser gets the Postgres lock's busy error and lands in the runner's existing wait instead of in a migration.

Taking over an expired lock reruns the interrupted migration. Every migration written since the guarded-DDL pattern survives running twice in the D1 tests; the older ones aren't covered, but in practice only an empty database still runs them. Without takeover, a site in auto mode would fail requests until someone clears the lock by hand, so I'd allow it.

If you want the lock, I can build it with D1 tests.