Repository navigation
fix(db): make concurrent first-run database setup safe - #49
Merged
Merged
Conversation
`next build` imports route modules from several worker processes at once, and each one opens data/journal.db at import time. On a fresh checkout those processes race to create the file, switch it to WAL and run the bootstrap. SQLite refuses some of those lock upgrades immediately (switching a rollback-journal file to WAL is the common one) without calling the busy handler, so the build failed intermittently with SQLITE_BUSY about half a second into "Collecting page data". Setup now lives in openJournalDatabase() in db/bootstrap.ts: - the WAL switch and the schema bootstrap retry SQLITE_BUSY with a short jittered backoff, up to 15 seconds; - BOOTSTRAP_SQL and every additive column check and ALTER run in one BEGIN IMMEDIATE transaction, so two processes can no longer both see a column as missing and both try to add it. The `db` export and runtime behavior are unchanged; a single process succeeds on the first attempt as before. The new test starts 12 Node processes that load the real bootstrap module and open the same fresh (or legacy rollback-journal) database at the same moment. Without the retries it fails with SQLITE_BUSY in most batches.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the intermittent
SQLITE_BUSY: database is lockedfailure inpnpm buildon CI (example run, seen on #48, which doesn't touch the database).Cause
apps/web/src/db/index.tsopensdata/journal.dbwhen the module is imported. Duringnext build, several worker processes import route modules at the same time, so on a fresh checkout they all create the file, switch it to WAL and run the bootstrap at once. SQLite refuses some of those lock upgrades right away instead of calling the busy handler. Switching a rollback-journal file to WAL is the common case, which is why the error came about 0.5s into "Collecting page data" rather than after the 5s busy timeout.Reproduced locally by running the bootstrap statements from 8 to 16 processes at once against an empty data dir: 12 of 20 rounds failed at 8 processes, and every failure was
PRAGMA journal_mode = WAL.Fix
Setup moves into
openJournalDatabase()indb/bootstrap.ts:SQLITE_BUSYwith a short jittered backoff, up to 15 seconds.BOOTSTRAP_SQLand every additive column check andALTER TABLErun in oneBEGIN IMMEDIATEtransaction. Before, several checks ran outside a transaction, so two processes upgrading the same database could both see a column as missing and both try to add it.The
dbexport and runtime behavior are unchanged, and a single process succeeds on the first attempt as before. I kept the build opening the database rather than skipping setup whenNEXT_PHASE === "phase-production-build". Skipping would leavedbwithout a ready schema at build time, and with the retries the build no longer depends on timing anyway.Test
apps/web/tests/db-concurrent-init.test.tsstarts 12 Node processes that load the real bootstrap module and open the same database at the same moment. It covers three fresh databases and one legacy rollback-journal database that is missing the additive columns. It then checks that the database is in WAL mode, passes the integrity check, has the upgraded columns and the seeded folders. With the retries turned off, 3 of the 4 cases fail withSQLITE_BUSY. With them on, the test passed 6 runs in a row.Checks run locally
pnpm install --frozen-lockfile,pnpm format:check,pnpm typecheck,pnpm test(64 files, 607 tests),node scripts/check-licenses.mjs,pnpm -r --filter "./packages/*" buildandpnpm buildfrom a fresh data dir all pass.