Skip to content

Persist OPFS SQLite databases via snapshots#3539

Draft
ashfame wants to merge 3 commits intotrunkfrom
opfs_sqlite_flush_risk
Draft

Persist OPFS SQLite databases via snapshots#3539
ashfame wants to merge 3 commits intotrunkfrom
opfs_sqlite_flush_risk

Conversation

@ashfame
Copy link
Copy Markdown
Member

@ashfame ashfame commented Apr 28, 2026

⚠️ DON'T REVIEW YET

Draft PR Description

Summary

This PR fixes OPFS persistence for WordPress sites using SQLite by preventing
transient SQLite sidecar files from being saved alongside the persisted
database.

Previously, OPFS persistence mirrored MEMFS writes at the byte level. That meant
the live SQLite database file and its temporary coordination files could be
copied into OPFS as-is, including files like .ht.sqlite-shm. Persisting those
sidecars can leave a saved site with a stale SQLite state on reload.

The fix changes OPFS persistence so the live SQLite DB and sidecars are excluded
from normal journal replay and initial MEMFS-to-OPFS sync. Instead, Playground
creates a SQLite-cooperative snapshot with VACUUM INTO, validates it, and
publishes only the canonical .ht.sqlite file into OPFS.

Commit Structure

The first commit intentionally adds only the regression test:

  1. Add OPFS SQLite sidecar regression test

That commit is expected to fail in CI against the existing implementation. The
test creates a stale .ht.sqlite-shm sidecar before saving a temporary site to
OPFS, then asserts the saved OPFS database directory does not contain that
sidecar. Without the fix, OPFS contains .ht.sqlite-shm and the test fails.

The second commit adds the implementation and supporting unit coverage:

  1. Persist OPFS SQLite databases via snapshots

With the implementation in place, the same regression passes because OPFS stores
only the coherent .ht.sqlite snapshot.

Implementation Details

  • Adds a platform-level SQLite snapshot routine exposed via
    /internal/shared/snapshot-sqlite.php.
  • Uses the SQLite PDO connection to run best-effort
    PRAGMA wal_checkpoint(TRUNCATE), then VACUUM INTO.
  • Validates the snapshot with PRAGMA integrity_check.
  • Excludes .ht.sqlite, .ht.sqlite-journal, .ht.sqlite-wal, and
    .ht.sqlite-shm from OPFS journal replay and initial MEMFS-to-OPFS sync.
  • Adds DirectoryHandleMount.persistSqliteSnapshot() and publishes snapshots
    through a temp file before replacing .ht.sqlite.
  • Wires snapshot persistence into OPFS initial sync, explicit flush, unmount,
    and debounced SQLite DB write handling.

Verification

Regression behavior was verified both ways:

  • With only the regression test commit applied, the Playwright test fails
    because OPFS contains .ht.sqlite-shm.
  • With the fix applied, the same Playwright test passes.

Commands run:

LD_LIBRARY_PATH=/tmp/playwright-deps/usr/lib/x86_64-linux-gnu PLAYWRIGHT_TEST_BASE_URL=http://127.0.0.1:5400/website-server/ npx playwright test --config=packages/playground/website/playwright/playwright.config.ts packages/playground/website/playwright/e2e/opfs-sqlite-flush-risk.spec.ts --project=chromium --retries=0

npm exec nx test playground-remote -- --run src/lib/playground-worker-endpoint.spec.ts --testTimeout=20000
npm exec nx test php-wasm-web -- --run src/lib/directory-handle-mount.spec.ts --testTimeout=20000
npm exec nx run php-wasm-web:typecheck
npm exec nx run playground-remote:typecheck
npm exec nx run playground-website:typecheck
npm exec nx test playground-wordpress
git diff --check

Note: this machine needed Playwright Chromium and local browser shared
libraries. The LD_LIBRARY_PATH prefix points at locally extracted Ubuntu
browser dependency packages used only for this verification environment.

ashfame added 3 commits April 28, 2026 05:57
OPFS persistence currently mirrors the live SQLite database directory byte-for-byte. That can copy transient SQLite sidecars such as .ht.sqlite-shm into saved OPFS sites, leaving restore to reopen a database beside stale coordination files.

Add a Chromium Playwright regression that creates a stale sidecar before saving a temporary site, verifies the saved OPFS database directory only contains the canonical .ht.sqlite database, reloads the site, and checks SQLite integrity plus WordPress loading. This commit is expected to fail without the snapshot persistence fix.
The OPFS mount previously persisted the live SQLite database and sidecars through normal MEMFS journal replay. That can leave saved sites with stale .ht.sqlite-journal, .ht.sqlite-wal, or .ht.sqlite-shm files next to the database on reload.

Add a platform SQLite snapshot primitive that runs VACUUM INTO, validates the result, and lets the worker publish only a coherent .ht.sqlite file into OPFS. Exclude the live database and sidecars from initial sync and journal replay, and wire snapshots into initial persistence, flush, unmount, and debounced DB-write handling.

Add focused unit coverage for sidecar filtering, snapshot persistence, and worker snapshot ordering. Include PR.md with the two-commit rollout and verification notes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant