Skip to content

fix: checkpoint WebKit LocalStorage WAL on startup to prevent unbounded growth#662

Open
kckylechen1 wants to merge 1 commit intojlcodes99:mainfrom
kckylechen1:fix/webkit-localstorage-wal-bloat
Open

fix: checkpoint WebKit LocalStorage WAL on startup to prevent unbounded growth#662
kckylechen1 wants to merge 1 commit intojlcodes99:mainfrom
kckylechen1:fix/webkit-localstorage-wal-bloat

Conversation

@kckylechen1
Copy link
Copy Markdown

Problem

WebKit WKWebView uses WAL mode for LocalStorage SQLite databases but never checkpoints them. When the app writes large blobs frequently (e.g. Windsurf/Codex account caches with quota snapshots, written every few minutes via auto-refresh), the WAL file grows unbounded.

Observed on my machine: 13 GB WAL file for only ~5 MB of actual data (45 key-value pairs).

~/Library/WebKit/com.jlcodes.cockpit-tools/WebsiteData/.../LocalStorage/
  localstorage.sqlite3     5.0 MB   (actual data)
  localstorage.sqlite3-wal  13 GB   (should be ~0)
  localstorage.sqlite3-shm  27 MB

After running PRAGMA wal_checkpoint(TRUNCATE) manually, the WAL dropped to 0 bytes.

Root Cause

The frontend writes full JSON-serialized account arrays to localStorage on:

  • Every page navigation/mount
  • Every auto-refresh cycle (configurable, ~5-30 min per platform)
  • Every user action (switch account, refresh token, etc.)

11 platform account stores + 11 instance stores = 22 caches being rewritten continuously. The heaviest key (agtools.windsurf.accounts.cache at ~5 MB) includes raw API response blobs (copilot_quota_snapshots, windsurf_user_status, etc.) with no size reduction.

WKWebView doesn't auto-checkpoint its SQLite databases, so the WAL accumulates indefinitely.

Fix

Add a startup maintenance step (modules/webkit_cache_maintenance.rs) that:

  1. Locates all localstorage.sqlite3 files under the WebKit data directory
  2. Runs PRAGMA wal_checkpoint(TRUNCATE) on each
  3. Runs in a background thread to avoid blocking startup
  4. Logs success/failure for observability

Runs at startup in lib.rs setup, before the WebView is fully loaded — so the lock is available.

Testing

  • Compiled successfully with cargo check (no new warnings)
  • Manual test: WAL went from 13 GB → 0 bytes after checkpoint

Future Improvements (not in this PR)

  • Add periodic checkpoint during runtime (e.g. every 30 min)
  • Strip large API response blobs before persisting to localStorage
  • Add size-based eviction for old cache entries
  • Consider moving heavy caches to Tauri's Rust-side SQLite instead of WebKit LocalStorage

…ed growth

WebKit WKWebView uses WAL mode for LocalStorage SQLite databases
but never checkpoints them. When the app writes large blobs
frequently (e.g. Windsurf account caches with quota snapshots),
the WAL file grows unbounded — observed at 13 GB for only ~5 MB
of actual data.

Add a startup maintenance step that:
- Locates all LocalStorage SQLite databases under the WebKit data directory
- Runs PRAGMA wal_checkpoint(TRUNCATE) on each
- Runs in a background thread to avoid blocking startup
- Logs success/failure for observability
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant