Add SQLite support, bring parity with dbos-transact-py#1288
Conversation
There was a problem hiding this comment.
Thanks for submitting this! It's a substantial contribution, so review may take some time. I'm frankly a bit concerned by a large LLM-generated dialect translator, and while it's not necessarily a blocker, we'll have to be careful with it. Two things we need to see before looking at this further:
- Like in Python, we need all (or almost all) unit tests to run against both SQLite and Postgres.
- Instead of bringing in a large new dependency with many transitive dependencies, is it possible to build this on Node's built-in sqlite? This may mean gating it behind more recent versions of Node, we'd have to make sure that's possible without breaking DBOS on older Node versions (DBOS supports Node >20, SQLite was added in 22 and stabilized in 26). This may not be possible--I'm not very familiar with the Node SQLite API.
|
@kraftp I will research and learn further on the native Gating behind only recent versions of node is a good point, I didn't consider and will come back with. |
|
Changes to depend solely on |
There was a problem hiding this comment.
I imagine maybe was a but presumptious to go ahead and bump the postgres:image but made sense in my head. The $PGDATA path change may not be optimal per the CI runners. Can't exactly verify, or perhaps should. May make sense to revert change on line 61
This change adds SQLite-backed system database support for local development and tests.
When no system database URL is provided, DBOS defaults to a per-app SQLite file. Explicit
Postgres
systemDatabaseUrlvalues are preserved and continue to use the existingPostgres path.
SQLite support is additive. Postgres remains the recommended production system database.
Intent is to help with #1226
Brief implementation description
better-sqlite3system database adapter with SQLite URL handling, schema setupand reset, WAL,
busy_timeout, foreign keys, and serialized logical client leasing.ANY(...), schema prefixes, CTE writes, UUID/time expressions, booleans, andRETURNINGvariants.changing the explicit Postgres configuration path.
shutdown behavior, and explicit Postgres
systemDatabaseUrlpreservation.Testing performed
node_modules/.bin/prettier --write src/sqlite_system_database.ts src/system_database.ts tests/sqlite_system_database.test.tsnpm run lintnpm run buildnode_modules/.bin/jest tests/config.test.ts tests/sqlite_system_database.test.ts --runInBandnpx tsc --project tsconfig.json --noEmitgit diff --checkAfter the final cleanup to
src/system_database.ts, the focused verification run was:node_modules/.bin/prettier --write src/system_database.tsnpx tsc --project tsconfig.json --noEmitgit diff --checkDevelopment note
Implemented with assistance from Codex 5.5 xhigh.