feat: add server-internal database storage (skserver.sqlite)#2424
Open
dirkwa wants to merge 7 commits intoSignalK:masterfrom
Open
feat: add server-internal database storage (skserver.sqlite)#2424dirkwa wants to merge 7 commits intoSignalK:masterfrom
dirkwa wants to merge 7 commits intoSignalK:masterfrom
Conversation
Introduce a server-managed relational database API that gives each plugin its own isolated SQLite database. The API supports migrations, parameterized queries, transactions, and pluggable provider backends. Two built-in providers ship with the server: - _builtin: SQLite via better-sqlite3 (default) - _builtin_nodesqlite: SQLite via node:sqlite (Node >= 22.5.0) Plugins can register alternative providers (e.g. PostgreSQL) via app.registerDatabaseProvider() and consumers access the API via app.getDatabaseApi().getPluginDb(plugin.id).
On Node >= 22.5.0, use the built-in node:sqlite module as the default database provider. This eliminates the native addon dependency for most users. On Node 20 (e.g. Victron Cerbo), better-sqlite3 remains the default since node:sqlite is not available. Both providers use lazy require() with try/catch guards so the server boots cleanly regardless of what is available. Plugin providers now correctly override either built-in default.
Align with the naming convention proposed in SignalK#2417: Provider = what plugins implement, Api = what plugins consume. The registry interface follows the <D>ProviderRegistry pattern consistent with WeatherProviderRegistry, RadarProviderRegistry, etc.
Add consumer-facing documentation (database_api.md) covering plugin usage, migrations, queries, transactions, and provider management REST endpoints. Add provider plugin documentation (database_provider_plugins.md) with interface requirements, registration example, and SQL portability notes. Update JSDoc in databaseapi.ts to reflect node:sqlite preference order.
Extend the Database API with server-internal storage (skserver.sqlite) for operational data like token activity and access request audit trails. - Add optional getServerDb() to DatabaseProvider interface - Implement in both built-in providers (node:sqlite, better-sqlite3) - Wire up in registry with fallback to built-in if the active community provider doesn't implement it - Not exposed to plugins — server-internal only
Contributor
Author
|
Further use cases: In #2379:
In BLE Provider API https://discord.com/channels/1170433917761892493/1478948783630057596:
|
Contributor
Author
|
Also worth looking into: @signalk/resources-provider
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Builds on #2409 (Database API). Adds
getServerDb()so the server itself can persist operational data to a dedicatedskserver.sqlitefile — separate from plugin databases.Motivated by the discussion in #2369 (review comment): tracking device token "last seen" timestamps needs persistence, and writing high-frequency data to
security.jsonisn't the right tool.getServerDb()added as optional method onDatabaseProvider— existing community providers won't breakDatabaseApiinterfaceskserver.sqliteis only created when a consumer first callsgetServerDb()PluginDbhandle (migrate, query, run, transaction)File layout
What this enables (future PRs)
Tests
Three new tests covering the key scenarios:
getServerDb(), verifies the registry falls back to the built-in provider and the server database works (migrate, insert, file exists)getPluginDb+closesatisfiesDatabaseProvider(nogetServerDbrequired)Tested
skserver.sqlitecreated when no consumer calls itDepends on: #2409