feat: Plugin Testsuite as Developer CI Workflow by dirkwa · Pull Request #2400 · SignalK/signalk-server

dirkwa · 2026-03-03T20:49:38Z

Summary

Adds a reusable GitHub Actions workflow that plugin developers can adopt with a single file. Plugins break silently when signalk-server updates — this gives every plugin cross-platform CI, even those without a test suite.

Quick start for plugin developers

Create .github/workflows/signalk-ci.yml:

name: SignalK Plugin CI on: push: branches: [main, master] pull_request: branches: [main, master]

jobs: test: uses: SignalK/signalk-server/.github/workflows/plugin-ci.yml@master

That's it — one file, full cross-platform CI.

Platforms

Platform	Architecture	Node	Notes
Linux	x64	22, 24	GitHub-hosted runner
macOS	arm64	22, 24	GitHub-hosted runner
Windows	x64	22, 24	GitHub-hosted runner
Linux	armv7 (Cerbo GX)	20	QEMU emulation

## Summary

Adds a reusable GitHub Actions workflow that plugin developers can adopt with a single file. Plugins break silently when signalk-server updates — this gives every plugin cross-platform CI, even those without a test suite.

Quick start for plugin developers

Create .github/workflows/signalk-ci.yml:

name: SignalK Plugin CI

on:
  push:
    branches: [main, master]
  pull_request:
    branches: [main, master]

jobs:
  test:
    uses: SignalK/signalk-server/.github/workflows/plugin-ci.yml@master

That's it — one file, full cross-platform CI.

Platforms

Platform	Architecture	Node	Notes
Linux	x64	22, 24	GitHub-hosted runner
macOS	arm64	22, 24	GitHub-hosted runner
Windows	x64	22, 24	GitHub-hosted runner
Linux	armv7 (Cerbo GX)	20	QEMU emulation

Why armv7 is pinned to Node 20

The Victron Cerbo GX runs Venus OS 3.70 which ships Node 20.18.2. The CI emulates this exact environment using QEMU with a node:20-bookworm-slim Docker image plus python3, make, and g++ — matching what's available on the Cerbo via opkg. The Node version is fixed and not user-configurable to keep the test environment as close to real hardware as possible. armv7 jobs are advisory and non-blocking — failures are reported but don't fail the overall CI, since most plugin developers target desktop platforms first.

Validation checks (desktop jobs)

Every desktop job runs these checks, even for plugins without a test suite:

package.json — signalk-node-server-plugin keyword, main or exports field, engines.node declaration
Entry point — plugin exports a constructor function
plugin.schema() — returns valid JSON Schema without crashing
Lifecycle — start() → stop() → start() (restart) with empty config; validates delta messages and registerDeltaInputHandler forwarding
API usage — scans for deprecated APIs (setProviderStatus), internal server properties (app.server, app.deltaCache), route anti-patterns (direct app.get() instead of registerWithRouter()), file storage anti-patterns (__dirname instead of app.getDataDirPath()), security anti-patterns (app.securityStrategy), and Node built-in version mismatches (node:sqlite without engines.node >= 22.5.0)
npm pack — all files referenced by main/exports are included in the published package
App Store compatibility — installs with --ignore-scripts (as the App Store does) and checks for native addon dependencies
Stray files — build/test doesn't leave untracked files in the repository

Integration tests (opt-in)

Enable with enable-signalk-integration: true to install a Signal K server, pack and install the plugin, auto-configure it, start the server with sample NMEA data, and verify the plugin loads. Runs npm run test:integration if defined.

Configuration

Input	Default	Description
`test-command`	`npm test`	Command to run test suite
`build-command`	`npm run build --if-present`	Build command
`node-versions`	`["22", "24"]`	Node versions for desktop platforms
`enable-armv7`	`true`	Test on armv7 (Cerbo GX) via QEMU
`enable-signalk-integration`	`false`	Start SignalK server for integration tests
`signalk-server-version`	`latest`	SignalK server version to test against

Plugins without a test script get all validation checks — tests are skipped with a notice.

Tested with

nfl-signalk — TypeScript plugin with jest tests and integration test
Kip — Angular monorepo with custom build/test commands
freeboard-sk — Angular webapp plugin
bt-sensors-plugin-sk — Webpack build, native deps, no test suite, embedded webapp

Files

File	Description
`.github/workflows/plugin-ci.yml`	Reusable workflow (the main deliverable)
`.github/examples/plugin-caller-example.yml`	Copy-paste example for plugin repos
`.github/examples/standalone-ci.yml`	Self-contained version for repos that prefer no external dependency
`docs/develop/plugins/ci.md`	Documentation for the docs site

dirkwa · 2026-03-06T03:43:03Z

Here a sample of plugin supporting the full integration test (full featured):
https://github.com/noforeignland/nfl-signalk/actions/runs/22747980095

dirkwa · 2026-03-06T04:02:00Z

Freeboard-SK - PASS

https://github.com/dirkwa/freeboard-sk/actions/runs/22748268277

With the following waring:

No "engines.node" field in package.json — plugins should declare their minimum Node.js version

KIP - FAILED

https://github.com/dirkwa/Kip/actions/runs/22748279814

Warning:

plugin/index.js:467 — registerHistoryProvider() is deprecated — use registerHistoryApiProvider() instead (see PR #2381)

No "engines.node" field in package.json — plugins should declare their minimum Node.js version

Error:

plugin/index.js — uses node:sqlite (requires Node >= 22.5.0) but package.json has no engines.node field. Add engines.node to declare the minimum version.

tkurki · 2026-03-06T05:27:38Z

The logic here is too aggressive and partly faulty, don’t start flagging plugins based on this.

But I will guard my time and won’t address this PR until others that are higher prio are done.

dirkwa · 2026-03-06T05:46:55Z

No problem, there are plenty of PRs in the queue.

This tool surfaced in my Plugins weak points I was not aware of.

Maintained wisely I am more than certain it will be a great addon to help Plugin developers and stabilize the SignalK ecosystem.

dirkwa · 2026-03-14T12:28:41Z

Test CI detects Plugin crashes Server

https://github.com/oyve/signalk-barometer-trend
causes

Mar 14 14:15:15 Uncaught exception: TypeError: Cannot read properties of null (reading 'tendency') at Object.getPredictions (/home/pi/.signalk/node_modules/barometer-trend/index.js:103:78) at onPressureUpdated (/home/pi/.signalk/node_modules/signalk-barometer-trend/barometer.js:131:31) at Object.handle (/home/pi/.signalk/node_modules/signalk-barometer-trend/barometer.js:41:84) at /home/pi/.signalk/node_modules/signalk-barometer-trend/barometer.js:84:46 at Array.forEach (<anonymous>) at /home/pi/.signalk/node_modules/signalk-barometer-trend/barometer.js:80:18 at Array.forEach (<anonymous>) at Object.onDeltasUpdate (/home/pi/.signalk/node_modules/signalk-barometer-trend/barometer.js:76:20) at /home/pi/.signalk/node_modules/signalk-barometer-trend/index.js:46:42 at /usr/lib/node_modules/signalk-server/node_modules/baconjs/dist/Bacon.js:1048:16 at /usr/lib/node_modules/signalk-server/node_modules/baconjs/dist/Bacon.js:463:23 at processAfters (/usr/lib/node_modules/signalk-server/node_modules/baconjs/dist/Bacon.js:359:11) at Object.inTransaction (/usr/lib/node_modules/signalk-server/node_modules/baconjs/dist/Bacon.js:439:9) at Dispatcher.push (/usr/lib/node_modules/signalk-server/node_modules/baconjs/dist/Bacon.js:1214:24) at buffer.push (/usr/lib/node_modules/signalk-server/node_modules/baconjs/dist/Bacon.js:2030:21) at Object.flush (/usr/lib/node_modules/signalk-server/node_modules/baconjs/dist/Bacon.js:1997:26) at Timeout.<anonymous> (/usr/lib/node_modules/signalk-server/node_modules/baconjs/dist/Bacon.js:2014:25) at listOnTimeout (node:internal/timers:585:17) at process.processTimers (node:internal/timers:521:7)
Mar 14 14:15:15 UNCAUGHT_EXCEPTION received. Shutting down gracefully...
Mar 14 14:15:16 Shutdown complete

TestCI Run: https://github.com/dirkwa/signalk-barometer-trend/actions/runs/23087891542

dirkwa · 2026-03-16T19:03:50Z

UPDATE - Done in 6153182

TODO: Verify semver - #2455

Sample:

https://www.npmjs.com/package/stingray-signalk/v/0.1.1?activeTab=code

Reusable GitHub Actions workflow that plugin developers can adopt with a single file in their repo. Tests plugins across Linux x64, Linux arm64 (Raspberry Pi), macOS, Windows (Node 22 + 24) and armv7/Cerbo GX (Node 20 via QEMU). Validation checks run even for plugins without a test suite: - package.json structure (keyword, main/exports, engines.node) - Entry point exports a constructor function - plugin.schema() returns valid JSON Schema - Lifecycle: start/stop/restart with empty config, delta validation - API usage scan (deprecated APIs, internal properties, route and file storage anti-patterns, security anti-patterns, node: built-in module version mismatches) - npm pack includes all required files - App Store compatibility (no native addons with --ignore-scripts) - No stray files left after build/test armv7 emulates Venus OS 3.70 on Cerbo GX using QEMU with node:20-bookworm-slim plus python3, make, and g++. The armv7 Node version is fixed to match the Cerbo and is not user-configurable. armv7 failures are advisory and non-blocking. Optional integration test installs a Signal K server, packs and installs the plugin, auto-configures it, starts the server with sample NMEA data, and verifies the plugin loads.

- plugin-caller-example.yml: copy-paste one-liner that calls the shared reusable workflow - standalone-ci.yml: self-contained version with all the same checks for repos that prefer no external workflow dependency

Documents the reusable workflow, platform matrix, validation checks, armv7/Cerbo GX emulation, integration tests, configuration options, and self-hosted runner setup.

Start the server with --sample-nmea0183-data --sample-n2k-data --override-timestamps so plugins have a rich data environment (navigation, wind, depth, temperature, battery, etc.) instead of just NMEA 0183 position/heading data.

…ational notice registerHistoryProvider() is the v1 history API (playback/snapshots), coexisting with registerHistoryApiProvider() (v2 time-series). The CI incorrectly flagged v1 as deprecated. Change to a ::notice:: that points plugin authors to v2 without implying v1 is wrong.

npm normalizes invalid versions (e.g. '0.1.01' → '0.1.1') in its registry metadata, but the installed package.json retains the original invalid string. This breaks semver.gt() in the appstore, crashing the entire plugin listing. Catch this before publishing by validating the version field against the semver spec regex.

- Add missing ServerAPI methods to lifecycle mock (readPluginOptions, savePluginOptions, getDataDirPath, selfContext, and other commonly used methods) - Fix schema check to handle both object and function forms, as both are valid per the Plugin interface - Add registerWithRouter() call before start() to match real server plugin loading sequence - Improve error messages to distinguish CI mock gaps from real plugin bugs

Remove standalone-ci.yml and its documentation references — the reusable workflow is the single source of truth, avoiding dual maintenance. Upgrade actions/checkout v4→v6, actions/setup-node v4→v6, and actions/upload-artifact v4→v7 to eliminate Node.js 20 deprecation warnings.

…3→v4 Eliminates remaining Node.js 20 deprecation warning on the armv7 job.

…check Native addons marked as optional in the lockfile (e.g. cpu-features from ssh2) now produce a warning instead of failing the build. The parent package declares them optional and is expected to fall back gracefully. Required native addons remain a hard error.

Each ::warning:: and ::error:: line becomes a separate annotation in the GitHub UI. Merge multi-line messages into single annotations for cleaner output.

dirkwa force-pushed the feat-ci-for-plugins branch from 0996aba to 4cff9e4 Compare March 5, 2026 17:40

dirkwa marked this pull request as draft March 5, 2026 19:50

dirkwa force-pushed the feat-ci-for-plugins branch 2 times, most recently from bdec939 to 8467b29 Compare March 6, 2026 02:56

dirkwa marked this pull request as ready for review March 6, 2026 02:57

dirkwa force-pushed the feat-ci-for-plugins branch 2 times, most recently from ac22586 to d30a179 Compare March 6, 2026 03:23

dirkwa changed the title ~~draft: Plugin Developer CI Workflow~~ Plugin Testsuite as Developer CI Workflow Mar 6, 2026

dirkwa changed the title ~~Plugin Testsuite as Developer CI Workflow~~ feat: Plugin Testsuite as Developer CI Workflow Mar 6, 2026

dirkwa added 7 commits March 17, 2026 07:49

feat(ci): add example workflows for plugin developers

2685b6b

- plugin-caller-example.yml: copy-paste one-liner that calls the shared reusable workflow - standalone-ci.yml: self-contained version with all the same checks for repos that prefer no external workflow dependency

docs(ci): add .github README for CI workflows and examples

c9df1ed

docs(ci): add plugin CI documentation to docs site

76683da

Documents the reusable workflow, platform matrix, validation checks, armv7/Cerbo GX emulation, integration tests, configuration options, and self-hosted runner setup.

dirkwa force-pushed the feat-ci-for-plugins branch from 957076b to 6153182 Compare March 16, 2026 19:51

dirkwa added 5 commits March 18, 2026 10:14

chore(ci): upgrade actions/cache v4→v5 and docker/setup-qemu-action v…

31b104e

…3→v4 Eliminates remaining Node.js 20 deprecation warning on the armv7 job.

fix(ci): consolidate native addon warnings into single annotations

eb070e2

Each ::warning:: and ::error:: line becomes a separate annotation in the GitHub UI. Merge multi-line messages into single annotations for cleaner output.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Plugin Testsuite as Developer CI Workflow#2400

feat: Plugin Testsuite as Developer CI Workflow#2400
dirkwa wants to merge 12 commits intoSignalK:masterfrom
dirkwa:feat-ci-for-plugins

dirkwa commented Mar 3, 2026 •

edited

Loading

Uh oh!

dirkwa commented Mar 6, 2026

Uh oh!

dirkwa commented Mar 6, 2026

Uh oh!

tkurki commented Mar 6, 2026

Uh oh!

dirkwa commented Mar 6, 2026 •

edited

Loading

Uh oh!

dirkwa commented Mar 14, 2026

Uh oh!

dirkwa commented Mar 16, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

dirkwa commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Quick start for plugin developers

Platforms

Quick start for plugin developers

Platforms

Why armv7 is pinned to Node 20

Validation checks (desktop jobs)

Integration tests (opt-in)

Configuration

Tested with

Files

Uh oh!

dirkwa commented Mar 6, 2026

Uh oh!

dirkwa commented Mar 6, 2026

Freeboard-SK - PASS

KIP - FAILED

Uh oh!

tkurki commented Mar 6, 2026

Uh oh!

dirkwa commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dirkwa commented Mar 14, 2026

Test CI detects Plugin crashes Server

Uh oh!

dirkwa commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dirkwa commented Mar 3, 2026 •

edited

Loading

dirkwa commented Mar 6, 2026 •

edited

Loading

dirkwa commented Mar 16, 2026 •

edited

Loading