feat: Source priority ux

[dirkwa]

Summary

This PR reworks how Signal K Server handles multiple data sources. On any boat with more than one GPS, a mix of NMEA 2000 and NMEA 0183 devices, or plugins that calculate derived values, the server receives the same data path from several sources. Until now, managing this was difficult: sources were identified by unstable addresses, lower-priority data was silently discarded, and there was no way to see what was actually happening on the bus.

This started from issue #2162 and the discussion with @tkurki, @macjl, and @KEGustafsson. It implements all seven phases from the revised plan, plus N2K device management that came up along the way.

What changed

Stable source identification

NMEA 2000 sources are now identified by their CAN Name — a globally unique identifier from the device's ISO Address Claim — instead of the source address. A Furuno SCX-20 shows up as can0.Furuno_SCX-20 regardless of what address it happens to claim on the bus. No more broken priority configurations after adding or removing a device.

For NMEA 0183, a new talker group configuration lets you tell the server that talker IDs GP, GL, and GN on serial0 are all the same GPS. Without this, a multi-constellation receiver looks like three separate sources.

All source data is preserved

Previously, toPreferredDelta() stripped non-preferred values from deltas before they entered the delta chain. Lower-priority sensors were invisible — the server, the UI, and all clients had no idea they existed.

Now all source data flows into the delta cache and data model. Priority filtering moved to the subscription layer: WebSocket clients and plugins receive preferred-source data by default, but can opt in to everything with sourcePolicy=all. This means you can actually see what all your sensors are reporting, compare readings, and debug issues — while the default behavior still delivers clean, prioritized data.

Source ranking

A new global source ranking lets you say "always prefer GPS A over GPS B for everything they both provide." This covers the common case without configuring every path individually. Path-level overrides still exist for the exceptions. Sources can be disabled entirely, and each fallback source has a configurable timeout.

N2K device discovery and instance management

The Source Discovery page shows every NMEA 2000 device on the bus in a sortable table with manufacturer, model, serial number, firmware version, and device class. You can edit device instances, data instances, battery instances (PGN 127508), DC instances (PGN 127506), and installation descriptions — directly from the browser, without needing an Actisense NGT-1 or dedicated PC software.

The server automatically detects instance conflicts — two devices with the same instance transmitting overlapping PGNs — and shows them as warnings with one-click filtering to the conflicting pair. This is a real time-saver on boats with multiple Victron MPPTs or similar devices that all default to instance 0.

Reorganized Data menu

The Data menu now has dedicated pages:

• Data Browser — simplified, faster, no more virtual table overhead. Includes a source-grouped view and a source priority toggle for comparing filtered vs unfiltered data.
• Source Priority — source ranking and path-level overrides, with a sidebar badge showing how many multi-source paths still need configuration.
• Source Discovery — the N2K device table described above.
• Meta Data — extracted from the old Data Browser into its own page.

The Data Browser dropped its virtual table implementation. The previous approach added complexity without meaningful benefit for the data volumes involved. The WebSocket subscription model was also reworked — instead of subscribing to everything and filtering client-side, it uses granular subscriptions with announceNewPaths to discover available paths while only receiving continuous updates for visible ones.

Developer API

Plugins can use sourcePolicy: 'all' in their subscription options to receive data from every source. WebSocket clients can pass ?sourcePolicy=all as a query parameter. The default remains preferred so existing clients are unaffected.

Breaking changes

Two things that existing users may need to update:

1. N2K $source format changed — from can0.22 to can0.Furuno_SCX-20. Existing sourcePriorities entries in settings.json that reference old-style addresses will no longer match. The Source Priority UI makes it easy to reconfigure.

2. All source data is now visible in the data model — plugins and clients that query the REST API or data model directly may see additional sources they didn't see before. WebSocket subscriptions are not affected unless sourcePolicy=all is explicitly requested.

Both are documented in docs/breaking_changes.md.

Not in this PR

Two items from the original discussion are natural follow-ups but separate in scope:

• SensESP / WebSocket sensor identification — sensors connecting via WebSocket could benefit from richer identification similar to what N2K devices get from PGN 60928. This needs input from the SensESP side on what metadata to expose.
• Aggregate data — computed values like min/max/rolling averages from source data. This needs a design decision on how aggregates identify themselves as sources and how they interact with the priority system.

Tests

Manual tests

• Verified every individual function manually with a N2k bus, 0183 and plugins emitting delta path.

Automatic Tests

Six test files with 1100+ new lines covering the priority engine (ranking, timeouts, disabled sources, fallback), delta cache (multi-source tracking, source removal, preferred source selection), NMEA 0183 talker groups, WebSocket sourcePolicy integration, source label resolution, and priorities store actions.

All 141 existing tests continue to pass.

Images

See #2162 (comment)*

Fixes: #2162
Fixes: #1916
Fixes: #1885
Fixes: #1768 - Workflow superseeded
Fixes: #1754
Fixes: #1747
Fixes: #1571
Fixes: #1555
Fixes: #1369
Fixes: #1327
Fixes: #1272
Fixes: #1258
Fixes: #1183
Fixes: #2474

[tkurki]

This PR is getting to the "too many things to digest at once" category. Would it make sense to split it up:

• N2K source changes
• NMEA0183 source related changes
• UI reorg
• priority logic related changes

I am open to other ways to split it up.

The proposed change to N2K sources misinterprets my comment about CAN Name, that refers to the persistent unique ID of the N2K device that we already support. We just need to switch using that as the primary ID and make N2K source related metadata via that. Also wouldn't the device name in sourceRef make it impossible to have >1 same devices?

Also there's now confusion about naming: what is priority and what is ranking?

[tkurki]

Hmmm, maybe it was me misunderstandding the N2K CAN name related comments, screenshots show the real thing.

[dirkwa]

I am open to other ways to split it up.

All of it plays hand in hand and has dependencies. I spent > 16 hours in the QA already. Splitting this would mean this effort by 3-4 times again. Now for each individul PR in every possible configurtion.

I am not really motivated just for a nice split in the PR history to go trough this.

[dirkwa]

This adds needed dedup and cleaning function when devices are broken or replaced.
I forgot to implement it.

Also the user can see now what former known devices are offline to troubleshoot his N2k install.

[dirkwa]

Note

For the Victron MPPT changes I used:

canboat/canboatjs#396

[dirkwa]

I have some very interesting beta tester feedback and want to analyze this first.
PR back to draft.

[dirkwa]

1. Rebased source-priority-ux on current upstream master (3 new commits: unitprefs docs, dep cleanup, Simple Line Icons) — no conflicts.

2. Simplified n2k-discovery.ts by replacing hardcoded Maretron/PGN detection with upstream newly added @canboat/ts-pgns APIs:

   • 40 lines of hardcoded PGN sets (21 PGNs in INSTANCE_FIELD_PGNS, 4 in DATA_INSTANCE_PGNS) → derived at module load from getAllPGNs() using PartOfPrimaryKey and LookupEnumeration field metadata
   • 30 lines of hardcoded enum maps (TEMPERATURE_SOURCE, HUMIDITY_SOURCE + reverse lookups) → replaced with getEnumerationName() / getEnumerationValue() calls
   • Net: -78 lines, +33 lines (1 file changed)
   • Auto future-proofing: new PGNs added to canboat are picked up automatically

[dirkwa]

Note:

Currently ts-pgns barrel-exports everything via index.js — ~1.6 MB raw.

Once PR canboat/ts-pgns#44 is merged and released this will be reduced to — ~0.115 MB raw.

Note on @canboat/ts-pgns size: The 1.6 MB barrel export is a server-side dependency only — it does not affect the admin UI bundle that ships to the browser. This is a pre-existing issue on master, not introduced by this PR. The tree-shaking improvement in canboat/ts-pgns#44 would reduce server startup memory, but is not a blocker for this work.

[dirkwa]

Way too much for this PR - must be discussed separately if still relevant.

• Send notifications on data from lower priority sources #1182