tools: data_forwarder_host: Add sensor data forwarder host GUI by pioso-dev · Pull Request #112 · nrfconnect/sdk-edge-ai

pioso-dev · 2026-06-19T11:23:06Z

Description

This PR adds data_forwarder_host, a new cross-platform desktop GUI tool that receives, visualises, records and exports sensor data forwarded from an nRF device over UART or BLE NUS (Nordic UART Service). It is the host-side counterpart to the data_forwarder device sample and decodes the device's COBS + CBOR v1 frame format. Multiple capture sessions can run side by side, each in its own tab.

Highlights

Sources: UART (pyserial) and BLE NUS (bleak), with a New Session dialog that scans for and connects to advertising BLE devices.
Out-of-process acquisition: byte reading + COBS/CBOR decode run in a spawn child process so the GUI stays responsive regardless of backend load. The GUI process drains decoded frames over an IPC queue at its own cadence with bounded, consumer-paced back-pressure that never drops control/metadata frames. Set DFH_SINGLE_PROCESS=1 to fall back to the in-GUI path.
Visualisation: combined overlay plot and collapsible per-channel charts (QtCharts), render-time min/max decimation, rolling time window, recording start/stop markers, and live ASCII / decoded-frame log consoles.
Recording: RAM-buffered capture dumped to CSV on stop (written incrementally off the event loop), each paired with a {stem}.txt metadata sidecar (host, transport, timing, device session info, channels, errors). A persistent banner shows the last saved file with open-file / open-folder shortcuts.
Diagnostics: live bandwidth/throughput readout, host CPU/RAM metrics, per-category error & loss analysis with transport-loss confirmation windows, First Failure Data Capture, and an animated pipeline-flow view of stage utilisation/back-pressure.
Architecture: strictly layered (platform -> source -> protocol -> pipeline -> core -> session -> gui). The acquisition/transport layers (platform, source, protocol, pipeline) are Qt-free and run in the headless child process; the data-model, session and GUI layers run in the GUI process and use Qt.

How to try it (recommended)

The easiest, dependency-free way to run and test the app — and the default way to try this PR — is to build the standalone single-file binary:

tools/data_forwarder_host$ ./scripts/build_linux_binary.sh

This provisions an isolated venv and runs PyInstaller, emitting the single file:

dist/data-forwarder-host

Run it directly (no Python or dependencies required on the machine):

tools/data_forwarder_host$ ./dist/data-forwarder-host

How to run (from source)

cd tools/data_forwarder_host
pip install -r requirements.txt
data-forwarder-host        # or: python -m data_forwarder_host / python main.py

Requires Python 3.12+ on Linux, Windows or macOS. PySide6 (Qt 6 with QtCharts) is a base dependency and is always installed. On Linux, UART sources need serial-port access (usually dialout group membership); on Ubuntu/Debian the Qt xcb platform plugin also needs sudo apt install libxcb-cursor0.

The matching device-side firmware is the data_forwarder sample (edge-ai/samples/data_forwarder); without it running there is no data to receive.

Verification

Manually verified UART and BLE NUS capture, multi-session tabs, recording → CSV + metadata sidecar, and GUI responsiveness under flood (out-of-process path) using the scripts/repro_freeze.py harness. Validated on Ubuntu; macOS and Windows are expected to run via bleak/CoreBluetooth/WinRT but are not yet validated.

Notes

All files are new additions; no existing code is modified.
Every file carries the SPDX header LicenseRef-Nordic-5-Clause.

Jira

NCSDK-39472

Szynkaa · 2026-06-23T08:14:08Z

+> |-----|----------|---------------------|
+> | **[Linux]**   | Linux / BlueZ        | ✅ Supported and validated. Most tunable from the host. |
+> | **[macOS]**   | macOS / CoreBluetooth | ✅ Supported. The OS auto-negotiates aggressively; almost nothing is user-tunable. |
+> | **[Windows]** | Windows / WinRT       | ⚠️ **Not yet supported / not validated.** Notes are included so you know what *would* apply, but treat them as untested. |


Not yet supported

is this true? or just the tweaks are not supported

Good catch — reworded. The host app itself is cross-platform (it uses bleak, which has a WinRT backend, and there's a wired-up WindowsPlatform adapter), so it should run on Windows; the accurate statement is that it's not validated and exposes no host-side tuning knobs — not that it's unsupported. Updated the table so only Linux/Ubuntu is marked validated, while macOS and Windows are now "Should run but not validated (untested)." Also changed the §4.2 "(not yet supported)" tag to "(not validated)." Note I downgraded macOS too, since only Ubuntu has actually been tested.

Szynkaa · 2026-06-23T09:13:51Z

+connect so the link does not rely on automatic negotiation timing. (The device
+sample already enables the buffers — `CONFIG_BT_L2CAP_TX_MTU=247`,


Are you referring to new data-forwarder sample here? Then name it

Yes - it's samples/data_forwarder; I've named it explicitly in §5.4. One caveat worth flagging: that sample is currently still a stub and the cited CONFIG_BT_L2CAP_TX_MTU=247 / CONFIG_BT_BUF_ACL_RX_SIZE=251 aren't in its prj.conf yet, so I changed "already enables" -> "should enable." Once the sample ships with those configs I'll switch the wording back to "already enables."

pioso-dev

Comments resolved

pioso-dev · 2026-06-23T12:17:23Z

+> |-----|----------|---------------------|
+> | **[Linux]**   | Linux / BlueZ        | ✅ Supported and validated. Most tunable from the host. |
+> | **[macOS]**   | macOS / CoreBluetooth | ✅ Supported. The OS auto-negotiates aggressively; almost nothing is user-tunable. |
+> | **[Windows]** | Windows / WinRT       | ⚠️ **Not yet supported / not validated.** Notes are included so you know what *would* apply, but treat them as untested. |


Good catch — reworded. The host app itself is cross-platform (it uses bleak, which has a WinRT backend, and there's a wired-up WindowsPlatform adapter), so it should run on Windows; the accurate statement is that it's not validated and exposes no host-side tuning knobs — not that it's unsupported. Updated the table so only Linux/Ubuntu is marked validated, while macOS and Windows are now "Should run but not validated (untested)." Also changed the §4.2 "(not yet supported)" tag to "(not validated)." Note I downgraded macOS too, since only Ubuntu has actually been tested.

pioso-dev · 2026-06-23T12:18:25Z

+connect so the link does not rely on automatic negotiation timing. (The device
+sample already enables the buffers — `CONFIG_BT_L2CAP_TX_MTU=247`,


Yes - it's samples/data_forwarder; I've named it explicitly in §5.4. One caveat worth flagging: that sample is currently still a stub and the cited CONFIG_BT_L2CAP_TX_MTU=247 / CONFIG_BT_BUF_ACL_RX_SIZE=251 aren't in its prj.conf yet, so I changed "already enables" -> "should enable." Once the sample ships with those configs I'll switch the wording back to "already enables."

mbarchie

First round of review - high level code inspection + running without data_forwarder firmware (just starting the host app). I'll continue checking the app when I get a sample of nrf54l15tag.

PR description mentions tests, but I can't find them anywhere in the tree.

One comment about GUI - on dark theme Ubuntu GUI is poorly readable when using dark mode in Ubuntu (dark grey background + mostly black fonts). I guess "automatic color" could fix it if there is such a setting.

mbarchie · 2026-06-23T10:43:58Z

+
+## Architecture (short)
+
+The application is strictly layered; the GUI-free layers carry no Qt imports:


Not exactly clear which layers are GUI-free (ofc besides gui). E.g. some classes in core/ import Qt.

You're absolutely right, thanks — the wording was misleading. core/ does import Qt (data_model, recorder, error_log are QObjects with signals), and so do session/ and one utils/ debug helper. I've reworded the architecture section to make the genuinely Qt-free layers explicit (platform, source, protocol, pipeline — the ones running in the headless child process) and clearly mark the layers that do use Qt

mbarchie · 2026-06-23T10:45:49Z

+- For UART sources: permission to access the serial port (on Linux this usually
+  means membership of the `dialout` group).
+
+## Install


I'd mention that libxcb-cursor0 has to be installed on Ubuntu (not sure about other deps)

I didn't have to install it, but maybe is a dependency of some other software I use

@mbarchie
Good point, thanks. I've added a note in the Install section to:
sudo apt install libxcb-cursor0
on Ubuntu/Debian (with a mention of the other XCB runtime libs for minimal installs), since that's what the xcb platform-plugin error usually needs

mbarchie · 2026-06-23T10:48:00Z

+packages = [
+    "data_forwarder_host",
+    "data_forwarder_host.core",
+    "data_forwarder_host.gui",
+    "data_forwarder_host.gui.dialogs",
+    "data_forwarder_host.gui.widgets",
+    "data_forwarder_host.platform",
+    "data_forwarder_host.protocol",
+    "data_forwarder_host.session",
+    "data_forwarder_host.source",
+    "data_forwarder_host.utils",
+]


data_forwarder_host.pipeline missing?

true - I forgot to add this one

mbarchie · 2026-06-23T12:22:40Z

+class RecordingCsvDump:
+    """Incremental writer for a finished recording's CSV.
+
+    The whole capture is RAM/spill-buffered while recording; at *stop* it must


There's a zero-width space in 'RAM/spill-buffered' part

Nice catch. There was a zero-width space (U+200B) tucked into RAM/spill-buffered — I've removed it

mbarchie · 2026-06-23T12:23:56Z

+    def _run_off_thread(self, fn, on_done, on_failed) -> None:
+        """Run a blocking *fn* on a worker thread; deliver result on GUI thread.
+
+        The result/failure handlers must run on the **GUI thread** (they touch


There's a zero-width space in "reset/failure" part

Same hidden U+200B character here in result/failure — removed. I also did a quick sweep of the tree, and these two were the only spots, so we should be clear now

mbarchie · 2026-06-23T12:29:16Z

+# SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
+"""Framing primitives.
+
+Ported verbatim from edge-ai/tools/data_forwarder_host/python_prototype/


stale reference to edge-ai/tools/data_forwarder_host/python_prototype/

The python_prototype/ tree is gone now, so I dropped the outdated "ported from …/simple_receiver.py" note from the docstring

mbarchie · 2026-06-23T12:29:57Z

+
+The Nordic / SEGGER detection heuristics (``NRF_HINTS``, ``_looks_like_nrf``,
+``_vid_pid``, ``_describe_port``, ``_short_port``) are ported verbatim from
+``edge-ai/tools/data_forwarder_host/python_prototype/simple_receiver/


stale reference to edge-ai/tools/data_forwarder_host/python_prototype/simple_receiver

Good point, thanks. That python_prototype/simple_receiver path no longer exists, so I've removed the stale reference from both the docstring and the inline comment

mbarchie · 2026-06-23T12:43:36Z

Missing reference to the data_forwarder sample (firmware)

Thanks, agreed that was missing. I've added a link to the device-side sample: samples/data_forwarder in both the intro and Requirements, and noted that it advertises over BLE as nRF DataFwd and that there's no data to receive without it running

Add data_forwarder_host, a cross-platform PySide6 desktop GUI that receives, visualises, records and exports sensor data forwarded from an nRF device over UART or BLE NUS, using the device's COBS + CBOR v1 framing. Multiple capture sessions can run side by side in their own tabs. By default, byte reading and frame decoding run in a spawn child process so the GUI stays responsive under load; the GUI drains decoded frames over an IPC queue and never drops control/metadata frames (toggle with DFH_SINGLE_PROCESS). Recordings are RAM-buffered and dumped to CSV on stop, each paired with a metadata sidecar. Live bandwidth, host metrics, error/loss analysis and FFDC are surfaced per tab. Ref: NCSDK-39472 Signed-off-by: Piotr Osowski <piotr.osowski@nordicsemi.no>

Add a PyInstaller one-file build that freezes the app into a single self-contained executable, so it runs on x86_64 Ubuntu without installing Python or any dependency on the target. Build it with: tools/data_forwarder_host$ ./scripts/build_linux_binary.sh This provisions an isolated venv and runs PyInstaller, emitting the single file: dist/data-forwarder-host Add data_forwarder_host.spec (one-file recipe collecting the dynamically-loaded bleak and pyserial backends) and scripts/freeze_entry.py (calls multiprocessing.freeze_support() first so the spawn child does not relaunch the GUI). Ignore the build artefacts, re-include the build script, and document it in the README. Ref: NCSDK-39472 Signed-off-by: Piotr Osowski <piotr.osowski@nordicsemi.no>

pioso-dev

@mbarchie

Thanks for the thorough first round, and for taking the time to run the host app standalone. Sounds good on picking it back up once you've got an nRF54L15 tag — the device side is the data_forwarder sample (edge-ai/samples/data_forwarder). I put the nRF54L15 tag with a battery on your desk

Tests: good catch. That line in the description was misleading for this PR, so I've fixed it. It no longer claims an in-tree pytest suite, and I also corrected the related "unit-testable headless" wording in the architecture section. Requirements and testing for the app are handled outside this PR; what's shipped here was verified through the manual end-to-end checks now listed under Verification (UART + BLE capture, multi-session tabs, recording → CSV + metadata sidecar, and GUI responsiveness under flood). If we want a dedicated automated test suite living next to the tool, I'm happy to add that as a follow-up.

Dark theme readability: you're right, and "automatic color" was a good point. Turned out there were three causes:

the "System" theme applied Qt's fixed light palette regardless of the OS color scheme;
the palette only defined a few roles, so a lot of widgets fell back to light shades in dark mode;
a handful of widgets hardcoded light-only colors.

Fixes: the System theme now follows the OS scheme via QStyleHints.colorScheme(), the light/dark palettes are complete (including disabled text), the app forces the Fusion style so the palette actually gets honored, and the hardcoded banner/label/status colors are now theme-aware (or picked to read fine on both). I also reworked the welcome screen's "New Session" button so it's a clear call-to-action in every mode. Next time you have it open, could you re-test on dark Ubuntu? It should be readable now in System/Light/Dark.

pioso-dev · 2026-06-23T13:25:20Z

+packages = [
+    "data_forwarder_host",
+    "data_forwarder_host.core",
+    "data_forwarder_host.gui",
+    "data_forwarder_host.gui.dialogs",
+    "data_forwarder_host.gui.widgets",
+    "data_forwarder_host.platform",
+    "data_forwarder_host.protocol",
+    "data_forwarder_host.session",
+    "data_forwarder_host.source",
+    "data_forwarder_host.utils",
+]


true - I forgot to add this one

pioso-dev · 2026-06-23T13:30:30Z

+class RecordingCsvDump:
+    """Incremental writer for a finished recording's CSV.
+
+    The whole capture is RAM/spill-buffered while recording; at *stop* it must


Nice catch. There was a zero-width space (U+200B) tucked into RAM/spill-buffered — I've removed it

pioso-dev · 2026-06-23T13:31:45Z

+    def _run_off_thread(self, fn, on_done, on_failed) -> None:
+        """Run a blocking *fn* on a worker thread; deliver result on GUI thread.
+
+        The result/failure handlers must run on the **GUI thread** (they touch


Same hidden U+200B character here in result/failure — removed. I also did a quick sweep of the tree, and these two were the only spots, so we should be clear now

pioso-dev · 2026-06-23T13:34:47Z

+
+The Nordic / SEGGER detection heuristics (``NRF_HINTS``, ``_looks_like_nrf``,
+``_vid_pid``, ``_describe_port``, ``_short_port``) are ported verbatim from
+``edge-ai/tools/data_forwarder_host/python_prototype/simple_receiver/


Good point, thanks. That python_prototype/simple_receiver path no longer exists, so I've removed the stale reference from both the docstring and the inline comment

pioso-dev · 2026-06-23T13:35:23Z

+# SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
+"""Framing primitives.
+
+Ported verbatim from edge-ai/tools/data_forwarder_host/python_prototype/


The python_prototype/ tree is gone now, so I dropped the outdated "ported from …/simple_receiver.py" note from the docstring

pioso-dev · 2026-06-23T13:37:21Z

Thanks, agreed that was missing. I've added a link to the device-side sample: samples/data_forwarder in both the intro and Requirements, and noted that it advertises over BLE as nRF DataFwd and that there's no data to receive without it running

pioso-dev · 2026-06-23T13:37:55Z

+- For UART sources: permission to access the serial port (on Linux this usually
+  means membership of the `dialout` group).
+
+## Install


@mbarchie
Good point, thanks. I've added a note in the Install section to:
sudo apt install libxcb-cursor0
on Ubuntu/Debian (with a mention of the other XCB runtime libs for minimal installs), since that's what the xcb platform-plugin error usually needs

pioso-dev · 2026-06-23T13:38:34Z

+
+## Architecture (short)
+
+The application is strictly layered; the GUI-free layers carry no Qt imports:


You're absolutely right, thanks — the wording was misleading. core/ does import Qt (data_model, recorder, error_log are QObjects with signals), and so do session/ and one utils/ debug helper. I've reworded the architecture section to make the genuinely Qt-free layers explicit (platform, source, protocol, pipeline — the ones running in the headless child process) and clearly mark the layers that do use Qt

pioso-dev requested review from Szynkaa, grochu and mbarchie June 19, 2026 11:35

pioso-dev force-pushed the NCSDK-39472 branch 3 times, most recently from f212008 to 35c9a87 Compare June 22, 2026 15:38

Szynkaa reviewed Jun 23, 2026

View reviewed changes

pioso-dev force-pushed the NCSDK-39472 branch from 831dfa0 to b30ea7a Compare June 23, 2026 12:20

pioso-dev commented Jun 23, 2026

View reviewed changes

mbarchie reviewed Jun 23, 2026

View reviewed changes

pioso-dev added 2 commits June 23, 2026 16:19

pioso-dev force-pushed the NCSDK-39472 branch from b30ea7a to f4764f0 Compare June 23, 2026 14:19

pioso-dev commented Jun 23, 2026

View reviewed changes

pioso-dev requested review from Szynkaa and mbarchie June 23, 2026 14:23

		connect so the link does not rely on automatic negotiation timing. (The device
		sample already enables the buffers — `CONFIG_BT_L2CAP_TX_MTU=247`,


		## Architecture (short)

		The application is strictly layered; the GUI-free layers carry no Qt imports:

Conversation

pioso-dev commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Highlights

How to try it (recommended)

How to run (from source)

Verification

Notes

Jira

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pioso-dev left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mbarchie left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pioso-dev Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mbarchie Jun 23, 2026 • edited by pioso-dev Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pioso-dev left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pioso-dev commented Jun 19, 2026 •

edited

Loading

pioso-dev Jun 23, 2026 •

edited

Loading

mbarchie Jun 23, 2026 •

edited by pioso-dev

Loading

pioso-dev Jun 23, 2026 •

edited

Loading