embr is a wireless, embedded voice-control system built to make Matter-enabled lighting respond quickly and naturally. It wakes on command, processes your voice locally, and triggers lighting changes in milliseconds, all designed around low power, real-time technology.
It combines on-device keyword detection with Thread networking to deliver responsive smart-light control across a local mesh. And embr is just one part of a broader Edge AI platform aimed at enabling practical Matter device control wherever you need it.
embr integrates with harth, a bespoke Matter bridge currently in active development. Over time, harth is intended to connect with an entire set of drop-in AI modules to support autonomous, intelligent, and safe control of your favorite spaces.
- Core board bring-up, logging, and platform wrappers (GPIO/LED/mic)
- WOS-triggered DMIC capture window + mic lifecycle controls (start/stop/reset/deinit)
- Edge Impulse wrapper integration with public stub backend and local real-model override
- Inference capture pipeline + classification label mapping + command selection
- Thread networking transport via OpenThread as a sleepy minimal end device (MTD/SED), sending multicast CoAP light commands
- Platform wrapper for NCS OpenThread and socket-based CoAP APIs
- Unit tests (Twister/Ztest), CI, and focused coverage for platform/inference/logic/transport layers
- PPK2 power baselines (blinky, DMIC continuous capture, DMIC WOS events, WOS + DSP/inference)
- WOS capture window + DSP/inference validation, via power profiling
- Thread/SED idle and event power profiling
- NFC-assisted or otherwise streamlined Thread commissioning/provisioning
- App-level integration testing of the
embr_appruntime
- Track resource metrics, telemetry, and performance benchmarking
- Companion translator node (separate device/firmware; developed concurrently):
- CoAP server endpoint → Matter cluster commands
- Documented end-to-end system demo:
- voice command → on-device inference → Thread/CoAP message → Matter/Thread smart bulb control
Voice control for lighting should feel intuitive, instant, and dependable. Relying on cloud services, Wi-Fi stability, or a vendor mobile app adds latency, fragility, and inconvenience to something that should "just work". embr is a local-first embedded project exploring on-device keyword detection paired with Thread-native control paths. An Edge AI-powered core and a clean boundary to Matter-enabled devices allows for fast, private control that remains interoperable as the system expands.
This repository is an incremental migration from a working prototype into a clean, testable codebase. The current public baseline includes bring-up, platform seams, a local inference pipeline with a public stub/default build path, and focused unit coverage across key platform and app modules; additional integration work and the full end-to-end system path will land here as they are refactored and integrated.
.github/workflows/→ CI workflowsapp/→ application logic and featuresboards/→ board-specific overlays and build configurationdocs/→ documentation, architecture decisions, and README assets (images, diagrams)platform/→ platform abstractions (drivers, wrappers, portability seams)scripts/→ helper scripts (docs, tooling)src/→ application entry points / glue codetests/unit/→ unit tests executed via Zephyr's Twisterthird_party/→ external dependencies and vendor librariesCMakeLists.txt→ CMake build configurationDoxyfile→ Doxygen documentation configurationprj.conf→ Kconfig application configuration
Architecture decisions are recorded in docs/adr.
- nRF Connect SDK (NCS) v2.4.0 (version used in development)
- Zephyr Toolchain with west metatool [included with NCS]
- Nordic Semiconductor Thingy:53
- Compatible Nordic Development Kit or SEGGER J-Link debugger
- Alternatively, the nRF Connect Programmer App (if flashing via DFU)
From the project root directory:
west build -p always -b thingy53_nrf5340_cpuapp
This will do a pristine build for the Thingy:53 target board (assuming a secure deployment environment).
prj.local.conf is gitignored and intended for local/private overrides only.
The default/public build uses a stub ei_wrap backend and does not require
private Edge Impulse deployment files.
To enable the real Edge Impulse-backed build locally, add the following
configuration to prj.local.conf at the repo root:
CONFIG_EDGE_IMPULSE=y
CONFIG_EDGE_IMPULSE_URI="third_party/edge_impulse/embr"
CONFIG_EI_WRAPPER_DATA_BUF_SIZE=32001
You must also provide the corresponding Edge Impulse deployment files locally under the configured URI path.
When CONFIG_EDGE_IMPULSE is not enabled, the firmware builds and runs with a
deterministic stub inference backend that emits an unknown result.
The firmware is configured as an OpenThread Minimal Thread Device / Sleepy End
Device (MTD/SED). embr is a leaf command source: it sends voice-triggered CoAP
commands and does not route Thread traffic for other nodes.
The SED role keeps wake-on-sound idle behavior reliable while the Thread stack
is enabled. With NCS v2.4.0, the default SED poll period resolves to 236000
ms; embr uses that default because it does not expect application-level
inbound messages during normal operation.
Thread network credentials remain local in prj.local.conf; the tracked
prj.conf owns the product role and stack sizing.
Local images can be configured to attach to an existing Thread network by providing values derived from the OTBR active dataset:
CONFIG_OPENTHREAD_NETWORKKEY="<16-byte colon-delimited network key>"
CONFIG_OPENTHREAD_PANID=<decimal PAN ID>
CONFIG_OPENTHREAD_CHANNEL=<11-26>
CONFIG_OPENTHREAD_XPANID="<8-byte colon-delimited extended PAN ID>"
The OTBR CLI prints the Extended PAN ID and network key as contiguous
hexadecimal characters. Zephyr expects colon-delimited byte strings for these
options, so insert a colon between each byte pair. The OTBR CLI prints the PAN
ID in hexadecimal form; convert it to decimal for CONFIG_OPENTHREAD_PANID.
Thread dataset values are environment-specific credentials and remain local
through the gitignored prj.local.conf override.
Build an image with local overrides using:
west build -p always -b thingy53_nrf5340_cpuapp -- -DEXTRA_CONF_FILE=prj.local.conf
The default/public build uses a stub inference backend. If you want to replace it with a real Edge Impulse deployment, your local model must match the app's label and timing contract.
See docs/edge_impulse.md for the required labels and audio/inference parameters.
Connect the Thingy:53 to the Nordic DK's J-Link DEBUG OUT header using a 10-pin, 2x5 1.27 mm IDC (SWD) ribbon cable.
From the project root directory:
west flash
Alternatively, the dfu_application.zip file created in build/zephyr can be programmed onto the Thingy:53 via Nordic's nRF Connect Programmer application.
Unit tests live under tests/unit and are executed via Zephyr's Twister test runner.
From the project root directory:
west twister -T tests/unit
| Profile | Avg Current | Peak Current | Notes |
|---|---|---|---|
| LED Blink Baseline | 10.46 µA | 13.60 mA | Mean of 3 trials; steady-state after init settles |
| DMIC Continuous Capture | 646.08 µA | 16.70 mA | Mean of 3 trials; 16 kHz mono PDM capture, no sleep/idle |
| WOS Armed Idle | 101.44 µA | 11.33 mA | Mean of 3 idle trials; 26.85 s windows, no wake events |
| WOS Wake Event | 673.25 µA | 14.83 mA | Mean of 6 wake selections; ~1.006 s active capture window per wake |
| WOS + DSP/Inference Armed Idle | 96.24 µA | 12.83 mA | Mean of 3 idle trials; 26.85 s windows, no wake events |
| WOS + DSP/Inference Event | 1.154 mA | 16.34 mA | Mean of 3 combined event selections; ~1.175 s capture + DSP/inference window |
| Thread Network Unavailable (Attach Retry) | 3.97 mA | 25.50 mA | Mean of 3 trials; repeated attach attempts, not idle floor |
| Thread-Connected Idle + WOS/DSP Armed | 113.98 µA | 10.22 mA | Mean of 3 trials; SED connected idle with WOS + DSP/inference armed |
| Thread-Connected WOS Capture | 713.67 µA | 14.68 mA | Mean of 3 command events; ~1.003 s capture selection |
| DSP/Inference + CoAP TX | 3.34 mA | 13.34 mA | Mean of 3 command events; ~217.8 ms DSP/inference + CoAP selection |
| Full WOS-to-CoAP Command Event | 1.187 mA | 14.68 mA | Mean of 3 command events; ~1.218 s capture-through-CoAP selection |
Full measurement notes and screenshots: docs/power.md
- Language: C
- Firmware base: Zephyr RTOS (via nRF Connect SDK v2.4.0)
- Voice pipeline: DMIC/PDM audio capture → buffering → on-device inference → command mapping
- On-device inference: Edge Impulse integration behind a thin wrapper with a public stub/default backend and local real-model override
- Networking: Thread via OpenThread + multicast CoAP messaging for local control paths
- Matter boundary: Integrates with harth (Thread-side messages translated to Matter cluster operations)
- Thingy:53, running custom 'embr' application firmware:
- nRF5340 SoC:
- Application Core: 128 MHz Arm Cortex-M33 CPU [1 MB Flash + 512 KB RAM]
- Network Core: 64 MHz Arm Cortex-M33 CPU [256 KB Flash + 64 KB RAM]
- nRF5340 SoC:
- nRF52840-Dongle, running vendor OpenThread Radio Co-Processor (RCP) firmware:
- nRF52840 SoC:
- 64 MHz Arm Cortex-M4 with FPU [1 MB Flash, 256 KB RAM]
- nRF52840 SoC:
- nRF54L15-DK, running custom 'harth' bridge firmware:
- nRF54L15 SoC:
- 128 MHz Arm Cortex-M33 [1.5 MB NVM, 256 KB RAM]
- nRF54L15 SoC:
- Power Profiler Kit II (PPK2):
- used to measure, quantify, and visualize power consumption during development and application runtime:
- 200nA to 1A measurement range
- Essentials A19/E26 Matter/Thread Smart Bulb
- Matter controller host: runs chip-tool for commissioning/testing
- Thread border router host: runs OTBR (Docker) + manages the Thread network; connected to nRF52840-Dongle with RCP firmware
- west: Zephyr/NCS workspace management + build/flash/debug wrapper
- CMake + Ninja: build system + fast backend generator
- Kconfig: feature and compile-time configuration menus/options
- Devicetree (DTS): hardware description (pins, buses, peripherals)
- SEGGER J-Link: SWD debug probe used by Nordic DKs
- RTT (Real-Time Transfer): low-overhead debug logging over J-Link
- nrfjprog / nrfutil: Nordic flashing, provisioning, and DFU utilities (as needed)
- GDB: via
west debugcommand
- chip-tool: CLI Matter controller for commissioning/testing devices
- ZAP (ZCL Advanced Platform): generates Matter cluster/attribute code from data model
- Used when implementing harth, but included here as a system development tool
- OpenThread Border Router (Docker): local Thread network + commissioning environment
- Twister: Zephyr's test runner for unit/integration tests
- Ztest: Zephyr's unit test framework
- GitHub Actions: CI for builds/tests on pushes and pull requests
- Docker: pinned, reproducible toolchain environment (NCS v2.4.0 + Zephyr)
This repository does not include private Edge Impulse model artifacts. The
public/default build remains usable through a stub inference backend, while
local real-model builds are enabled separately through prj.local.conf and a
local Edge Impulse deployment.