This project is a backend solution for prototyping Internet of Things (IoT) services. It leverages the power of Docker containers to easily integrate Grafana, InfluxDB, and Node-RED into a seamless, scalable, and robust backend system. The solution aims to provide rapid prototyping capabilities for IoT applications that need real-time data visualization, storage, and workflow automation.
- Eclipse Mosquitto: MQTT broker
- Grafana: Real-time data visualization and monitoring dashboard
- InfluxDB: High-performance data storage
- NodeRED: Flow-based development tool for visual programming and data flow automation
- Jupyter Notebook: a web application for creating and sharing computational documents
- Interfacer API: an API for ingestion into the Interfacer application
Data is collected by IoT devices, e.g. an ESP32 based power monitor. These devices publish their data via MQTT into a topic in a message broker. We use Eclipse Mosquitto as a MQTT message broker.
Node-RED is used to read and transform or combine data and to implement more sophicsticated use cases like notifications or worksflow. Node-RED subscribes to topics in Mosquitto and can be used to save transformed data into a database.
As our data is bases on time, we are using a time series database to store information. We used InfluxDB as this database.
Dashboards can already be created in Node-RED, but to be more flexible (and include more options) we are using Grafana. Grafana reads data from our database and other sources (like CSV files on the Internet) and displays them in a nice dashboard.
-
Download the Daedalus arm64 image zipfile for the Raspberry Pi 5 nightly builds on the Devuan ARM files site (the zipfile begins with
rpi-5-devuan-daedalus-and ends with.zip). -
Transfer the image to the SD card using your prefered disk tool.
-
Insert the SD card into the Devuan RPi5.
-
Power up the Devuan RPi5, log in (devuan/devuan), and ensure the Raspberry Pi is connected to your network (either through the ethernet interface or you must configure the WiFi details for your local network with the command
menu-config). -
Note down the IP address allocated to the Devuan RPi5.
-
On the ansible provisioning host, ssh into the RPi5 via username and password (devuan/devuan) via ssh
devuan@IP_address_noted, and issuesudo su -followed byssh-keygen. Press return twice. Log out (control-D). -
On the ansible provisioning host, generate an SSH key pair (
ssh-keygen) and add the public key to the RPi5 root SSH authorized_keys configuration file/root/.ssh/authorized_keys.
git clone https://github.com/dyne/lauds-iot-backend.git
cd lauds-iot-backend/ansible- On the ansible provisioning host a hostname and local IP address must be uniquely defined for the Devuan RPi5 in the
inventory.ymlfile (eg, a host configuration example forflirc-rpi5is given as an example)
The unique hostname should be related to the LAUDS factory sitename or consortium member name and the IP address must correspond to the address noted in the previous step.
- Individual host VPN IP address configuration is configured in
ansible/host_vars/with the configuration file matching the hostname (eg, the host configuration example file in the repo isansible/host_vars/flirc-pi5):
client_ip_addr: 192.168.10.[address]where [address] is a unique host address that will be allocated to your install.
- On the ansible provisioning host, create or add to the file
../.envwith the following content:
# LAUDS GATEWAY SERVER DETAILS
#
# export GATEWAY_SERVER_HOSTNAME="flirc-pi5"
#
## WIREGUARD VPN DETAILS (used in ansible)
#
export VPN_SERVER_IPV4="138.201.89.108"
export VPN_SERVER_IPV6="2a01:4f8:c17:3dd5::1"
export VPN_SERVER_HOSTNAME="zorro.free2air.net"
export VPN_SERVER_PORT="51194"
# TODO: Publish public key & lookup in DNS when BIND9.18.33 SIG0 update bugfix is published
export VPN_SERVER_PUBLIC_KEY="IstwnIfVuvgfb7LzaE3YLb24FAT2oUEhVcsZILDhHXk="
export VPN_CLIENT_IPV4_NETMASK="/24"
On the ansible provisioning host:
source ../.env && ansible-playbook -i inventory.yml playbook.ymlCreate file .env to set default credentials
# NodeRed admin user password
# export NR_ADMIN_PW="<your NR admin user login password>"
export NR_ADMIN_PW="laudsgateway"
# Jupyter Notebook token
# export JUPYTER_TOKEN="<your Jupyter Notebook login token>"
export JUPYTER_TOKEN="laudsgateway"Most Docker containers are off-the-shelf, but the Node-RED container is built with some useful plugins included. To build and run these Docker containers in a single step:
Assuming required credentials above are set in .env
source .env && docker compose --file software/container/docker-compose.yml up --force-recreate --buildEclipse Mosquitto is an open source message broker which implements a server for MQTT. It runs in Docker and is exposed on the default MQTT port 1883. You can subscribe to and push into topics:
mosquitto_sub -h localhost -t '#' -p 1883
mosquitto_pub -h localhost -p 1883 -t '/' -m $(date --utc +%s)A useful tool to debug MQTT is MQTT Explorer found on Github.
Node-RED is a low-code programming tool for wiring together hardware devices, APIs and online services in new and interesting ways.
It provides a browser-based editor that makes it easy to wire together flows using the wide range of nodes in the palette that can be deployed to its runtime in a single-click.
Node-RED is also running in Docker and is exposed on port 1880: http://localhost:1880/
InfluxDB is a database for any time series data. It runs in Docker and is exposed on port 8086: http://localhost:8086/ (you have to create an initial user in just a few simple steps)
Grafana is visualisation software for building operational dashboards. It runs in Docker and is exposed on port 3000:
You can login to Grafana: http://localhost:3000/login (admin:admin)
Jupyter Notebook is a web application for creating and sharing computational documents.
Interfacer API is an API service for interacting with the Interfacer Project
A simple introduction to Node-RED can be found - along with the nodes / the code - in this repository.
A more sophisticated exampe on how to connect Node-RED to influx is also available in this repository.
Have a look at the the document in this repository.
As an example you can use a Shelly plug flashed with Tasmota and feed MQTT data with Node-RED into InfluxDB and visualize it with Grafana. Have a look, here.
This is just a prototype. You should not expose the services as-it-is to the public internet.
Also you have to think about a backup solution. At the moment data lives in Docker volumes, that could be backed up:
- influxdb2: The time series database
- grafana-dashboards: Grafana dashboards
- grafana-data: Grafana data
- nodered-data: Flows created in Node-RED
Feel free to open an issue for bugs, feature requests, or questions. Contributions are welcome.
This project follows the REUSE Specification and is licensed under the GPL 3.0 or later License - see the LICENSE file for details.
