POC: MCUmgr: Forward Tree potocol

[nandojve]

smp: Add forward tree protocol

Introduce the Forward Tree protocol (FT). The FT route data downstream in a tree of MCUmgr devices. This solution target a system composed by multiple units that are physically interconnected and have a single point of external communication. Some example of possible scenarios:

• SoM (System on Module) composed by multiple MCUs
• Wheel chair
• autonomous vehicles (ROS) [ cars, boats, planes, drones etc)

The Flag 0x80 was selected to indicate that the data payload contains additionally the FT 8 bytes. This 8 bytes are placed at end of regular group payload. The FT payload is composed by 16 nimbles. The first nimble is the number of hops down in the tree, which in total can handle 15. The remaining nimbles identify the downstream port index.

The FT is a generic downstream route protocol mechanism and it is not designed to create network were a downstream device can send communication upstream or downstream.

    FT example:
                              host
                               |
                               D1
                              /|\
                         ----- | -----
                        /      D3     \
                       D2      |      D4
                              / \
                      D5------   ------D6

The host is the controller which sends SMP requests. This means that data flows from host to devices in the downstream direction. A response from a device to host is the upstream direction.

In this scenario, all upstream messages are processed as a transparent forward response to the host or without any changes the content.

A downstream message needs to be inspected when the FORWARD_TREE flag bit (0x80) is set. The FT payload format is compose by 16 nimbles. The most left nimble is the hops counter. All the other nimbles are port indexes.

           ---- Nimble 15       ---- Nimble 1
          /                    /
      0xH0'00'00'00'00'00'00'0N
        \
         --- hops

Assuming that the host wants to send data to D4 device:

1- set the hops property to 1
2- add the FT port[0] nimble as 2

The correspondent FT big endian data payload in the wire will be:

                                ---- Nimble 1
                               /
      0x10'00'00'00'00'00'00'02
        \
         --- hops

In this case, when host send the data to D1 the device will inspect the
content and forward downstream in the D1 port[2], where D4 is connected.

In the same way, when host wants to send a request to D6:

1- set the hops property to 2
2- set the FT port[1] nimble to 1 and port[0] nimble to 1

                                ---- Ninble 2
                               /---- Nimble 1
                              //
      0x20'00'00'00'00'00'00'11
        \
         --- hops

This means that each time a device must inspect the FT payload and forward downstream the hops count should be reduced by 1 and when hops is evaluated to 0 the device must consume the package.

The system will use the hops value to index the correspondent FT port nimble. In summary, the FT lower port index represents the route closer to the destine and the high port index represents the node closer to the device that start the request.

Forward Tree Protocol proposal:

Devicetree example

/ {
	chosen {
		/* Compatibility mode - MCUmgr forward is disabled */
		zephyr,uart-mcumgr = &cdc_acm_uart0;
	};

	mcumgr_upstream: upstream {
		compatible = "zephyr,smpmgr-transport";
		status = "okay";

		transport = <&cdc_acm_uart0>;
	};

	mcumgr_downstream0: downstream0 {
		compatible = "zephyr,smpmgr-transport";
		status = "okay";

		transport = <&uart0>;
	};

	mcumgr_downstream1: downstream1 {
		compatible = "zephyr,smpmgr-transport";
		status = "okay";

		transport = <&uart1>;
	};

	mcumgr_forward: mcumgr_forward {
		compatible = "zephyr,smpmgr-forward";
		status = "okay";

		upstream = <&mcumgr_upstream>;
		downstream = <&mcumgr_downstream0>, <&mcumgr_downstream1>;
	};
};

&zephyr_udc0 {
	cdc_acm_uart0: cdc_acm_uart0 {
		compatible = "zephyr,cdc-acm-uart";
		label = "Zephyr USB CDC-ACM";
	};
};

Python Client:
JPHutchins/smp#49
intercreate/smpmgr#62

Example of command line for the 2 examples in the comments:

smpmgr --port /dev/ttyACMx --forward-tree "[1, 2]" os echo test

smpmgr --port /dev/ttyACMx --forward-tree "[2, 1, 1]" os echo test

Pros:
1- Add a capability to update a complex scenario with multiple devices interconnected
2- No changes in the protocol allowing compatibility
3- Very small implementation to inspect and forward
4- Support 15 levels of interconnection and 16 ports per level

Cons:
1- Require some reworking in the transports to better handle in/out and store transports more efficient
2- Overhead of 8 bytes in downstream direction

Notes:
1- This still a proposal and it is open to comments
2- There is no intention to create a network and make devices to talk between then

CC: @otavio

[sonarqubecloud]

Quality Gate passed

Issues
10 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[nandojve]

Hi @nordicjm ,

I would like to read from you about the proposal. What do you think ?

[nordicjm]

Hi @nordicjm ,

I would like to read from you about the proposal. What do you think ?

Will see when I get some time this week to have a look yes, also added @de-nordic for comment

[nandojve]

Hi @nordicjm ,
I would like to read from you about the proposal. What do you think ?

Will see when I get some time this week to have a look yes, also added @de-nordic for comment

That will be cool!

Just to mention, after thinking more about future:

1- It will be necessary to reserve 1 nimble. So, n14 hop won't be available.
2- There is no needed to separate between upstream and downstream. We can defined that the first item is the upstream port.
3- At command line --forward-tree "[1, 2]" the first element don't need to be the hops value. We can direct infer that number by the number of elements.
4- A new group named "forward_tree" should be create. This will allow an external entity (cellphone, for instance) to enumerate the tree dynamically.
5- Because now there is the concept of port and those could be defined by the transport, it will be possible to use a bus like RS-485, CAN etc. The number of nodes will be limited by the number of ports available in the protocol (16 at moment) but a new bit in the reserved area of the forward-tree could tell that port is in byte mode instead nimble mode and then expand to 256 ports per hop.

[nordicjm]

My first thought on this is it's forking the protocol and only works for a static setup with UART, it can't be used with any other transport and that's not really how MCUmgr should be expanded. Another point is there are 8 bytes added to the end of the payload, so a device needs the full packet before it can know it's for it or to be forwarded. E.g. real use-case for devices is forwarding data to other devices over MCUmgr using bluetooth - you need to know what device to connect to, it's not a static device, and you need an address for that. Likewise with UDP, you would need a IPv4 or IPv6 address

[nordicjm]

Dominik reminded me of this: #44224

[nandojve]

Hi @nordicjm ,

My first thought on this is it's forking the protocol and only works for a static setup with UART, it can't be used with any other transport and that's not really how MCUmgr should be expanded. Another point is there are 8 bytes added to the end of the payload, so a device needs the full packet before it can know it's for it or to be forwarded. E.g. real use-case for devices is forwarding data to other devices over MCUmgr using bluetooth - you need to know what device to connect to, it's not a static device, and you need an address for that. Likewise with UDP, you would need a IPv4 or IPv6 address

Thank you for your comments.

I’ve updated the beginning of the proposal to include more details about the real-world scenarios. The situation you mentioned, involving multiple BLE units, is not the main target of this proposal. While BLE or Ethernet can serve as the primary entry point, the systems that this proposal was made generally do not use these technologies to communicate directly with each other. The systems that already have an IP-based infrastructure will likely address each node directly, making the proposed protocol unnecessary in those cases.

However, this approach becomes meaningful when the targets are systems such as a SoM composed of multiple MCUs, a wheelchair, or an autonomous vehicle (ROS-based systems — cars, boats, planes, drones, etc.). In these cases, the devices are typically interconnected via wired connections. BLE, Wi-Fi, or Ethernet can still act as the main entry point, and with minimal overhead, it becomes possible to upgrade all connected devices efficiently.

The 8-byte header is placed at the end of the message to keep things simple and to not change the current protocol. This allows even very small MCUs to handle routing with almost no additional flash memory usage. Having the 8 bytes at the end enables a straightforward forwarding mechanism that is transport-independent. As you can see, it is implemented in the SMP layer, above the transport layer. If a custom encoding/decoding mechanism is used, a device can still perform forwarding in the layer just above transport.

The proof of concept (PoC) reworks the serial transport to demonstrate the idea, but there are no functional changes to the transport itself. The actual changes made in the serial transport were only to let the upper layer identify which instance to forward or respond to.

Ultimately, the only modification required in the protocol is to reserve a flag. This could be defined as a “customer option,” allowing anyone to extend the protocol’s behavior without violating the existing specification.

Why is this approach better than a pass-through?

A pass-through approach may seem elegant for very small systems (e.g., those with just two devices). However, in more complex systems—such as a robotic arm composed of many devices—dynamic reconfiguration across multiple cascaded hops becomes much more complicated. It also becomes difficult to manage cases where a hop is connected to more than one device.

As last remark, this proposal extends the current protocol and it is fully optional. This means that devices that are in the edges don't need to have the forward protocol enabled or users that don't wanted will not have a penalty in terms of code side.

I hope this clarify the main idea and scenarios.

[nordicjm]

I’ve updated the beginning of the proposal to include more details about the real-world scenarios. The situation you mentioned, involving multiple BLE units, is not the main target of this proposal. While BLE or Ethernet can serve as the primary entry point, the systems that this proposal was made generally do not use these technologies to communicate directly with each other. The systems that already have an IP-based infrastructure will likely address each node directly, making the proposed protocol unnecessary in those cases.

Sure but, this isn't really related to MCUmgr, it isn't solving the issue of being able to address multiple devices and implements a limited transport system that works over one transport, that is not how MCUmgr is designed to function. The serial transport has the additional framing on top but ignoring that I can use any MCUmgr call with the same command set over any transport - that is UART, BLE, UDP(S), LoRaWAN, chip-to-chip from one transport to another (transparently, the issue is that this should be configurable i.e. the linked issue), etc. and I'm sorry but the way this works is not acceptable for MCUmgr given the huge constraint of this and given it does not comply/match up with the MCUmgr spec. I would be happy to review a system that does properly allow a client to configure a device to route received commands from one to another over any supported transport using proper MCUmgr commands, but this is not it

[nandojve]

Hi @nordicjm ,

Sure but, this isn't really related to MCUmgr, it isn't solving the issue of being able to address multiple devices and implements a limited transport system that works over one transport, that is not how MCUmgr is designed to function. The serial transport has the additional framing on top but ignoring that I can use any MCUmgr call with the same command set over any transport - that is UART, BLE, UDP(S), LoRaWAN, chip-to-chip from one transport to another (transparently, the issue is that this should be configurable i.e. the linked issue), etc. and I'm sorry but the way this works is not acceptable for MCUmgr given the huge constraint of this and given it does not comply/match up with the MCUmgr spec. I would be happy to review a system that does properly allow a client to configure a device to route received commands from one to another over any supported transport using proper MCUmgr commands, but this is not it

I started to think that it is better to have a meeting in some WG to clarify a few points. I had the impression that use cases are mixed. This is a specific feature extension to allow communicate from a outside world to a inside wired scenario. I already have all this working with BLE and Serial and it can be demonstrated. So it is clear that it can be easily expandable to be used in all transports. The meeting could be good to understand/define what is the MCUmgr spec too.

[sonarqubecloud]

Quality Gate passed

Issues
13 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[carlescufi]

Architecture WG meeting:

• @nandojve presents MCUmgr - Forward Tree Protocol.pdf
• @bjarki-andreasen suggests this is an actual protocol extension, since it is extending functionality. @nandojve agrees
• @nordicjm expresses his worry that this will not work with multiple connections or with connections that need metadata like an IP address, because it won't be forwarded down the tree
• @de-nordic says that the metadata being placed after the data may be an issue for small devices that have pre-allocated buffers that do not account for this extra 8 bytes. Alternatives are enumerated (adding the 8 bytes to the header, having a channel-switching mechanism)
• @nordicjm has additional concerns like the limitations of the extension itself, as well as the problem with backwards compatibility: the extension would not be backwards-compatible with devices that are not aware of this extension

[nandojve]

@nordicjm ,

This is the place that flag and len are adjusted to be consumed locally. The buffer is adjusted few lines above.

[nandojve]

Linking information from Architecture WG meeting:

#44224
https://docs.zephyrproject.org/latest/boards/ezurio/pinnacle_100_dvk/doc/index.html - device supports (and is tested with) bluetooth, UART and UDP (ethernet) transports all at once
https://docs.zephyrproject.org/latest/boards/ezurio/mg100/doc/index.html - was previously tested with bluetooth, UART and UDP (over cellular) all at once

[nordicjm]

*correction the above are both bluetooth, cellular and UART
https://docs.zephyrproject.org/latest/boards/ezurio/bl5340_dvk/doc/index.html is bluetooth, ethernet and UART

[nordicjm]

@nordicjm ,

This is the place that flag and len are adjusted to be consumed locally. The buffer is adjusted few lines above.

this has multiple red and yellow warning/error boxes around it in code view so seemingly was missed

[nordicjm]

I remain opposed to this approach, the approach given in #44224 from years ago does not break the protocol, likely would have less or equal NVM usage, is transparent to users of it and fully supports connection orientated transports, it would even support opening connections on those transports e.g. UDP connecting to another host, and is fully dynamic at run time not needing a complete setup in devicetree (which isn't even really connecting physical hardware) plus would fully support custom transports and even custom ways of linking transports out of tree using the MCUmgr hooks functionality.

[nordicjm]

See comment above

[github-actions]

This pull request has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this pull request will automatically be closed in 14 days. Note, that you can always re-open a closed pull request at any time.