Zigbee coordinator firmware with RTS / CTS Hardware Flow Control enabled compatible with Sonoff Zigbee 3.0 USB Dongle Plus adapter by ITead

[Hedda]

Request Z-Stack 3.x.0 Zigbee coordinator firmware with RTS / CTS Hardware Flow Control enabled and compatible with the new Sonoff Zigbee 3.0 USB Dongle Plus adapter by ITead which hardware support Hardware Flow Control when enabled via dip-switch:

https://sonoff.tech/wp-content/uploads/2021/09/Zigbee-3.0-USB-dongle-plus-firmware-flashing-1-1.docx

(Guide above linked from https://sonoff.tech/product-review/zigbee-3-0-usb-dongle-plus/ written by ITead).

2. Enable hardware flow control and generate firmware (optional) section says "2.1 Import the ZNP project of CC1352P into CCS. Open the .syscfg configuration file in the ZNP project. Enable serial flow control in the UART option in the .syscfg configuration file."

Configuration in the image from that document looks like tick the box for "Flow Control" under "UART" and use configuration:

UART Peripheral = Any(UART0)
TX Pin = DIO13/4 (XDS110 UART)
RC Pin = DIO12/3 (XDS110 UART)
CTS Pin = DIO19/19 (Header)
RTS Pin = DIO18/36 (Header)

This request was initially brought up as a wish in #316 but since that issue specifically regarding Zigbee router firmware and nor Zigbee coordinator firmware I thought that I would post a seperate issue as a request for Zigbee coordinator firmware for this.

Would you be willing to compile and host firmware images that support using hardware flow control with this specific dongle?

I understand switching to hardware flow control mode on the new Sonoff Zigbee 3.0 USB Dongle Plus adapter by ITead is done manually via the physical dip-switch on the adapters board but also understand that we need a custom firmware built for it too(?).

Picture from https://itead.cc/product/sonoff-zigbee-3-0-usb-dongle-plus/ show an "DIP switch for hardware flow control":

So would need custom coordinator firmware before enabling hardware flow control in the software application like Zigbee2MQTT:

https://www.zigbee2mqtt.io/information/configuration.html

# Optional: RTS / CTS Hardware Flow Control for serial port (default: false)
rtscts: true

Hardware flow control should in theory make for more stable serial communication since the flow control is off-loaded from the application and the system CPU to the MCU, but not sure if it will really be more stable in the real world for Zigbee2MQTT.

PS: Know I read that openHAB's Zigbee developer recommended hardware flow control for commercial use for added stability, however then we are talking about Zigbee Coordinators and Zigbee devices, not about dedicated stand-alone Zigbee routers.

[Koenkk]

I prefer not to provide additional firmwares since it leads to more confusion for users and more maintenance for me unless it provides a significant improvement.

[Hedda]

Would configuring it in firmware actually break functionality if the same firmware was flashed to another adapter without physical hardware flow control pins connected if the optional hardware flow control setting is not enabled in the application?

That is, if a user flash such firmware with Flow Control activated as per ITead's instructions to for example Tube's CC2652P2 USB Coordinator (based on RFSTAR RF-BM-2652P2) but do not enable it Zigbee2MQTT would that then not connect via SW serial?

https://www.zigbee2mqtt.io/information/configuration.html

# Optional: RTS / CTS Hardware Flow Control for serial port (default: false)
rtscts: false

[Koenkk]

Not sure, can you investigate this?

[Hedda]

Maybe, if you could provide compiled coordinator firmware with flow control activated so we can test it on different dongles?

If we get a test FW image of such firmware then I can at least ask for help from the communities that requested this feature.

Perhaps it could at first be tested on just the Sonoff Plus dongle which if it works should work with the DIP-switch in both positions as long as we set the correct Flow Control configuration in Zigbee2MQTT (and other applications) that matches.

Only other CC2652 dongle I currently own is a zzh (CC2652R) that is in "production" at home so will not test on it myself now.

[tteck]

if you could provide compiled coordinator firmware with flow control activated so we can test it on different dongles?

I'd also be willing to test.

[Hedda]

@guozi7788 As this issue and questions are related to your comments about pins for Hardware Flow Control in this PR -> #329

Do you happen to have some connections that you could report this request for firmware to relevant personnel at ITead/Sonoff?

The community would like a pre-compiled firmware image with Hardware Flow Control enabled to test on the SONOFF Zigbee 3.0 USB Dongle Plus by ITead. If someone at ITead/Sonoff or the engineering company it uses could please compile provide such a pre-compiled firmware image then we would get willing testers from the community to test it and report their results.

PS: Tip is that experimental firmware images could be provided by on a GitHub repo similar to the other Sonoff Zigbee 3.0 dongle:

https://github.com/xsp1989/zigbeeFirmware/

[Hedda]

FYI, submitted the question via e-mail to ITead and got a reply email from ITead's project manager for the Sonoff Zigbee 3.0 USB Dongle Plus who unfortunately said that ITead has no plans on maintaining and distributing their own "ITead official" firmware images that supports hardware flow control on it, and continued to say that they currently only plan to provide tutorials on how any user in the community themselves can set up and generate the appropriate tools for building and compiling firmware as needed, meaning that they rely on the community to compile our custom firmware if we want it, (i.e. use community firmware builds like https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator/Z-Stack_3.x.0 ).

Again, ITead/Sonoff engineer guozi7788 did however at least confirm in #329 which pins should be mapped when building it:

I have added the information you expect to know in this PR:
BSL Trigger Pin: DIO15
Auto-BSL: yes
LED: DIO_7(Not welded) Power LED(Red)
Hardware Flow Control: DIO13: TX DIO12: RC DIO19: CTS(Need DIP-switch ON) DIO18: RTS(Need DIP-switch ON)

[Merwenus]

Great, this is the second Zigbee router I bought for them and they suck in different ways, excellent and they just don't care.

[Hedda]

Great, this is the second Zigbee router I bought for them and they suck in different ways, excellent and they just don't care.

Know that there is no point in using Hardware Flow Control firmware if used as a Zigbee router as Zigbee routers are not relying on any serial communication on a host while in operation. Zigbee router just needs power to work.

Hardware Flow Control firmware is only of interest for a Zigbee Coordinator to off-load flow control from CPU to MCU in a setup in a non-dedicated system where you might be running a lot of other stuff on the main System CPU, (like maybe on a Qnap or Synology NAS with a virtual machine than run Linux which in turn also runs other applications than Zigbee2MQTT). It should also be noted that at least so far Software Flow Control firmware has been proven to be stable in practically all real-world scenarios.

[dkwireless]

Do we have benefits on CC2652P side with HW flow control? If yes then we should pursuit this, if no, I think there is no need...

[Hedda]

Do we have benefits on CC2652P side with HW flow control? If yes then we should pursuit this, if no, I think there is no need...

My guess is that we will not know the benefits of Hardware Flow Control over Software Flow Control without long term testing. That is, make different stress-test comparisons where Hardware Flow Control versus Software Flow Control is put to their limits.

At least on paper Hardware Flow Control should theoretically create a more robust and solid connection, even if used when the system CPU and/or operating system is under very heavy load, which is when a Software Flow Control is more likely to temporarily lag/delays or even drop the connection that can cause missed messages. So if you hypothetically simply want the best theoretical connection that should be possible then Hardware Flow Control is the best, on paper.

Software Flow Control for Zigbee Coordinator adapters/controllers might perhaps make stable enough in a majority or even all of the real-world daily scenarios for home automation usage in private residents, but I understand that Hardware Flow Control is the norm standard for Zigbee in real-world commercial use such as industry, medical (hospitals and hospice or home care), utility companies (e.g. "Smart Energy" meters for electricity, gas, heating, and water utility metering), as well when targeting IoT within commercial buildings (for example lightning and HVAC heating or cooling).

Note that we know that unstable serial connection can cause issues for Z-Stack, which is why Wi-Fi based remote adapters are not recommended. See -> https://www.zigbee2mqtt.io/guide/adapters/#notes

Network adapters connected via WiFi might have reduced stability as the serial protocol does not have enough fault tolerance to handle packet loss or latency delays that can normally occur over WiFi connections. If cannot use a locally connected USB or UART/GPIO adapter then the recommendation is to use a remote adapter that is connected via Ethernet (wired) to avoid issues.

So based on issues we can see over WiFi we can assume that the Z-Stack ZNP interface protocol is currently not designed with robustness enough to deal with a lot of lag/delays and temporarily dropped connections. Instead, it is designed to work with a direct connected Zigbee serial adapter. You can read more about the concept of in robustness computer science https://en.wikipedia.org/wiki/Robustness_(computer_science)

PS: Disclaimer is that I am a bit biased as find it a little hard to be subjective on this topic as professionally I worked as a server infrastructure engineer for over 20-years and there having redundancy and utilizing robust hardware components/systems for high-availability (as well as having backup systems for safe fail-overs) is one of the essential keys to achieving maximum uptime as few unplanned downtimes as possible. Otherwise, all of the robustness must be achieved by application developers by running several instances of all applications and their dependencies in high availability, which is something that usually costs more and is harder to justify the overhead for.

[verjus]

I was able to enable hardware flow control in Koenkk's firmware by making the following modifications to two files, after following the instructions provided here.

The two files are: ti_drivers_config.c and ti_drivers_config.h. They reside in the syscfg directory, after applying Koenk's patch.

Here is a diff for each file:

diff ti_drivers_config.h ~/syscfg.or/ti_drivers_config.h 
97,104d96
<   /* Owned by CONFIG_DISPLAY_UART as  */
< extern const uint_least8_t CONFIG_GPIO_3_CONST;
< #define CONFIG_GPIO_3 19
< 
< /* Owned by CONFIG_DISPLAY_UART as  */
< extern const uint_least8_t CONFIG_GPIO_4_CONST;
< #define CONFIG_GPIO_4 18
< 
225,226d216
<  *  CTS: DIO19
<  *  RTS: DIO18

and

diff ti_drivers_config.h ~/syscfg.or/ti_drivers_config.h 
97,104d96
<   /* Owned by CONFIG_DISPLAY_UART as  */
< extern const uint_least8_t CONFIG_GPIO_3_CONST;
< #define CONFIG_GPIO_3 19
< 
< /* Owned by CONFIG_DISPLAY_UART as  */
< extern const uint_least8_t CONFIG_GPIO_4_CONST;
< #define CONFIG_GPIO_4 18
< 
225,226d216
<  *  CTS: DIO19
<  *  RTS: DIO18
PCAXS::syscfg>diff ti_drivers_config.c ~/syscfg.or/ti_drivers_config.c
269,272c269,270
<     /* Owned by CONFIG_DISPLAY_UART as RTS */
<     GPIO_CFG_OUTPUT_INTERNAL | GPIO_CFG_OUT_STR_MED | GPIO_CFG_OUT_LOW, /* CONFIG_GPIO_4 */
<     /* Owned by CONFIG_DISPLAY_UART as CTS */
<     GPIO_CFG_INPUT_INTERNAL | GPIO_CFG_IN_INT_NONE | GPIO_CFG_PULL_DOWN_INTERNAL, /* CONFIG_GPIO_3 */
---
>     GPIO_CFG_NO_DIR, /* DIO_18 */
>     GPIO_CFG_NO_DIR, /* DIO_19 */
307,308d304
< const uint_least8_t CONFIG_GPIO_3_CONST = CONFIG_GPIO_3;
< const uint_least8_t CONFIG_GPIO_4_CONST = CONFIG_GPIO_4;
813,814c809,810
<     .ctsPin             = CONFIG_GPIO_3,
<     .rtsPin             = CONFIG_GPIO_4,
---
>     .ctsPin             = GPIO_INVALID_INDEX,
>     .rtsPin             = GPIO_INVALID_INDEX,

The relevant sections of each file should look like this for ti_drivers_config.c

void* gpioUserArgs[31];

const uint_least8_t CONFIG_GPIO_1_CONST = CONFIG_GPIO_1;
const uint_least8_t CONFIG_GPIO_2_CONST = CONFIG_GPIO_2;
const uint_least8_t CONFIG_GPIO_3_CONST = CONFIG_GPIO_3;
const uint_least8_t CONFIG_GPIO_4_CONST = CONFIG_GPIO_4;
const uint_least8_t CONFIG_RF_24GHZ_CONST = CONFIG_RF_24GHZ;

this

    GPIO_CFG_NO_DIR, /* DIO_16 */
    GPIO_CFG_NO_DIR, /* DIO_17 */
    /* Owned by CONFIG_DISPLAY_UART as RTS */
    GPIO_CFG_OUTPUT_INTERNAL | GPIO_CFG_OUT_STR_MED | GPIO_CFG_OUT_LOW, /* CONFIG_GPIO_4 */
    /* Owned by CONFIG_DISPLAY_UART as CTS */
    GPIO_CFG_INPUT_INTERNAL | GPIO_CFG_IN_INT_NONE | GPIO_CFG_PULL_DOWN_INTERNAL, /* CONFIG_GPIO_3 */
    /* Owned by CONFIG_NVS_SPI_0 as Button GPIO */
    GPIO_CFG_OUTPUT_INTERNAL | GPIO_CFG_OUT_STR_MED | GPIO_CFG_OUT_HIGH, /* CONFIG_GPIO_0 */
    GPIO_CFG_NO_DIR, /* DIO_21 */
    GPIO_CFG_NO_DIR, /* DIO_22 */
    GPIO_CFG_NO_DIR, /* DIO_23 */
    GPIO_CFG_NO_DIR, /* DIO_24 */

and this

static const UARTCC26XX_HWAttrsV2 uartCC26XXHWAttrs[CONFIG_UART_COUNT] = {
  {
    .baseAddr           = UART0_BASE,
    .intNum             = INT_UART0_COMB,
    .intPriority        = (~0),
    .swiPriority        = 0,
    .powerMngrId        = PowerCC26XX_PERIPH_UART0,
    .ringBufPtr         = uartCC26XXRingBuffer0,
    .ringBufSize        = sizeof(uartCC26XXRingBuffer0),
    .rxPin              = CONFIG_GPIO_2,
    .txPin              = CONFIG_GPIO_1,
    .ctsPin             = CONFIG_GPIO_3,
    .rtsPin             = CONFIG_GPIO_4,
    .txIntFifoThr       = UARTCC26XX_FIFO_THRESHOLD_1_8,
    .rxIntFifoThr       = UARTCC26XX_FIFO_THRESHOLD_4_8,
    .errorFxn           = NULL
  },
};

and for ti_drivers_config.h

/*
 *  ======== GPIO ========
 */
/* Owned by CONFIG_DISPLAY_UART as  */
extern const uint_least8_t CONFIG_GPIO_1_CONST;
#define CONFIG_GPIO_1 13

/* Owned by CONFIG_DISPLAY_UART as  */
extern const uint_least8_t CONFIG_GPIO_2_CONST;
#define CONFIG_GPIO_2 12

  /* Owned by CONFIG_DISPLAY_UART as  */
extern const uint_least8_t CONFIG_GPIO_3_CONST;
#define CONFIG_GPIO_3 19

/* Owned by CONFIG_DISPLAY_UART as  */
extern const uint_least8_t CONFIG_GPIO_4_CONST;
#define CONFIG_GPIO_4 18

/* Owned by /ti/drivers/RF as  */
extern const uint_least8_t CONFIG_RF_24GHZ_CONST;
// #define CONFIG_RF_24GHZ 28

Then rebuild the firmware.

I tested that it works with the latest dev z2m, with rtscts enabled in the configuration file:

advanced:
  # Optional: Baud rate speed for serial port, this can be anything firmware support but default is 115200 for Z-Stack and EZSP,
  # 38400 for Deconz, however note that some EZSP firmware need 57600.
  #baudrate: 115200
  # Optional: RTS / CTS Hardware Flow Control for serial port (default: false)
  rtscts: true

and of course, the with hardware switch activated. I haven't tested the firmware itself but z2m communicates with the stick.

Hope this is useful to someone. Disclaimer, I haven't tested the firmware's performance or even its ability to pair. I just checked that the firmware allows for the serial port to be open:

Zigbee2MQTT:debug 2021-12-05 20:44:29: Using zigbee-herdsman with settings: '{"adapter":{"concurrent":null,"delay":null,"disableLED":false},"backupPath":"/opt/zigbee2mqtt/data/coordinator_backup.json","databaseBackupPath":"/opt/zigbee2mqtt/data/database.db.backup","databasePath":"/opt/zigbee2mqtt/data/database.db","network":{"channelList":[11],"extendedPanID":[221,221,221,221,221,221,221,221],"networkKey":"HIDDEN","panID":6754},"serialPort":{"path":"/dev/ttyUSB0","rtscts":true}}'
  zigbee-herdsman:controller:log Starting with options '{"network":{"networkKeyDistribute":false,"networkKey":[redacted],"panID":6754,"extendedPanID":[221,221,221,221,221,221,221,221],"channelList":[11]},"serialPort":{"rtscts":true,"path":"/dev/ttyUSB0"},"databasePath":"/opt/zigbee2mqtt/data/database.db","databaseBackupPath":"/opt/zigbee2mqtt/data/database.db.backup","backupPath":"/opt/zigbee2mqtt/data/coordinator_backup.json","adapter":{"disableLED":false,"concurrent":null,"delay":null}}' +0ms
  zigbee-herdsman:adapter:zStack:znp:log Opening SerialPort with /dev/ttyUSB0 and {"baudRate":115200,"rtscts":true,"autoOpen":false} +0ms
  zigbee-herdsman:adapter:zStack:znp:log Serialport opened +6ms
  zigbee-herdsman:adapter:zStack:znp:SREQ --> SYS - ping - {"capabilities":1} +0ms

[tteck]

@verjus are you gonna share the firmware for testing?