Thingy:91 - LED not coming on, can't see logs on terminal and not registering with AWS IoT Core

[jrdeck]

I have followed this doco to set up AWS and create a customised firmware.

My only deviation was that, rather than flashing from the Linux box where I built, I downloaded the certificates and HEX file to my PC (Windows 10) and applied manually through LTE Link Monitor (certs) and Link Monitor (HEX). Both of those activities showed success messages.

After applying the custom firmware, the LED no longer lights up and device is clearly behaving differently. However, there is no evidence that it is communicating with AWS. Further, I modified overlay-low-power.conf to re-enable the LED and UART, but this does not have an effect on the device. I have rebuilt/reapplied the firmware multiple times, to multiple devices.

I'm very confused and running blind, because I can't see anything on a terminal Link Monitor or J-link RTT Viewer. I have also tried the Trace Collector application, however, the files it generates are 0 bytes.

I see no evidence in AWS CloudTrail of the devices attempting to contact AWS.

It's clear that something on the firmware is not happy, but how to troubleshoot and resolve are not clear to me. I am hoping by my description, someone familiar with the devices and this code base might have some ideas of where I have gone wrong.

In build/zephyr/.config, it appears the relevant settings are applied (eg. for UART and LED), however, when I connect the device LED Link Monitor, I don't see any output and there is no LED lit up on the device.

#
# Device Drivers
#
# CONFIG_IEEE802154 is not set
# CONFIG_LORA is not set
CONFIG_UART_CONSOLE_ON_DEV_NAME="UART_0"
CONFIG_CONSOLE=y
CONFIG_CONSOLE_INPUT_MAX_LINE_LEN=128
CONFIG_CONSOLE_HAS_DRIVER=y
# CONFIG_CONSOLE_HANDLER is not set
CONFIG_UART_CONSOLE=y
CONFIG_UART_CONSOLE_INIT_PRIORITY=60
# CONFIG_UART_CONSOLE_DEBUG_SERVER_HOOKS is not set
# CONFIG_UART_CONSOLE_MCUMGR is not set
CONFIG_UART_CONSOLE_INPUT_EXPIRED=y
CONFIG_UART_CONSOLE_INPUT_EXPIRED_TIMEOUT=15000
# CONFIG_USB_UART_CONSOLE is not set
# CONFIG_RAM_CONSOLE is not set
# CONFIG_IPM_CONSOLE_SENDER is not set
# CONFIG_IPM_CONSOLE_RECEIVER is not set
# CONFIG_UART_PIPE is not set
# CONFIG_UART_MCUMGR is not set
# CONFIG_SEMIHOST_CONSOLE is not set
# CONFIG_UART_CONSOLE_LOG_LEVEL_OFF is not set
# CONFIG_UART_CONSOLE_LOG_LEVEL_ERR is not set
# CONFIG_UART_CONSOLE_LOG_LEVEL_WRN is not set
CONFIG_UART_CONSOLE_LOG_LEVEL_INF=y
# CONFIG_UART_CONSOLE_LOG_LEVEL_DBG is not set
CONFIG_UART_CONSOLE_LOG_LEVEL=3
# CONFIG_GSM_MUX is not set
CONFIG_HAS_SEGGER_RTT=y
...
CONFIG_UI_MODULE=y
CONFIG_UI_THREAD_STACK_SIZE=512
CONFIG_LED_CONTROL=y
CONFIG_LED_USE_PWM=y
CONFIG_LED_PWM_DEV_NAME="PWM_0"
CONFIG_LED_RED_PIN=29
CONFIG_LED_GREEN_PIN=30
CONFIG_LED_BLUE_PIN=31
# CONFIG_LED_CONTROL_LOG_LEVEL_OFF is not set
# CONFIG_LED_CONTROL_LOG_LEVEL_ERR is not set
# CONFIG_LED_CONTROL_LOG_LEVEL_WRN is not set
CONFIG_LED_CONTROL_LOG_LEVEL_INF=y
# CONFIG_LED_CONTROL_LOG_LEVEL_DBG is not set
CONFIG_LED_CONTROL_LOG_LEVEL=3

[jrdeck]

Settings used for RTT Viewer (says no USB device is connected, when pressing ok):

Output when using LTE Link Monitor:

[coderbyheart]

Converted issue to a discussion because it's not a bug in asset-tracker-cloud-firmware-aws.

[jrdeck]

Thanks, apologies for putting it in the wrong place!

[coderbyheart]

No worries!

[coderbyheart]

Just asking the obvious, on the same device using the same USB cable, the AT client produces UART output?

[coderbyheart]

Second Cpt. Obvious question: are you maybe building for the Thingy:91 but flashing it to the 9160DK?

[jrdeck]

Cable port etc all physically ok, as I can but it back into MCUboot and restore Asset Tracker v2. I have tried a couple cables just in case.

The device is a Thingy 91. Maybe I am doing the opposite?

Here is the build command, and the path of the HEX file I grab at the end.

docker run --rm -v ${PWD}:/workdir/ncs/firmware asset-tracker-firmware-docker /bin/bash -c 'cd /workdir/ncs/firmware; west build -p always -b thingy91_nrf9160ns -- -DOVERLAY_CONFIG="overlay-debug.conf;overlay-low-power.conf;asset-tracker-cloud-firmware-aws.conf"'
ls -la build/zephyr/merged.hex

[coderbyheart]

Looks good.

[coderbyheart]

Please post full

• prj.conf
• overlay-low-power.conf
• asset-tracker-cloud-firmware-aws.conf

[jrdeck]

prj.conf

#
# Copyright (c) 2021 Nordic Semiconductor ASA
#
# SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
#

# General config
CONFIG_NEWLIB_LIBC=y
CONFIG_NEWLIB_LIBC_FLOAT_PRINTF=y
CONFIG_ASSERT=y
CONFIG_REBOOT=y
CONFIG_RESET_ON_FATAL_ERROR=n
CONFIG_FPU=y

# Heap and stacks
CONFIG_HEAP_MEM_POOL_SIZE=32768
CONFIG_MAIN_STACK_SIZE=1280
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048
CONFIG_HW_STACK_PROTECTION=y

# Logging
CONFIG_LOG=y
CONFIG_LOG_MODE_IMMEDIATE=y

# DK - Used for buttons and LEDs in UI module.
CONFIG_DK_LIBRARY_INVERT_LEDS=n

# nRF modem library
CONFIG_NRF_MODEM_LIB=y

# AT Host library - Used to send AT commands directy from an UART terminal and to allow
#                   integration with nRF Connect for Desktop LTE Link monitor application.
CONFIG_AT_HOST_LIBRARY=y
CONFIG_UART_INTERRUPT_DRIVEN=y

# Network
CONFIG_NETWORKING=y
CONFIG_NET_NATIVE=n
CONFIG_NET_SOCKETS=y
CONFIG_NET_SOCKETS_OFFLOAD=y

# LTE link control
CONFIG_LTE_AUTO_INIT_AND_CONNECT=n
CONFIG_LTE_NETWORK_MODE_LTE_M_GPS=y
## Power saving timers.
### 320 hours PSM.
CONFIG_LTE_PSM_REQ_RPTAU="11000001"
### 6 seconds active time.
CONFIG_LTE_PSM_REQ_RAT="00000011"

# Cloud backend - To use a specific cloud backend uncomment the corresponding
#                 section.

# # AWS IoT
# ## Mandatory application specific configurations for AWS_IOT.
CONFIG_AWS_IOT=y
CONFIG_AWS_IOT_TOPIC_UPDATE_DELTA_SUBSCRIBE=y
CONFIG_AWS_IOT_TOPIC_GET_REJECTED_SUBSCRIBE=y
CONFIG_AWS_IOT_AUTO_DEVICE_SHADOW_REQUEST=y
CONFIG_AWS_IOT_MQTT_RX_TX_BUFFER_LEN=2048
CONFIG_AWS_IOT_MQTT_PAYLOAD_BUFFER_LEN=2048
CONFIG_AWS_IOT_APP_SUBSCRIPTION_LIST_COUNT=1
CONFIG_AWS_IOT_CLIENT_ID_APP=y
CONFIG_AWS_FOTA=y

# ## AWS IoT options that must be configured.
CONFIG_AWS_IOT_SEC_TAG=42
CONFIG_AWS_IOT_BROKER_HOST_NAME="xxxxxxxxxxxxxx-ats.iot.us-west-2.amazonaws.com"

# # Azure IoT Hub
# ## Mandatory application specific configurations for AZURE_IOT_HUB.
# CONFIG_AZURE_IOT_HUB=y
# CONFIG_AZURE_IOT_HUB_DPS=y
# CONFIG_AZURE_IOT_HUB_AUTO_DEVICE_TWIN_REQUEST=y
# CONFIG_AZURE_IOT_HUB_DEVICE_ID_APP=y
# CONFIG_AZURE_IOT_HUB_MQTT_PAYLOAD_BUFFER_LEN=2048
# CONFIG_AZURE_IOT_HUB_MQTT_RX_TX_BUFFER_LEN=2048
# CONFIG_AZURE_IOT_HUB_TOPIC_PROPERTY_BAG_PREFIX=n
# CONFIG_AZURE_IOT_HUB_STACK_SIZE=4096
# CONFIG_AZURE_FOTA=y
# CONFIG_AZURE_FOTA_TLS=y

# ## Azure IoT Hub options that must be configured.
# CONFIG_AZURE_IOT_HUB_DPS_HOSTNAME="global.azure-devices-provisioning.net"
# CONFIG_AZURE_IOT_HUB_DPS_ID_SCOPE=""
# CONFIG_AZURE_IOT_HUB_SEC_TAG=11
# CONFIG_AZURE_FOTA_SEC_TAG=11

# nRF Cloud
#CONFIG_NRF_CLOUD=y
#CONFIG_AGPS=y
#CONFIG_AGPS_SRC_NRF_CLOUD=y
#CONFIG_NRF_CLOUD_AGPS=y
## Because PGPS consumes a lot of data its disabled by default.
#CONFIG_NRF_CLOUD_PGPS=n
#CONFIG_NRF_CLOUD_CONNECTION_POLL_THREAD=y
#CONFIG_NRF_CLOUD_SEND_TIMEOUT=y
#CONFIG_GPS_MODULE_NMEA=y

# MQTT
# Keepalive should be set to the maximum specified MQTT keepalive timeout of the
#        selected cloud library.
#        1200 seconds for AWS IoT and nRF Cloud.
#        1767 seconds for Azure IoT Hub.
CONFIG_MQTT_KEEPALIVE=1200
CONFIG_MQTT_CLEAN_SESSION=y

# Settings - Used to store real-time device configuration to flash.
CONFIG_SETTINGS=y
CONFIG_SETTINGS_FCB=y
CONFIG_FCB=y

# FOTA - Needed by AWS_FOTA and AZURE_FOTA.
CONFIG_FOTA_DOWNLOAD=y
CONFIG_DFU_TARGET=y
CONFIG_DOWNLOAD_CLIENT=y
CONFIG_DOWNLOAD_CLIENT_HTTP_FRAG_SIZE_1024=y
CONFIG_DOWNLOAD_CLIENT_STACK_SIZE=4096
CONFIG_DOWNLOAD_CLIENT_BUF_SIZE=2300
CONFIG_DOWNLOAD_CLIENT_MAX_HOSTNAME_SIZE=128

# Flash - Used in FOTA, settings and storage for P-GPS.
CONFIG_FLASH=y
CONFIG_FLASH_PAGE_LAYOUT=y
CONFIG_FLASH_MAP=y
CONFIG_STREAM_FLASH=y
CONFIG_MPU_ALLOW_FLASH_WRITE=y

# MCUBOOT
CONFIG_BOOTLOADER_MCUBOOT=y
CONFIG_IMG_MANAGER=y
CONFIG_MCUBOOT_IMG_MANAGER=y
CONFIG_IMG_ERASE_PROGRESSIVELY=y

# Watchdog
CONFIG_WATCHDOG_APPLICATION=y

# A-GPS - SUPL
# To use A-GPS, a source of assistance data must be selected. By default, SUPL
# is selected. However, due to licensing, the SUPL library must be manually
# downloaded and configured according to the instructions found in the SUPL
# client library documentation in NCS.
# CONFIG_AGPS=y
# CONFIG_AGPS_SRC_SUPL=y

# Event Manager
CONFIG_EVENT_MANAGER=y
CONFIG_LINKER_ORPHAN_SECTION_PLACE=y
CONFIG_EVENT_MANAGER_LOG_EVENT_TYPE=n

# cJSON - Used in AWS FOTA and cloud data traffic encoding.
CONFIG_CJSON_LIB=y

overlay-low-power.conf

#
# Copyright (c) 2021 Nordic Semiconductor ASA
#
# SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
#

# This file contains configurations to achieve the lowest power consumption by
# disabling LEDs and logging over UART.

# Disable serial interface.
#CONFIG_SERIAL=n
CONFIG_SERIAL=y

# Disable LEDs
#CONFIG_LED_CONTROL=n
CONFIG_LED_CONTROL=y

# Disable UART console.
#CONFIG_UART_CONSOLE=n
CONFIG_UART_CONSOLE=y

# Disable event logging.
#CONFIG_EVENT_MANAGER_SHOW_EVENTS=n
CONFIG_EVENT_MANAGER_SHOW_EVENTS=y

# Disable AT host library.
#CONFIG_AT_HOST_LIBRARY=n
CONFIG_AT_HOST_LIBRARY=y

asset-tracker-cloud-firmware-aws.conf

# Project-specific tweaks for
# https://github.com/NordicSemiconductor/asset-tracker-cloud-firmware-aws

## Enable long download URLS for FOTA
CONFIG_DOWNLOAD_CLIENT_MAX_HOSTNAME_SIZE=255
CONFIG_DOWNLOAD_CLIENT_MAX_FILENAME_SIZE=255

AWS URL has been obfuscated.

Thanks for your help, @coderbyheart!

[coderbyheart]

Thanks, will try to reproduce with this config.
Which NCS version are you using?