[BUG] ESP32: esptool below 4.8 silently produces broken firmware.

[cederom]

Description / Steps to reproduce the issue

1. Compiler.

esp32-elf-cc --version
xtensa-esp32-elf-cc (crosstool-NG esp-2021r2-patch5) 8.4.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

2. With esptool 4.3 - cannot build firmware image - visible error:

% /usr/bin/time -h ./tools/configure.sh -B esp32-devkitc:nsh
        3,92s real              1,98s user              2,23s sys

% /usr/bin/time -h gmake -j8
 /usr/bin/time -h gmake -j8
Create version.h
Cloning Espressif HAL for 3rd Party Platforms
Clone: chip/esp-hal-3rdparty LN: platform/board to /XXX/nuttx-apps.git/platform/dummy
Register: nsh
Register: sh
Espressif HAL for 3rd Party Platforms: 20690e67695f0a8170a19ec99e2e9a13b620e94d
CPP:  /XXX/nuttx.git/arch/xtensa/src/chip/esp-hal-3rdparty/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld-> /XXX/nuttx.git/arch/xtensa/src/chip/esp-hal-3rdparty/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld.tmpLD: nuttx
CP: nuttx.hex
MKIMAGE: ESP32 binary
esptool.py -c esp32 elf2image --ram-only-header -fs 4MB -fm dio -ff 40m -o nuttx.bin nuttx
usage: esptool [-h] [--chip {auto,esp8266,esp32,esp32s2,esp32s3beta2,esp32s3,esp32c3,esp32c6beta,esp32h2beta1,esp32h2beta2,esp32c2,esp32c6}] [--port PORT] [--baud BAUD] [--before {default_reset,usb_reset,no_reset,no_reset_no_sync}] [--after {hard_reset,soft_reset,no_reset,no_reset_stub}] [--no-stub] [--trace]
               [--override-vddsdio [{1.8V,1.9V,OFF}]] [--connect-attempts CONNECT_ATTEMPTS]
               {load_ram,dump_mem,read_mem,write_mem,write_flash,run,image_info,make_image,elf2image,read_mac,chip_id,flash_id,read_flash_status,write_flash_status,read_flash,verify_flash,erase_flash,erase_region,merge_bin,get_security_info,version} ...
esptool: error: unrecognized arguments: --ram-only-header
gmake: *** [tools/Unix.mk:558: nuttx] Error 2
        40,96s real             1m32,23s user           1m1,82s sys

% pip show esptool
Name: esptool
Version: 4.3
Summary: A serial utility to communicate & flash code to Espressif chips.
Home-page: https://github.com/espressif/esptool/
Author: Fredrik Ahlberg (themadinventor) & Angus Gratton (projectgus) & Espressif Systems
Author-email:
License: GPLv2+
Location: /XXX/venv3.9embedded/lib/python3.9/site-packages
Requires: bitstring, cryptography, ecdsa, pyserial, reedsolo
Required-by: #N/A

3. With latest esptool 4.8.1 - all works fine.

(venv3.9embedded) pip install -U esptool

(venv3.9embedded) /usr/bin/time -h gmake -j8
CPP:  /XXX/nuttx.git/arch/xtensa/src/chip/esp-hal-3rdparty/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld-> /XXX/nuttx.git/arch/xtensa/src/chip/esp-hal-3rdparty/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld.tmpLD: nuttx
CP: nuttx.hex
MKIMAGE: ESP32 binary
esptool.py -c esp32 elf2image --ram-only-header -fs 4MB -fm dio -ff 40m -o nuttx.bin nuttx
esptool.py v4.8.1
Creating esp32 image...
Image has only RAM segments visible. ROM segments are hidden and SHA256 digest is not appended.
Merged 1 ELF section
Successfully created esp32 image.
Generated: nuttx.bin
        4,19s real              2,96s user              3,44s sys

(venv3.9embedded) /usr/bin/time -h gmake flash
        15,26s real             3,03s user              3,24s sys

(venv3.9embedded) cu -l /dev/cuaU0 -s 115200
Stale lock on cuaU0 PID=4016... overriding.
Connected
ts Jul 29 2019 12:21:46

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3ffb1f40,len:1388
ho 0 tail 12 room 4
load:0x40080000,len:20320
entry 0x40082650
*** Booting NuttX ***
dram: lma 0x00001020 vma 0x3ffb1f40 len 0x56c    (1388)
iram: lma 0x00001594 vma 0x40080000 len 0x4f60   (20320)
padd: lma 0x00006508 vma 0x00000000 len 0x9af0   (39664)
imap: lma 0x00010000 vma 0x400e0000 len 0x10d78  (68984)
padd: lma 0x00020d80 vma 0x00000000 len 0xf298   (62104)
dmap: lma 0x00030020 vma 0x3f400020 len 0x2848   (10312)
total segments stored 6

NuttShell (NSH) NuttX-12.7.0
nsh> uname -a
NuttX 12.7.0 10e44f8915-dirty Oct  4 2024 22:22:13 xtensa esp32-devkitc
nsh> free
                 total       used       free    maxused    maxfree  nused  nfree
      Umem:     323568       6424     317144       6800     186696     22      3

4. With esptool 4.7.0 (default system package on FreeBSD) - all seems fine, no errors, but firmware is broken.

% gmake clean distclean

% /usr/bin/time -h ./tools/configure.sh -B esp32-devkitc:nsh
        3,89s real              1,77s user              2,41s sys

% /usr/bin/time -h gmake -j8
Create version.h
Cloning Espressif HAL for 3rd Party Platforms
Clone: chip/esp-hal-3rdparty LN: platform/board to /XXX/nuttx-apps.git/platform/dummy
Register: nsh
Register: sh
Espressif HAL for 3rd Party Platforms: 20690e67695f0a8170a19ec99e2e9a13b620e94d
CPP:  /XXX/nuttx.git/arch/xtensa/src/chip/esp-hal-3rdparty/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld-> /XXX/nuttx.git/arch/xtensa/src/chip/esp-hal-3rdparty/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld.tmp
LD: nuttx
CP: nuttx.hex
MKIMAGE: ESP32 binary
esptool.py -c esp32 elf2image --ram-only-header -fs 4MB -fm dio -ff 40m -o nuttx.bin nuttx
esptool.py v4.7.0
Creating esp32 image...
ROM segments hidden - only RAM segments are visible to the ROM loader!
Merged 1 ELF section
Successfully created esp32 image.
Generated: nuttx.bin
        39,74s real             1m31,54s user           1m2,00s sys

%  /usr/bin/time -h gmake flash
CPP:  /XXX/nuttx.git/arch/xtensa/src/chip/esp-hal-3rdparty/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld-> /XXX/nuttx.git/arch/xtensa/src/chip/esp-hal-3rdparty/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld.tmpLD: nuttx
CP: nuttx.hex
MKIMAGE: ESP32 binary
esptool.py -c esp32 elf2image --ram-only-header -fs 4MB -fm dio -ff 40m -o nuttx.bin nuttx
esptool.py v4.7.0
Creating esp32 image...
ROM segments hidden - only RAM segments are visible to the ROM loader!
Merged 1 ELF section
Successfully created esp32 image.
Generated: nuttx.bin
esptool.py -c esp32 -p /dev/cuaU0 -b 115200  write_flash -fs detect -fm dio -ff 40m 0x1000 nuttx.bin
esptool.py v4.7.0
Serial port /dev/cuaU0
Connecting.........
Chip is ESP32-D0WD-V3 (revision v3.0)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: XXX
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 4MB
Flash will be erased from 0x00001000 to 0x00033fff...
Compressed 206984 bytes to 72429...
Wrote 206984 bytes (72429 compressed) at 0x00001000 in 6.7 seconds (effective 247.0 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
        15,25s real             3,07s user              3,17s sys


% cu -l /dev/cuaU0 -s 115200
Connected
ets Jul 29 2019 12:21:46

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3ffb1f40,len:1388
ho 0 tail 12 room 4
load:0x40080000,len:20320
entry 0x40082650
*** Booting NuttX ***
dram: lma 0x00001020 vma 0x3ffb1f40 len 0x56c    (1388)
iram: lma 0x00001594 vma 0x40080000 len 0x4f60   (20320)
padd: lma 0x00006508 vma 0x00000000 len 0xaaf0   (43760)
imap: lma 0x00011000 vma 0x400e0000 len 0x10d78  (68984)
padd: lma 0x00021d80 vma 0x00000000 len 0xf298   (62104)
dmap: lma 0x00031020 vma 0x3f400020 len 0x2848   (10312)
total segments stored 6

>>> NO NSH HERE DEVICE HANGS <<<

Lets try with the CoreMark:

% gmake clean distclean

% /usr/bin/time -h ./tools/configure.sh -B esp32-devkitc:coremark
        3,93s real              1,73s user              2,46s sys

% /usr/bin/time -h gmake -j8
        42,46s real             1m35,59s user           1m0,58s sys

%  /usr/bin/time -h gmake flash
        13,78s real             2,69s user              2,63s sys

% cu -l /dev/cuaU0 -s 115200
Connected
ets Jul 29 2019 12:21:46

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3ffb30a0,len:2016
load:0x40080000,len:30100
entry 0x40083940
*** Booting NuttX ***
dram: lma 0x00001020 vma 0x3ffb30a0 len 0x7e0    (2016)
iram: lma 0x00001808 vma 0x40080000 len 0x7594   (30100)
padd: lma 0x00008da8 vma 0x00000000 len 0x8250   (33360)
imap: lma 0x00011000 vma 0x400e0000 len 0xde14   (56852)
padd: lma 0x0001ee1c vma 0x00000000 len 0x21fc   (8700)
dmap: lma 0x00021020 vma 0x3f400020 len 0x14fc   (5372)
total segments stored 6

>>> COREMARK DOES NOT RUN <<<

I found this commit 457f9d3 to update the CI in order to build for ESP32 but manual builds should return error on older esptool versions or the older bootloader should be used? :-)

On which OS does this issue occur?

[OS: BSD]

What is the version of your OS?

13.3-RELEASE-p7 GENERIC amd64

NuttX Version

12.7.0-RC0

Issue Architecture

[Arch: xtensa]

Issue Area

[Area: Build System]

Verification

• I have verified before submitting the report.

[acassis]

@fdcavalcanti please take a look ^

[cederom]

Thanks @acassis, @fdcavalcanti this is quite important issue as it needs to be fixed in 12.7.0 release (it was found during 12.7.0-RC0 testing).

[fdcavalcanti]

I believe the proper way to proceed is using the most up-to-date esptool.
Let me check if there's any nuance we're missing.

[cederom]

• If version required is not present (i.e. 4.8 is required but 4.7 present) then error must be reported with build abort. So far no error is reported and silently the faulty image is built.
• We should be able to build the firmware image also with older version of esptool (use older bootloader?).

[fdcavalcanti]

@almir-okato

[jerpelea]

@cederom any update ?

[cederom]

Waiting for a fix proposition from @fdcavalcanti and @almir-okato of Espressif.

[cederom]

For sure we may leave current state as-is "using the most up-to-date esptool" then people with esptool <4.8 will get broken firmware without warning. Two impacts possible:

1. NuttX users that want to play with ESP32 will probably never come back to ESP32.
2. ESP32 users that want to play with NuttX will probably never come back to NuttX.

The simplest fix is just to enforce latest esptool and error build when older tool is in use. Not elegant nor compatible.

The best fix would be to adapt ESP32 bootloader to what esptool version is available (i.e up to -3 minor, or build error).

https://github.com/apache/nuttx/blob/master/INVIOLABLES.md

[jerpelea]

@cederom what do you think about adding a config option which is set to 4.8 but can be changed by the user?

[cederom]

@cederom what do you think about adding a config option which is set to 4.8 but can be changed by the user?

This part should be done automatically based on the available tool version with some log message on what happened (something like: esptool version X.Y implies bootloader version P.Q). Adding version to config will not change anything because that version needs to be checked anyway which is not done right now what causes the problem.

Do we know why exactly 4.8 esptool needs to be used for current NuttX master? Are there any specific features in the new bootloader that were not there before and we need them for some reason? Why older esptool creates invalid firmware image silently? @fdcavalcanti @almir-okato

I think we should bump this issue on esptool repo too maybe we get more details that way :-)