Skip to content

data_collection: OTA over SMP/UDP, and use the upstream RPi camera shield - #69

Merged
rosterloh merged 5 commits into
mainfrom
esp32p4-csi-imx219
Aug 11, 2026
Merged

data_collection: OTA over SMP/UDP, and use the upstream RPi camera shield#69
rosterloh merged 5 commits into
mainfrom
esp32p4-csi-imx219

Conversation

@rosterloh

Copy link
Copy Markdown
Owner

Draft: blocked on rosterloh/zephyr-drivers. west.yml pins rosterloh-drivers to main, and this app now needs the board's 360 MHz CPU fix, the corrected PSRAM size and the csi_* shield labels, plus the CSI receiver driver. CI will fail until rosterloh/zephyr-drivers#28 and the CSI PR merge and main advances. Mark ready after that.

MCUboot works on this board after all

The last attempt concluded that MCUboot could not run on the ESP32-P4-Nano because it is a rev v1.3 engineering sample — the ROM loaded the second-stage bootloader corrupt and it panicked with an illegal instruction before reaching the application, and this reproduced on Espressif's own esp32p4_function_ev_board.

That was an upstream software bug, not a silicon limitation, and Espressif fixed it:

adc3d53fd33  soc: esp32p4: Fix MCUboot RAM layout on rev 1.3   (Raffael Rostagno, Espressif)

  Pre-v3 P4 puts the 2nd-stage bootloader in low SRAM (below 0x4ff3abd0),
  not at the top of the app region like v3.x. [...] Fixes MCUboot startup
  crash on rev 1.3.

Corroborating: Espressif's in-tree waveshare_esp32p4_eth board also selects SOC_ESP32P4_REV_1_3 and defaults BOOTLOADER_MCUBOOT=y. MCUboot on pre-v3 P4 is upstream-supported now.

So the README section telling readers OTA was impossible on this hardware is deleted and OTA is enabled.

Revert-capable OTA, not just OTA

soc/espressif/Kconfig.sysbuild overrides the generic sysbuild default with default MCUBOOT_MODE_OVERWRITE_ONLY if SOC_FAMILY_ESPRESSIF_ESP32, so relying on defaults gives overwrite-only: the uploaded image replaces slot0 permanently, image test is indistinguishable from image confirm, and a bad image bricks the board until someone reflashes over USB. Wrong trade for a device whose purpose is being managed over the network, so sysbuild.conf selects swap-using-move explicitly.

Verified end to end on hardware

Over SMP/UDP against the board at a DHCP address, both paths:

Step Result
image upload (359932 B) ~78 KiB/s, 4 s
image test <hash> + reset slot0 new image active (unconfirmed), slot1 old confirmed
reset without confirming reverted to the old image, active confirmed
image confirm <hash> + reset new image active confirmed, persists

Boot chain:

I (soc_init): MCUboot 2nd stage bootloader
I (soc_init): chip revision: v1.3
I (boot): Loading image 0 - slot 0 from flash, area id: 2
*** Booting Zephyr OS build v4.4.0-10994-gcee159bb557d ***

Camera: use the upstream shield

Zephyr ships boards/shields/raspberry_pi_camera_module_2 for exactly the IMX219 module this board's CSI connector accepts, so the hand-rolled 61-line board overlay was reimplementing an upstream file. Deleted; the shield now supplies the imx219@10 node, endpoint wiring and chosen zephyr,camera. SHIELD is set in CMakeLists.txt rather than left to the caller, because the camera is part of this application and the poe app task has no --shield passthrough.

Two gotchas documented

  • mcumgr image confirm with no hash fails against this firmware with Error: 3 and changes nothing; passing the running image's hash works. The request is rejected at img_mgmt_state.c:833 before reaching the confirm logic.
  • The DHCP address is effectively never logged. main.c registers the NET_EVENT_IPV4_ADDR_ADD callback ~10.6 s in, but the lease lands just after link-up at ~2.7 s, so the event fires with nobody listening. Documented, not fixed — the fix is to also read the current address at startup.

Not fixed here

CSI capture still fails at runtime with Failed to dequeue frame (-11) / video_esp32_csi: DW-GDMA error status 0x40000000. Pre-existing, unchanged by this work, tracked separately.

🤖 Generated with Claude Code

rosterloh and others added 4 commits July 8, 2026 15:19
Wire the WaveShare ESP32-P4-Nano's IMX219 (RPi Camera v2) into data_collection
over MIPI CSI-2 using the new rosterloh-drivers video_esp32_csi receiver:

- Board overlay: imx219@10 on i2c0 (self-clocked module, 2 data lanes),
  CSI receiver endpoint, chosen zephyr,camera. Schematic-confirmed wiring.
- prj.conf: enable VIDEO + VIDEO_ESP32_CSI + the video shell, and PSRAM-backed
  frame buffers (shared multi-heap, external attribute) for RAW10 frames.
- main.c: set format, enqueue buffers, stream, dequeue one frame and log stats.

WIP: builds and flashes. Runtime capture not yet verified - blocked by a
board-level PSRAM early-boot hang and a pre-existing data_collection console
silence over the CH343 UART port (the original app is equally silent).
Requires the matching rosterloh-drivers branch esp32p4-csi-imx219.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MCUboot works on the rev v1.3 ESP32-P4-Nano. The corrupt-second-stage
crash that blocked it was an upstream software bug, fixed by zephyr
adc3d53fd33 "soc: esp32p4: Fix MCUboot RAM layout on rev 1.3" (pre-v3 P4
puts the bootloader in low SRAM, not at the top of the app region), and
Espressif's own rev-v1.3 waveshare_esp32p4_eth board now defaults to
MCUboot.

Enable the MCUmgr image group and its flash/slot plumbing, and rewrite
the README section that told the reader OTA was impossible on this
silicon. No sysbuild.conf or sysbuild/mcuboot.conf is needed: the board's
Kconfig.sysbuild already defaults BOOTLOADER_MCUBOOT and
BOOT_SIGNATURE_TYPE_NONE, Espressif's sysbuild Kconfig already defaults
MCUBOOT_MODE_SWAP_USING_MOVE, and sysbuild propagates
CONFIG_BOOTLOADER_MCUBOOT to this image. Deliberately no
sysbuild/mcuboot.overlay either - supplying one replaces MCUboot's own
app.overlay and silently drops its zephyr,code-partition = &boot_partition,
which makes the runner flash the bootloader into slot0.

Verified on hardware (16MB, MAC 80:f1:b2:d2:cf:1d, rev v1.3): ROM ->
MCUboot "Loading image 0 - slot 0" -> Zephyr banner and shell, with PSRAM
enumerated and the Ethernet PHY found. The image upload itself is still
untested: it needs a DHCP-served Ethernet link.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
soc/espressif/Kconfig.sysbuild overrides the generic sysbuild default
with "default MCUBOOT_MODE_OVERWRITE_ONLY if SOC_FAMILY_ESPRESSIF_ESP32",
so relying on defaults gave overwrite-only: the uploaded image replaces
slot0 permanently, "image test" is indistinguishable from "image confirm",
and a bad image bricks the board until someone reflashes it over USB. That
is the wrong trade for a device managed over the network, so select
swap-using-move explicitly.

Verified on hardware over SMP/UDP, both paths:

- upload -> test -> reset boots the new image with flags "active"
  (unconfirmed) and keeps the old one "confirmed" in slot1
- reset without confirming reverts to the old image
- confirm then reset makes the upgrade permanent

VERSION is 0.1.2, which is what the board now runs.

Two gotchas documented in the README. "mcumgr image confirm" with no hash
fails with Error: 3 and changes nothing; passing the running image's hash
works. And the DHCP address is effectively never logged, because main()
registers the NET_EVENT_IPV4_ADDR_ADD callback seconds after the lease
arrives - worth fixing separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Zephyr ships boards/shields/raspberry_pi_camera_module_2 for exactly the
IMX219 module on this board's CSI connector, so the hand-rolled 61-line
board overlay was reimplementing an upstream file. Delete it and let the
shield supply the imx219@10 node, the endpoint wiring and chosen
zephyr,camera; the board definition now carries the csi_* labels the
shield binds to.

SHIELD is set in CMakeLists.txt rather than left to the caller, because
the camera is part of this application, not an option - and the poe app
task has no --shield passthrough.

The overlay's speculative header comment goes with it: the wiring
questions it listed were settled from the schematic (SCCB is i2c0, the
module is self-clocked so the P4 drives no XCLK, and there is no
MCU-controlled reset), and the shield encodes the same conclusions.

Also correct the console description. That USB-C port is the on-board
USB-UART bridge on uart0, not the USB-Serial-JTAG; the USB-A connector is
a host port.

Verified on hardware: shield resolves imx219@10 onto i2c0, sensor answers,
app reads back 1640x1232. The pre-existing DW-GDMA capture error is
unchanged and tracked separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@rosterloh
rosterloh marked this pull request as ready for review August 11, 2026 12:02
@rosterloh

Copy link
Copy Markdown
Owner Author

Unblocked: rosterloh/zephyr-drivers#28 and #29 have both merged, so rosterloh-drivers main now carries the 360 MHz CPU fix, the corrected 32 MB PSRAM size, the csi_* shield labels and the CSI receiver driver.

Re-verified against the merged pins rather than local branches, after poe west-update (which also advanced Zephyr cee159bb5573a6406439c5):

  • poe app data_collection --sysbuild builds clean, shield picked up (-- Shield(s): raspberry_pi_camera_module_2)
  • flashed both domains and booted:
I (soc_init): MCUboot 2nd stage bootloader
I (boot): Loading image 0 - slot 0 from flash, area id: 2
I (hex_psram): density      : 0x07 (256 Mbit)
<inf> phy_mii: PHY (1) ID 2430C54
*** Booting Zephyr OS build v4.4.0-11199-g3a6406439c5a ***

data_collection was missing from the build matrix, so CI never compiled
it and a green tick on a PR touching only this app meant nothing. It is
built --sysbuild, the same way rasprover is, because that is the
configuration it ships in: --sysbuild is what produces MCUboot and the
signed, upgradeable image, and a plain build exercises neither.

pt_mcp is also absent from the matrix. Left alone here as unrelated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@rosterloh
rosterloh merged commit 243dfc4 into main Aug 11, 2026
15 checks passed
@rosterloh
rosterloh deleted the esp32p4-csi-imx219 branch August 11, 2026 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant