CustomBoot-32 blog

_posts/2025-08-20-CustomBoot-32.md

@@ -0,0 +1,181 @@
+---
+layout: post
+title: CustomBoot-32
+tags:
+  - Embedded C
+  - PCB Designing
+  - Bootloaders
+  - OTA
+  - UART
+description: Designing a custom PCB with OTA support and dual image Bootloader
+---
+
+---
+-- [Varun Patil](https://github.com/varun05050505/)
+
+--[Omkar Nanajkar](https://github.com/nomkar24)
+
+--[Archit More](https://github.com/avm1234567)
+
+
+## Progress Report (Weeks 1–2)
+
+During the **first week** of Eklavya, our focus was mainly on completing the prerequisites of the project, as we did not yet have access to the hardware. Most of this time was dedicated to brainstorming with the resources provided by our mentors. We referred to [this playlist](https://youtu.be/vaCVh2SAZY4?si=P55ba3Obu3_-rc_V), which explained the entire workflow of PCB design using KiCad. Additionally, we studied [this video](https://youtu.be/aVUqaB0IMh4?si=N4PDonKOkEUr_ZC0), which gave us insights into how real development boards, such as the **STM32 Blue Pill**, are designed in KiCad.
+
+In the **second week**, we transitioned to working on the actual schematics of our custom PCB. For this, we carefully referred to the following documents and datasheets:
+
+  -- [ESP32 datasheet](https://www.digikey.in/en/htmldatasheets/production/3267269/0/0/1/esp32-devkitc-32d)  
+  -- [STM32F103C8T6 datasheet](https://www.st.com/resource/en/datasheet/stm32f103c8.pdf)  
+  -- [CP2102 datasheet](https://www.digikey.in/htmldatasheets/production/121410/0/0/1/cp2102-gm.html?gclsrc=aw.ds&gad_source=1&gad_campaignid=146895304&gbraid=0AAAAADrbLlhJZZqjvv2UCZZ3iVlr4LYsd&gclid=Cj0KCQjw5JXFBhCrARIsAL1ckPsLeNYVgjHdsQhaNjdFA3oVTSbDaxx1qRKgEMt8qiXLX2qIpof5GuIaAlVaEALw_wcB)  
+  -- [AMS1117 datasheet](https://mm.digikey.com/Volume0/opasdata/d220001/medias/docus/5011/AMS1117.pdf)  
+
+<figure style="text-align: center;">
+  <div style="display: flex; justify-content: center; gap: 10px;">
+    <img src="/assets/posts/CustomBoot-32/sch1.png" alt="Control flow 1" style="width:45%;">
+    <img src="/assets/posts/CustomBoot-32/sch2.png" alt="Control flow 2" style="width:45%;">
+  </div>
+  <figcaption>Schematic</figcaption>
+</figure>
+
+
+
+## Progress report (Week 3-4)
+### OTA
+Once we completed the schematic, we received the hardware components, including the ESP and STM Blue Pill. We decided to begin with the software aspect of our project, focusing on **OTA firmware transfer from a laptop to the ESP**. To facilitate firmware storage on the ESP, we created a  [custom partition table](https://github.com/avm1234567/Customboot-32/blob/varun/OTA/partitions.csv) that divides the flash memory into multiple partitions. The final partition, labeled `"storage"`, is dedicated to the **file system**. We chose **SPIFFS** for this purpose because it is lightweight, easy to use, and ideal for small-scale file management on resource-constrained devices.
+
+For the file upload interface, we made [index.html](https://github.com/avm1234567/Customboot-32/blob/varun/OTA/spiffs/index.html) page that provides a simple form for uploading two firmware files. The ESP code performs several key functions: it connects to Wi-Fi, initializes SPIFFS on the storage partition, hosts the web interface on the assigned IP address, and handles the reception of binary files uploaded via the website.
+
+-- [OTA CODE](https://github.com/avm1234567/Customboot-32/blob/varun/OTA/main/main.c)
+
+<video controls width="600">
+  <source src="/assets/posts/CustomBoot-32/Website_rec.webm" type="video/webm">
+  Your browser does not support the video tag.
+</video>
+
+### UART
+UART communication between ESP and STM is the second important target after OTA in our project. Initially we tried to set communication between ESP and STM but it did not work. So we tested UART of ESP and STM individually using **Loopback Test.**
+Then we established duplex UART communication between both boards. All the codes are provided below:
+
+  -- [ESP Loopback Test](https://github.com/avm1234567/Customboot-32/tree/archit/esp_loopback_test)  
+  -- [ESP to STM one way UART](https://github.com/avm1234567/Customboot-32/tree/varun/ESP_STM_UART1)  
+  -- [ESP to STM two way UART](https://github.com/avm1234567/Customboot-32/tree/varun/ESP_STM_UART2)
+<figure style="width: 600px; margin: auto;">
+  <video controls width="600" preload="metadata" poster="/assets/posts/CustomBoot-32/Website_rec_poster.png">
+    <source src="/assets/posts/CustomBoot-32/Uart.mp4" type="video/mp4">
+    Your browser does not support the video tag.
+  </video>
+  <figcaption style="text-align: center; margin-top: 5px;">
+    UART two way communication between ESP and STM
+  </figcaption>
+</figure>
+
+### Simple Bootloader
+Before creating the final bootloader, we tested a simple Bootloader that simply checks **state of PA0 pin** and accordingly switches between two applications residing at two different starting adddresses.
+Refer [this](https://github.com/avm1234567/Customboot-32/tree/varun/Dual_Image_Bootloader) for full code along with test applications. Make sure you read the vector offsets of applications and flash them to its respective start address.
+<figure style="width: 600px; margin: auto;">
+  <video controls width="600" preload="metadata" poster="/assets/posts/CustomBoot-32/Website_rec_poster.png">
+    <source src="/assets/posts/CustomBoot-32/Dual_image_bootloader.mp4" type="video/mp4">
+    Your browser does not support the video tag.
+  </video>
+  <figcaption style="text-align: center; margin-top: 5px;">
+    Simple dual image bootloader
+  </figcaption>
+</figure>
+
+
+## Progress Report (Week 5-6)
+### Binary file transfer from ESP to STM using File Protocol
+During weeks 3–4, we successfully tested and established UART communication between the ESP and the STM. Our next objective was to transfer binary files from the ESP’s SPIFFS to the STM’s flash memory. To accomplish this, we designed a **Custom File Protocol.** Although we reviewed several existing file transfer protocols, we opted to develop our own in order to gain finer control over how data is transmitted via UART. For additional details, refer to our [Notion notes](https://www.notion.so/Esp-to-Stm-File-transfer-File-Protocol-2438c356155f8017bc71f075e02f5cac) on this topic.
+
+<div style="border: 2px solid rgb(242, 71, 132); padding: 10px; width: fit-content; margin: 10px auto; border-radius: 6px; text-align: center;">
+  <p><strong>Our Custom Chunk Structure (File Protocol):</strong></p>
+  <p>[START (1 byte)] &nbsp; [LENGTH (2 bytes)] &nbsp; [DATA (512 bytes)] &nbsp; [CRC32 (4 bytes)] &nbsp; [END (1 byte)]</p>
+</div>
+<figure style="width: 600px; margin: auto;">
+  <video controls width="600" preload="metadata" poster="/assets/posts/CustomBoot-32/Website_rec_poster.png">
+    <source src="/assets/posts/CustomBoot-32/FirmwareTranfer.mp4" type="video/mp4">
+    Your browser does not support the video tag.
+  </video>
+  <figcaption style="text-align: center; margin-top: 5px;">
+    Binary file transfer from Esp to STM
+  </figcaption>
+</figure>
+
+### Default Application
+To ensure system reliability, we reserved space in flash memory for a default application located below the main firmware. This application is a simple rapid LED blink program. Its purpose is to act as a fallback in case the user forgets to upload firmware to the ESP via OTA and directly requests the ESP to forward firmware to the STM. In such a scenario, the ESP has nothing valid to transmit, and the system could otherwise enter a hang state. To prevent this, the ESP instead sends a **fail-safe chunk** to the STM. The STM’s bootloader validates this chunk and, upon detection, executes the default application—providing a clear visual indication to the user that something went wrong.
+
+### CRC32
+**CRC32 (Cyclic Redundancy Check 32-bit)** is an error-detecting code commonly used to verify data integrity in digital communications and storage.  
+
+On the STM side, hardware CRC32 registers are available. When a 32-bit word is fed into the CRC32 register, it produces a checksum value. If another 32-bit word is subsequently provided without resetting the register, the checksum accumulates across both values. Using this mechanism, we feed the entire 512-byte chunk received from the ESP word by word. At the end of this process, the STM generates the CRC32 checksum for the entire chunk. Refer to [this section of the bootloader](https://github.com/avm1234567/Customboot-32/blob/a2e9b7c672e7306225e5a45dbd3d18ca81f498fd/ESP_STM_FILE_CRC/Bootloader_C8TX/src/main.c#L43) for the exact implementation.
+
+On the ESP side, hardware CRC32 is not available. Therefore, we implemented a **software-based CRC32 calculation**, which replicates the same operations performed by the STM’s hardware. You can find the complete implementation [here](https://github.com/avm1234567/Customboot-32/blob/a2e9b7c672e7306225e5a45dbd3d18ca81f498fd/ESP_STM_FILE_CRC/OTA/main/main.c#L269).
+
+As specified in our file protocol, the 4 bytes following the data field are reserved for the CRC32 value, which is transmitted from the ESP to the STM. The STM compares its computed checksum with the received one, and only if both values match does it send an **acknowledgement string** to the ESP, prompting the transfer of the next chunk.
+<figure style="width: 600px; margin: auto;">
+  <video controls width="600" preload="metadata" poster="/assets/posts/CustomBoot-32/Website_rec_poster.png">
+    <source src="/assets/posts/CustomBoot-32/CRC.mp4" type="video/mp4">
+    Your browser does not support the video tag.
+  </video>
+  <figcaption style="text-align: center; margin-top: 5px;">
+    CRC32 Implementation
+  </figcaption>
+</figure>
+
+### LibOpenCM3
+
+Our STM32F103C8T6 has only 64 KB of flash memory, making it essential for the bootloader to be highly optimized for size. The earlier version of our bootloader was written entirely using HAL libraries, which added unnecessary bloat, introduced slower execution, and caused the bootloader size to exceed 15 KB—about **25% of the total flash space!**
+
+To address this, we migrated the entire codebase from HAL to **LibOpenCM3**, which provides direct access to register-level operations while maintaining good code readability. This shift eliminated the HAL overhead, significantly reducing both size and complexity.  
+
+You can explore our LibOpenCM3-based bootloader [here](https://github.com/avm1234567/Customboot-32/tree/varun/ESP_STM_FILE_CRC/Bootloader_C8TX). Remarkably, this version of the bootloader is under **2 KB**, representing a massive improvement in efficiency.  
+
+For a more detailed explanation, refer to our [Notion notes on LibOpenCM3](https://www.notion.so/LibOpenCM3-2478c356155f80a6afcfc1611a41d326).
+
+
+## Progress Report (Week 7-8)
+### Perfboard Testing
+Our firmware is now complete, and the schematic has also been finalized. The next step was to test the circuit on a perfboard to verify its functionality. During this process, we discovered several errors in the schematic.
+
+### Issues Identified
+
+- **STM32 Powering Issue**  
+  When we first soldered the **SoC** onto the **breakout board**, it failed to connect to the **STM32 CubeProgrammer.**  
+  After troubleshooting, we realized that we had not provided power to **VDDA** and **VSSA** pins, assuming they were unnecessary since we were not using any analog features.  
+  In reality, these pins are essential for proper powering of the STM32, and their omission caused the device to power up incorrectly.
+
+- **ESP UART1 Pin Mapping**  
+  We also identified a mistake in the UART1 configuration of the ESP module, where the wrong GPIO pins had been assigned. After correcting this, communication was established successfully.
+
+### Successful Testing
+
+After fixing these issues, we tested our previously written code on the perfboard-based circuit, and it worked flawlessly.
+
+-- [PerfBoard Testing video link](https://drive.google.com/file/d/1SP9IA6iG8lHjdz1XjbkrW86w_i9Vo2Ad/view?usp=sharing)
+
+### PCB Routing and Gerber File Generation
+
+Another major step in preparing our PCB for fabrication was **routing the board** and generating the [Gerber files](https://drive.google.com/file/d/1XeakQWj5FKYVasDnnbstgLmBvl30qeD6/view?usp=drive_link). 
+To ensure compatibility with the manufacturer, we referred to the hardware design constraints specified on the [LionCircuits official website](https://www.lioncircuits.com/pcb-manufacturing-capabilities/rigid-pcb?utm_source=chatgpt.com).
+
+Additionally, we integrated **3D component models** into our KiCad project, which allowed us to review the board in **3D visualization mode.**  
+This step was particularly useful for verifying component placement, orientation, and overall mechanical fit before moving forward with fabrication.
+
+<br><br>
+
+
+<figure style="text-align: center;">
+  <div style="display: flex; justify-content: center; gap: 10px;">
+    <img src="/assets/posts/CustomBoot-32/PCBSC1.png" alt="Control flow 1" style="width:55%;">
+    <img src="/assets/posts/CustomBoot-32/PCBSC2.png" alt="Control flow 2" style="width:55%;">
+  </div>
+  <figcaption>3D front and back view</figcaption>
+</figure>
+
+
+
+## References:
+-- [Reference Project](https://github.com/SurajSonawane2415/USB-OTA-Bootloader-Integrated-Custom-STM-Board)<br>
+-- [BootLoader](https://embeddedinventor.com/embedded-bootloader-and-booting-process-explained/)<br>
+-- [OTA](https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-reference/system/esp_https_ota.html)<br>
+-- [STM32](https://www.youtube.com/watch?si=apB8K7ngO-cd8fAq&v=EaZuKRSvwdo&feature=youtu.be)<br>