Assessment: Arduino to ESP-IDF migration scope exceeds feasible automated refactoring

[Copilot]

The issue requests complete removal of Arduino framework dependency and migration to native ESP-IDF. This is a multi-month architectural refactor affecting the entire codebase.

Analysis

Current state:

• Hybrid Arduino + ESP-IDF build (platformio.ini line 22: framework = arduino, espidf)
• 8+ core files depend on Arduino.h
• Deep integration: WiFi.h, Preferences.h, LittleFS, HardwareSerial
• Third-party Arduino libraries: TMCStepper, FastAccelStepper, ArduinoWebsockets, ArduinoJson

Required work:

• Rewrite all peripheral drivers (GPIO, UART, I2C, SPI, ADC) using ESP-IDF APIs
• Replace Arduino networking stack with esp_wifi.h + lwIP
• Migrate NVS from Preferences.h to nvs_flash.h/nvs.h
• Port or replace 4 major Arduino libraries
• Convert PlatformIO build to pure ESP-IDF CMake
• Rewrite stepper motor control without Arduino abstractions
• Convert all FreeRTOS task management from hidden loop() to explicit tasks

Estimation: 3-6 months engineering effort, ~100% codebase modification.

Recommendation

This cannot be accomplished as an automated minimal-change refactor. Requires phased migration:

1. Establish parallel ESP-IDF build system
2. Migrate subsystems incrementally (BLE → WiFi → stepper → filesystem)
3. Integration testing between phases
4. Remove Arduino once all components ported

Architecture decisions, library ports, and extensive validation require human expertise and project management.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[FR] Architectural Refactor:</issue_title>
<issue_description>Migrate from Arduino Core to native ESP-IDF

This issue proposes a major architectural refactor: to remove the dependency on the Arduino framework and migrate the entire SmartSpin2k project to the native Espressif IoT Development Framework (ESP-IDF).

Justification
While the Arduino framework is excellent for rapid prototyping and ease of use, migrating to native ESP-IDF offers several key advantages:

• Performance: Direct access to hardware and FreeRTOS for more optimized performance and real-time control.

• Control: Fine-grained control over system components, peripherals, and power management.
• Features: Full and immediate access to the latest features, bug fixes, and drivers provided by Espressif, which are not always exposed through the Arduino abstraction layer.
• Stability: Removes a layer of abstraction, which can simplify debugging and improve long-term stability.

Scope of Work
This is a large task that will touch most of the codebase. The work can be broken down into the following key areas:

1. Build System & Project Structure:
   • [ ] Reconfigure the project to be built using the ESP-IDF build system (CMake / idf.py).
   • [ ] Update or replace any CI/CD scripts (e.g., GitHub Actions) to use the new idf.py build and flash commands.
   • [ ] Manage all components and third-party libraries using the ESP-IDF component manager or as local CMake components.
2. Core Application Refactoring:
   • [ ] Replace all Arduino Core includes (e.g., Arduino.h, WiFi.h, Wire.h) with their native ESP-IDF equivalents (e.g., freertos/FreeRTOS.h, esp_wifi.h, driver/gpio.h, driver/i2c.h, etc.).
   • [ ] Refactor peripheral initialization and management (GPIO, I2C, SPI, ADC) from Arduino functions (digitalWrite, analogRead) to the corresponding ESP-IDF drivers.
   • [V] Explicitly manage FreeRTOS tasks for application logic, replacing the hidden loop() task.
   • [ ] Migrate NVS (Non-Volatile Storage) access from Preferences.h to the native nvs_flash.h and nvs.h APIs.
3. Library & Dependency Migration:
   • [ ] Identify all third-party libraries currently used (e.g., for BLE, Web Server, display).
   • [ ] Find and integrate ESP-IDF-compatible replacements for these libraries.
   • [ ] If no replacement exists, the library's functionality will need to be ported or re-implemented using native ESP-IDF APIs.
4. Testing & Validation:
   • [ ] Ensure the project compiles successfully using the ESP-IDF build system.
   • [ ] Perform comprehensive testing to verify all original SmartSpin2k functionality is preserved and works as expected:
   • [ ] BLE connectivity and services.
   • [ ] Web UI and server functionality.
   • [ ] Hardware interactions (display, buttons, sensors).
   • [ ] Firmware upgrade process (if it's also being refactored).
5. Documentation:
   • [ ] Update the README.md and any developer documentation with the new build instructions, dependencies, and setup requirements (e.g., installing the ESP-IDF toolchain).
   Acceptance Criteria
   • The project builds cleanly with the idf.py build command.
   • The Arduino framework is completely removed as a dependency.
   • All features of the last Arduino-based release are fully functional in the new ESP-IDF version.</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes [FR] Architectural Refactor: #693

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[doudar]

@copilot , let's then create sub issues under the main issue which work these problems one at a time.

[Copilot]

@copilot , let's then create sub issues under the main issue which work these problems one at a time.

I don't have the capability to create GitHub issues directly. However, I can provide a detailed breakdown of sub-issues that you can create. Here's a suggested structure:

Phase 1: Build System Setup

• Issue: Set up pure ESP-IDF build system alongside PlatformIO
• Issue: Configure ESP-IDF CMakeLists.txt for all components
• Issue: Update CI/CD to support ESP-IDF builds

Phase 2: Core Dependencies

• Issue: Migrate NVS storage from Preferences.h to nvs_flash.h/nvs.h
• Issue: Replace LittleFS Arduino wrapper with ESP-IDF VFS
• Issue: Convert HardwareSerial to ESP-IDF UART driver

Phase 3: Networking Stack

• Issue: Replace WiFi.h with esp_wifi.h and esp_netif
• Issue: Migrate HTTP server from Arduino WebServer to ESP-IDF httpd
• Issue: Port ArduinoWebsockets to ESP-IDF WebSocket server

Phase 4: BLE Stack

• Issue: Replace NimBLE-Arduino with native ESP-IDF NimBLE component
• Issue: Migrate BLE services (FTMS, CPS, CSC, Heart Rate, etc.)

Phase 5: Hardware Control

• Issue: Port TMCStepper library to ESP-IDF UART
• Issue: Port FastAccelStepper to ESP-IDF timer/GPIO
• Issue: Convert peripheral init (GPIO, ADC) to ESP-IDF drivers

Phase 6: Testing & Validation

• Issue: Validate BLE connectivity and services
• Issue: Test web UI functionality
• Issue: Hardware integration testing

Phase 7: Cleanup

• Issue: Remove Arduino framework dependency
• Issue: Update documentation for ESP-IDF build

Would you like me to format this as markdown you can copy-paste when creating the issues?