Skip to content

Commit 0c23373

Browse files
committed
update wording, add CLion, drop MCP not used in this experiment
1 parent f631f17 commit 0c23373

File tree

1 file changed

+51
-45
lines changed
  • content/blog/2025/11/debugging-embedded-graphics-with-wokwi-and-ai

1 file changed

+51
-45
lines changed

content/blog/2025/11/debugging-embedded-graphics-with-wokwi-and-ai/index.md

Lines changed: 51 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -17,14 +17,16 @@ Raylib is a popular game programming library that recently gained software rende
1717

1818
The initial port compiled and ran, but there was a problem: instead of showing solid colors, the display showed vertical yellow/blue stripes. The framebuffer format was clearly wrong, but determining exactly where and how required multiple iterations.
1919

20-
## Enter Wokwi: Hardware Simulation in CI/CD
20+
## Enter Wokwi: Hardware Simulation for Bug Hunting
2121

22-
Rather than flash to physical hardware for every test, we integrated Wokwi - a hardware simulator that can emulate ESP32 boards with displays. This gave us several advantages:
22+
Rather than flash to physical hardware for every test, we used Wokwi - a powerful hardware simulator that can emulate ESP32 boards with displays. Wokwi is particularly valuable for bug hunting in embedded graphics applications because it provides:
2323

24-
1. **Fast iteration cycles**: No need to physically flash devices
25-
2. **Screenshot capability**: Wokwi can capture display output as PNG files
26-
3. **Reproducible testing**: Same simulation conditions every time
27-
4. **CI/CD integration**: Automated testing in GitHub Actions
24+
1. **Screenshot Capability**: Capture display output as PNG files at any point in execution - crucial for visual debugging
25+
2. **Integrated Debugger**: Step through code, set breakpoints, and inspect variables without physical hardware
26+
3. **Fast Iteration Cycles**: No need to physically flash devices between tests
27+
4. **Reproducible Testing**: Same simulation conditions every time, eliminating hardware variability
28+
5. **IDE Integration**: Works seamlessly with VSCode and CLion for professional development workflows
29+
6. **CI/CD Ready**: Automated testing in GitHub Actions with screenshot validation
2830

2931
Our Wokwi setup included:
3032
- ESP32-S3-BOX-3 board definition (320x240 ILI9341 display)
@@ -47,7 +49,7 @@ Here's where it got interesting. Instead of manually analyzing each screenshot a
4749

4850
### 1. Initial Problem Identification
4951

50-
**Human**: "The screen should have a red background, but as you can see, it's not true. It has some kind of Yellow, blue, vertical line patterns."
52+
**Developer**: "The screen should have a red background, but as you can see, it's not true. It has some kind of Yellow, blue, vertical line patterns."
5153

5254
**AI Action**: Read the screenshot, identified the vertical stripe pattern indicating likely color format mismatch or byte-order issues.
5355

@@ -126,60 +128,64 @@ This was then integrated into the CI pipeline using ImageMagick.
126128
Here's what a typical debugging iteration looked like:
127129

128130
```text path=null start=null
129-
┌─────────────────────────────────────────────────┐
130-
│ 1. Human describes problem + shares screenshot
131-
└────────────────┬────────────────────────────────┘
131+
┌─────────────────────────────────────────────────────
132+
│ 1. Developer describes problem + shares screenshot │
133+
└────────────────┬────────────────────────────────────
132134
133-
┌────────────────▼────────────────────────────────┐
134-
│ 2. AI analyzes code + screenshot via vision API │
135-
│ - Identifies vertical stripes pattern │
136-
│ - Reviews framebuffer handling code │
137-
│ - Checks RGB565 format documentation │
138-
└────────────────┬────────────────────────────────┘
135+
┌────────────────▼────────────────────────────────────
136+
│ 2. AI analyzes code + screenshot via vision API
137+
│ - Identifies vertical stripes pattern
138+
│ - Reviews framebuffer handling code
139+
│ - Checks RGB565 format documentation
140+
└────────────────┬────────────────────────────────────
139141
140-
┌────────────────▼────────────────────────────────┐
141-
│ 3. AI proposes code changes │
142-
│ - Direct framebuffer access │
143-
│ - Byte swapping for endianness │
144-
└────────────────┬────────────────────────────────┘
142+
┌────────────────▼────────────────────────────────────
143+
│ 3. AI proposes code changes
144+
│ - Direct framebuffer access
145+
│ - Byte swapping for endianness
146+
└────────────────┬────────────────────────────────────
145147
146-
┌────────────────▼────────────────────────────────┐
147-
│ 4. Build and simulate │
148-
│ $ idf.py build │
149-
│ $ wokwi-cli --screenshot-time 5000 ... │
150-
└────────────────┬────────────────────────────────┘
148+
┌────────────────▼────────────────────────────────────┐
149+
│ 4. Build and simulate in Wokwi │
150+
│ $ idf.py build │
151+
│ $ wokwi-cli --screenshot-time 5000 ... │
152+
│ [Wokwi captures display screenshot] │
153+
└────────────────┬────────────────────────────────────┘
151154
152-
┌────────────────▼────────────────────────────────┐
153-
│ 5. AI reads new screenshot │
154-
│ tool: read_any_files(screenshot.png) │
155-
│ Result: Solid green color ✅ │
156-
└─────────────────────────────────────────────────┘
155+
┌────────────────▼────────────────────────────────────
156+
│ 5. AI reads new screenshot
157+
│ tool: read_any_files(screenshot.png)
158+
│ Result: Solid green color ✅
159+
└────────────────────────────────────────────────────
157160
```
158161

159162
## Key Insights
160163

161164
### Why This Worked Well
162165

163-
1. **Visual Feedback is Critical**: Color bugs can't be debugged with logs alone - you need to see the actual output
164-
2. **Wokwi's Consistency**: Same simulation every time meant we could isolate variables
165-
3. **AI Vision + Code Understanding**: The AI could correlate visual output with code changes
166-
4. **Fast Iteration**: ~2 minutes per cycle (build + simulate + analyze) vs. hours with physical hardware
167-
5. **Executable Investigation**: The AI could directly build, simulate, and capture screenshots
166+
1. **Visual Feedback is Critical**: Color bugs can't be debugged with logs alone - you need to see the actual output. Wokwi's screenshot capability made this trivial
167+
2. **Wokwi's Debugging Power**: The integrated debugger would have allowed stepping through framebuffer operations if screenshots weren't sufficient
168+
3. **Wokwi's Consistency**: Same simulation every time meant we could isolate variables and prove fixes definitively
169+
4. **AI Vision + Code Understanding**: The AI could correlate visual output with code changes by reading Wokwi screenshots
170+
5. **Fast Iteration**: ~2 minutes per cycle (build + simulate + analyze) vs. hours with physical hardware
171+
6. **Professional Development Environment**: Wokwi's VSCode and CLion integration meant we could debug just like with physical hardware
168172

169173
### Limitations We Hit
170174

171-
1. **MCP Connection Drops**: Wokwi MCP server would occasionally disconnect, requiring restart
172-
2. **Screenshot Timing**: Had to carefully time captures to match test sequence (red at 2.7s, green at 3.9s, etc.)
173-
3. **Endianness Assumptions**: Even AI made wrong guesses about byte order initially
175+
1. **Screenshot Timing**: Had to carefully time captures to match test sequence (red at 2.7s, green at 3.9s, etc.) - though Wokwi's debugger could have helped pause at exact moments
176+
2. **Endianness Assumptions**: Even AI made wrong guesses about byte order initially, requiring multiple iterations
174177

175178
## Lessons for Embedded Development
176179

177-
### 1. Simulation-First Development
180+
### 1. Simulation-First Development with Wokwi
178181

179-
Don't wait until hardware is available. Tools like Wokwi let you:
180-
- Test display output without physical devices
181-
- Capture exact visual state for comparison
182-
- Integrate visual testing into CI/CD
182+
Don't wait until hardware is available. Wokwi provides a complete development environment:
183+
- **Screenshot capture** for visual debugging - essential for graphics bugs
184+
- **Integrated debugger** with breakpoints and variable inspection
185+
- **IDE integration** with VSCode and CLion for professional workflows
186+
- **Test display output** without physical devices
187+
- **Capture exact visual state** for comparison and regression testing
188+
- **CI/CD integration** for automated visual testing
183189

184190
### 2. AI as Debugging Partner
185191

@@ -196,7 +202,7 @@ The key was having a repeatable process:
196202
Observe → Hypothesize → Code → Build → Simulate → Capture → Verify → Repeat
197203
```
198204

199-
Each step was automated or AI-assisted, keeping human focus on problem-solving.
205+
Each step was automated or AI-assisted, keeping developer focus on problem-solving. Wokwi's screenshot capability was the key enabler - without it, we'd need physical hardware and manual camera captures for each test.
200206

201207
## The Complete Solution
202208

0 commit comments

Comments
 (0)