Geekatplay Studio — Vladimir Chopine
The web dashboard talks to the Freenove ESP32-S3 Media Kit over USB-CDC serial
(115200 baud, 8N1, DTR/RTS held low so opening the port does not reset the
board). Every command is an ASCII line terminated by \n. Responses are plain
log lines plus a small set of machine-parseable markers listed below.
| Command | Effect |
|---|---|
VERSION |
Emit [NANOAI_VERSION_RESP] {json} (see below) |
DIAG |
Emit diagnostics (SD state, PSRAM, heap, camera) |
LIST |
List SD card files (- File: /name.jpg (N bytes) per file) plus capacity summary |
SNAP |
Capture a photo. Saves /pictureN_normal.jpg at the sensor's native resolution (1600×1200 UXGA JPEG by default — Config → HiRes original) |
CONVERT (alias STYLIZE) |
Run the generative convert on the last still → /pictureN_styled.jpg (same N) |
SHOOT (alias SNAPSTYLE) |
One-shot: SNAP then CONVERT |
XL (alias XLCONV) |
XL-model convert of the last still: 4.3M-param NQ4 net from the xlmodel flash partition, ~70 s/frame, 480×480 styled export paired like CONVERT. Emits [PERF] xl_infer_ms=…. Needs a model flashed (training/flash_xl_model.ps1); replies with a hint if the partition is empty. |
TAP |
Simulate the BOOT button short-press (white LED torch toggle — not the shutter) |
HOLD |
Simulate the BOOT button long-press (also the torch — DEC-031) |
STYLE n |
Select style 0–15 (see style table) |
STYLES |
Emit [NANOAI_STYLES_RESP] {"count":16,"mask":M,"styles":[{"i":0,"name":"…","neural":true,"enabled":true},…]} — the style table + user selection |
STYLESEL m |
Set the user style selection bitmask (bit i = style i offered; persisted in NVS; No AI always kept enabled). Device menus and the dashboard row list only selected styles |
QUALITY n |
Set neural resolution: 32, 48, 64, 96, or 128 (persisted; higher = more recognizable, slower) |
ROTATE n |
Set camera rotation 0–3 (n×90°; 3 = 90° CCW default; persisted) |
MENU |
Debug: open the on-device main menu, as if joystick Center were held |
NAV UP|DOWN|LEFT|RIGHT|CENTER |
Debug: inject a joystick key press — drives menu/gallery navigation and the delete confirm without physical hardware access |
LCDSHOT |
Debug: JPEG-encode whatever the LCD is currently displaying and stream it as /lcd_debug.jpg — lets a display bug be seen off-device |
DUMP /path (alias GET /path) |
Stream file as base64 JPEG (≤512 KB; ~8.2 KB/s, so a full-res original takes ~10 s) |
DELETE (alias DEL) |
Delete the most recent normal+styled pair. To delete a specific gallery entry, use MENU → NAV CENTER to open Files, NAV LEFT/NAV RIGHT to browse to it, then NAV CENTER twice (arm, then confirm) — same as pressing Center on the physical joystick |
| Marker | Format |
|---|---|
| Version | [NANOAI_VERSION_RESP] {"fw":"…","creator":"Geekatplay Studio - Vladimir Chopine","nn_size":N,…} |
| Capture | [PHOTO_CAPTURED] /pictureN_normal.jpg style=S (style= field added Aug 2026; older firmware omits it) |
| SD ok | [SD CARD SUCCESS] |
| File row | - File: /pictureN_normal.jpg (12345 bytes) |
| Capacity | `Capacity: N MB Total |
| JPEG stream start | [PHOTO_JPEG_START] filename=/path bytes=N |
| JPEG payload | base64 lines, 60 chars per line |
| JPEG stream end | [PHOTO_JPEG_END] |
| Button echo | [BUTTON] STYLE SELECT / [BUTTON] CAMERA MODE / [BUTTON] Style → <name> |
| Style ack | [SERIAL] Style set to <n> |
| Index | Name | Engine |
|---|---|---|
| 0 | Pixel Art | Neural TinyML (int8 TFLite Micro), CPU fallback |
| 1 | Steampunk Vintage | CPU filter |
| 2 | 3D Relief Emboss | CPU filter |
| 3 | Clay / Ceramic | CPU filter |
| 4 | Oil Painting | CPU filter (kuwahara-style smoothing + edge darkening) |
| 5 | Victorian | CPU filter (sepia + vignette + soft plate grain) |
| 6 | Retro Film | CPU filter (faded curve + warm cast + grain) |
| 7 | Cartoon | CPU filter (bilateral-ish flatten + bold edges) |
| 8 | AI Enhance | Neural TinyML denoise (int8 TFLite Micro) |
| 9 | No AI / Normal JPEG | passthrough (always enabled) |
| 10 | Picasso Cubist | CPU generative (sheared facets, displaced sampling, bold banded palette, ink seams) |
| 11 | Van Gogh Swirl | CPU generative (vortex-swirled resampling, stroke bands, cobalt/chrome-yellow push) |
| 12 | Watercolor | CPU generative (pigment pooling, paper-white lift, wet-edge darkening) |
| 13 | Pencil Sketch | CPU generative (graphite edge lines + shadow cross-hatch on warm paper) |
| 14 | Pop Art Warhol | CPU generative (luma banded to 4 flat silkscreen inks + halftone dots) |
| 15 | Mosaic Glass | CPU generative (jittered flat tiles + dark grout) |
Styles 10–15 (and 0/7 at ≥95% strength) engage the pure-style compose: the
export crossfades away from the photo's luma channel, so the result reads as a
genuine generative restyling rather than a color filter over the photo.
Matching neural training targets exist for the pack:
python -m model.train --style picasso|van_gogh|watercolor|sketch|pop_art|mosaic.
The neural resolution (QUALITY) applies to styles 0 and 8; both models are
fully convolutional, so all five sizes share the same trained weights family
(one export per size, distinct symbols, all embedded in flash).
Times from [PERF] lines on a Freenove FNK0102A with an OV3660 sensor.
Inference scales as O(N²) exactly as the fully-convolutional topology predicts.
| Quality | Tensor arena | Inference | Compose | Styled export |
|---|---|---|---|---|
| 32 (Fast) | 160 KB (internal SRAM) | 1.49 s | 0.23 s | 320×320 |
| 48 | 240 KB (internal SRAM) | 3.37 s | 0.24 s | 320×320 |
| 64 (Balanced) | 352 KB (PSRAM) | 6.01 s | 0.24 s | 320×320 |
| 96 | 672 KB (PSRAM) | 13.54 s | 0.53 s | 480×480 |
| 128 (Best) | 1120 KB (PSRAM) | 24.08 s | 0.53 s | 480×480 |
CPU look filters (styles 1–7) cost 4–18 ms; only styles 0 and 8 run the network.
JPEG transfer over DUMP runs at ~8.2 KB/s (a 25 KB photo takes ~3.1 s).
- The
DUMPstream is non-blocking on the device (queue depth 2, CDC backpressure aware) and capped at 512 KB per file, PSRAM-backed. - The host should wait ~2.5 s after opening the port before sending the first
command (USB-CDC boot settle), then send
VERSIONfollowed byLIST. - Photos are saved as pairs sharing one picture number:
/pictureN_normal.jpg(1600×1200 UXGA original, sensor JPEG bytes written directly, no re-encode) and, if converted,/pictureN_styled.jpg(320×320 or 480×480 stylized export depending on neural quality — DEC-050). - For 90°/270° capture rotations the hi-res original carries an EXIF Orientation tag (6 = 90° CW, 8 = 270° CW) instead of rotated pixels: the sensor JPEG stays untouched, browsers rotate it automatically, and the on-device Files gallery parses the tag and rotates the decoded preview. This fixes originals appearing 90° clockwise in gallery/web preview while the LCD showed them correctly at capture time.
SNAPdoes not auto-stream the original preview (an 85 KB file would tie up the serial link for ~10 s after every shot); fetch it on demand withDUMPonce the gallery is opened.- The on-device Files gallery lists exactly what is on the card — newest shot first, original immediately followed by its styled export — with no synthetic "last shot" entry (DEC-049).