Skip to content

GETTING_STARTED.md

Latest commit

 

History

History
130 lines (90 loc) · 6.1 KB

GETTING_STARTED.md

File metadata and controls

130 lines (90 loc) · 6.1 KB
Raw

Getting Started — NanoAi Studio

Practical step-by-step with why each step exists. For architecture math see ARCHITECTURE.md; for locked decisions see DECISION_LOG.md.

1. One-time environment

Python (training only lives under training/)

cd training
python -m pip install -r requirements.txt

Why NumPy 1.x: TensorFlow/Keras may import pandas → pyarrow. NumPy 2.x + an older pyarrow wheel fails with _ARRAY_API not found. Pins in requirements.txt keep NumPy <2 and OpenCV headless <4.12.

PlatformIO (firmware only lives under firmware/)

Install PlatformIO CLI or the VS Code/Cursor PlatformIO extension. Target envs are tiered by neural size: esp32s3 (default, N=32), esp32s3_48, esp32s3_64, esp32s3_96, esp32s3_128.

2. Train the TinyML model

cd training
python -m pytest tests/ -v
python -m model.train --style pixel_art --size 64

Why: Style 0 on the device is a real INT8 CNN embedded as firmware/include/models/model_nn_<N>.h, selected at build time by -DNANOAI_NN_SIZE via firmware/include/model_data.h (DEC-022). Training without flashing does nothing on hardware.

Other trainable targets (--style): picasso, van_gogh, watercolor, sketch, pop_art, mosaic — matching the CPU-generative styles 10–15 (DEC-053). --model enhance trains the AI Enhance (denoise) model for Style 8 instead. Prefer the dashboard's 🚀 Train Model button (below) over the raw CLI — it runs the same command but also copies the export to ui/models/ for the Model Test Lab.

Expect:

  • params=3347 (order of ~3.4k — the net is fully convolutional, so params are the same at every --size; only the tensor arena grows)
  • TFLite bytes=~13k (TFL3 OK)
  • Export path ending in firmware\include\models\model_nn_<N>.h

3. Flash firmware (serial port discipline)

Why uploads fail with Access denied: Windows serial is exclusive. Chrome WebSerial, pio device monitor, and upload cannot share COMx.

Checklist before upload:

  1. Web UI → Disconnect
  2. No Serial Monitor running
  3. Board visible in Device Manager (e.g. CH343 → COM6)
cd firmware
pio run -e esp32s3_64 -t upload

If still busy: unplug USB 2 seconds, plug in, retry.

4. First photo with AI style

  1. LED green = Live (camera streaming).
  2. Joystick Center tap → shutter (LED purple while saving, then a 3 s freeze preview, then back to Live).
  3. Joystick Center hold (≥700 ms) → main menu (Files / Nano Ai / Config); the camera pauses so the full 5-way ladder works (DEC-034).
  4. In Nano Ai, Up/Down picks a style from the ones currently enabled (out of 16 — see docs/SERIAL_PROTOCOL.md for the full table, and the dashboard's style-selection panel to change which are offered); Center/Right converts the last shot.
  5. BOOT is the white LED torch toggle, unrelated to the shutter (DEC-031).
  6. MicroSD should contain:
    • /pictureN_normal.jpg — original frame, sensor-resolution JPEG
    • /pictureN_styled.jpg — AI / look-filtered frame, once converted (Style 9 "No AI": only /pictureN_normal.jpg)

Why dual files: You can compare the unstyled photo and the TinyML/CPU look without re-shooting (DEC-016).

5. The dashboard (WebSerial + local training + Model Test Lab)

Preferred: serve the dashboard through the local training server so the page's Train Model button (S1) and Model Test Lab (S2c) work:

python training/dashboard_server.py

then open http://127.0.0.1:8137 → Connect → use LIST, SNAP, TAP, HOLD, VERSION, DIAG. The Train Model button runs the real python -m model.train pipeline locally (train → INT8 quantize → firmware header + ui/models/*.tflite), and the Model Test Lab runs the exact exported INT8 flatbuffer in the browser (TFLite WASM) on any image you upload.

Opening ui/index.html directly from disk still works for WebSerial, the style selection and the canvas previews, and the page shows an amber banner explaining the limits. Chrome forbids a file:// page from reading sibling files or calling a local server, so Train Model only prints the CLI command and the trained-model dropdown stays empty — but the Model Test Lab can still run a model you pick with “load .tflite from disk” (point it at ui/models/*.tflite).

Why disconnect before next upload: Same COM port lock as section 3.

Problem → cause → fix

Symptom Likely cause Fix
_ARRAY_API not found on train start NumPy 2 + old pyarrow pip install -r requirements.txt (forces NumPy 1.26.x)
Train noise but still exports Import recovered after error path Prefer clean NumPy 1.x; trust [EXPORT] Wrote ... line
Could not open COMx / Error 2 Port held (WebSerial) Disconnect UI / monitors; replug USB
Style 0 unchanged Placeholder models/model_nn_<N>.h for the flashed tier, or no re-flash Retrain with matching --size + upload matching -e esp32s3_<N> (or esp32s3 for N=32)
Preview in Serial, empty card SD mount/write fail Reseat SD; look for [PHOTO_CAPTURED] vs [PHOTO_PREVIEW_ONLY]
Joystick Center hold ignored / opens menu too eagerly Held < 700 ms, or >= 700 ms respectively Center tap = shutter, hold ≥700 ms = menu (DEC-034)
COM number changed USB re-enumeration Check Device Manager; pass new --upload-port
Dashboard console shows CORS errors; Train Model / Model Test Lab empty Page opened as file:// Serve it: python training/dashboard_server.py → http://127.0.0.1:8137
A style is missing from the device menu / BOOT cycle It's outside the current style_mask selection Dashboard → style-selection panel → tick it → Apply to Device, or send STYLESEL <mask>

Environment boundary (governance)

Allowed Forbidden
Python under training/ TensorFlow inside firmware/
C++ under firmware/ Training loops inside firmware
Shared files: include/models/model_nn_<N>.h (via model_data.h selector) Hand-edited fake model bytes for demos

Geekatplay Studio — Vladimir Chopine.