tools: AMYboard World recorder (capture + beep-sync + stitch) + SysEx control API doc

[bwhitman]

What

A self-contained tool, tools/amyworld_recorder/, to audition and record every AMYboard World sketch for a video, plus a new doc for the AMYboard MIDI SysEx control protocol.

The recorder

For each World sketch: pull it from the World API → push it to a USB-connected AMYboard over the same SysEx write-to-sketch flow the web editor uses → audition it (random arp+chord patterns) → mark good / not interesting / not working / denylist → record clean audio → stitch the good ones into a captioned montage.

Highlights, most of them learned the hard way on real hardware:

• amyboard_link.py — a reusable Python client for the AMYboard SysEx protocol (transfer, run-python, reboot, error frames, notes).
• Clean audio via sounddevice/PortAudio, not ffmpeg — ffmpeg's avfoundation audio input slips samples against the device's USB clock (inaudible in silence, clicks on signal); PortAudio follows it exactly. Extracts a chosen channel pair (e.g. Model 16 ch 13/14).
• Watchdog: detects a sketch that hangs the board — on load or mid-audition — recovers with a pure-C zB reboot, and denylists it, with correct attribution (a delayed hang is blamed on the sketch that caused it, not the next one).
• OLED reset to the boot logo between sketches; optional amy.send(volume=…) boost.
• Gapless montage: PCM segments + a single AAC encode, with title/username/description overlays.

Two capture modes

1. On-Mac (Continuity/iPhone video muxed per clip) → stitch plan / stitch render.
2. External-video (--external-video) — the phone films the whole session (works around an iPhone shutter/exposure flicker on the OLED that can't be fixed through ffmpeg). A two-pulse sync beep plays at each sketch; stitch sync detects the beeps in the phone video(s), locates each sketch, and muxes in the clean Model 16 audio. Supports multiple videos across resumed runs.

Doc

docs/amyboard/control_api.md — documents the AMYboard MIDI SysEx control protocol (file transfer both directions, state dump, run-python, reboot, ACK/error/version frames) for anyone writing their own tools. Linked from the AMYboard docs index.

Notes

• Self-contained under tools/; no changes to firmware or build. Setup is a venv + requirements.txt (README included). Not wired into CI.
• .gitignore keeps the venv, per-user config.json, session_out/, and media out of git; config.example.json is the template.
• No submodule/pin changes.

🤖 Generated with Claude Code