You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: examples/theremin/README.md
+72-61Lines changed: 72 additions & 61 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,80 +1,96 @@
1
1
# Theremin Simulator
2
2
3
-
The **Theremin Simulator** example lets you create and control a virtual theremin instrument using an interactive web interface, producing synthesized audio output through a connected **USB** audio device with minimal latency.
3
+
The **Theremin Simulator** example lets you create and control a virtual theremin instrument using an interactive web interface, producing synthesized audio output through a connected **USB** audio device with low latency.
4
4
5
5
> **Note**: This example must be run in **[Network Mode](learn/network-mode)** or **[SBC Mode](learn/single-board-computer)**, since it requires a **USB-C® hub** and a **USB speaker**.
This example generates real-time audio by creating sine waves at varying frequencies and amplitudes based on user input from the web interface. The workflow involves receiving mouse/touch coordinates from the frontend, calculating the corresponding frequency and amplitude, generating audio blocks using a sine wave generator, and playing them through a **USB** audio device with minimal latency.
10
-
9
+
This example generates real-time audio by creating sine waves at varying frequencies and amplitudes based on user input from the web interface. The workflow involves receiving mouse/touch coordinates from the frontend and updating a **Wave Generator** Brick, which handles the audio synthesis, smoothing, and streaming to the USB device automatically.
11
10
12
11
## Bricks Used
13
12
14
13
-`web_ui`: Brick that provides the web interface and a WebSocket channel for real-time control of the theremin.
15
-
-`wave_generator`: Brick that generates continuous audio waveforms and streams them to the USB speaker with smooth frequency and amplitude transitions.
16
-
14
+
-`wave_generator`: Brick that handles audio synthesis, envelope control (smoothing), and streaming to the USB audio device.
17
15
18
16
## Hardware and Software Requirements
19
17
20
18
### Hardware
21
19
22
20
-[Arduino UNO Q](https://store.arduino.cc/products/uno-q) (x1)
23
21
-**USB-C® hub with external power (x1)**
24
-
- A power supply (5 V, 3 A) for the USB hub adapter with external power (x1)
- A **power supply** (5 V, 3 A) for the USB hub (e.g. a phone charger)
30
28
31
-
> **Not supported:****HDMI audio and Bluetooth® Speakers**output is not supported by this App.
29
+
> **Not supported:****HDMI audio** and **Bluetooth® Speakers**are not supported by this App.
32
30
33
31
### Software
34
32
35
33
- Arduino App Lab
36
34
37
-
**Note:** A **USB-C® hub is mandatory** for this example. The UNO Q's single port must be used for the hub, which provides the necessary connections for both the power supply and the USB audio device. Consequently, this example must be run in **[Network Mode](learn/network-mode)** or **[SBC Mode](learn/single-board-computer)**.
38
-
35
+
**Note:** A **USB-C® hub is mandatory** for this example. The UNO Q's single port must be used for the hub, which provides the necessary connections for both the power supply and the USB audio device. consequently, this example must be run in **[Network Mode](learn/network-mode)** or **[SBC Mode](learn/single-board-computer)**.
39
36
40
37
## How to Use the Example
41
38
42
-
1. Connect your **USB audio device** (e.g., USB speaker, wireless USB receiver, or USB‑C→3.5 mm dongle) to a powered **USB-C® hub** attached to the UNO Q.
39
+
1.**Hardware Setup:**Connect your **USB audio device** (e.g., USB speaker, wireless USB receiver) to a powered **USB-C® hub** attached to the UNO Q.
2. Launchthe App by clicking the **Play** button in the top-right corner. Wait until the App has launched.
42
+
2.**Launch:** Launch the App by clicking the **Play** button in the top-right corner. Wait until the App has launched.
46
43
47
-
3. Open the App in your browser at `<UNO-Q-IP-ADDRESS>:7000`*(typically 192.168.x.x, e.g., http://192.168.1.11:7000)*.
48
-
4. Click and drag your mouse (or use touch) on the interactive area to play:
49
-
-**Horizontal movement (X-axis)** controls the **pitch** (frequency).
50
-
-**Vertical movement (Y-axis)** controls the **volume** (amplitude).
51
44
45
+
3.**Open Interface:** Open the App in your browser at `<UNO-Q-IP-ADDRESS>:7000`*(typically 192.168.x.x, e.g., http://192.168.1.11:7000)*.
46
+
47
+
### Interacting with the Interface
48
+
49
+
Once the web page loads, follow these steps to make sound:
50
+
51
+
1.**Turn on Power:** Locate the orange control panel at the bottom. Click the **POWER** switch to toggle it **ON** (the small LED indicator will light up).
52
+
**Note: No sound will be produced if this switch is OFF.*
53
+
2.**Set Master Volume:** Adjust the **VOL** knob to a comfortable level (e.g., 70-80%). This sets the maximum output limit for the application.
54
+
3.**Play the Instrument:** Drag your mouse (or use your finger on a touchscreen) inside the large gray background area:
55
+
***Horizontal (Left ↔ Right):** Controls **Pitch**. Moving right increases the frequency (higher notes).
56
+
***Vertical (Bottom ↕ Top):** Controls **Note Volume**. Moving up increases the amplitude (louder). Moving to the very bottom silences the note.
57
+
4.**Waveform Display:** The screen in the center of the panel visualizes the real-time sine wave, frequency (Hz), and amplitude data.
58
+
59
+
#### Additional Controls
60
+
***GRID Switch:** Toggles a background grid overlay. Use this to visually reference specific pitch intervals or volume levels.
52
61
53
62
## How it Works
54
63
55
-
The application creates a real-time audio synthesizer controlled by a web interface. User interactions on the webpage are sent to the Python backend via a WebSocket. The backend uses the `wave_generator` brick to continuously generate and stream audio to the connected **USB** audio device with smooth transitions.
64
+
The application uses the `wave_generator` Brick to create a continuous audio stream. The Python backend receives user coordinates via WebSocket and updates the generator's frequency and amplitude. The Brick automatically handles **envelope smoothing** (attack, release, glide) to ensure the audio changes sound natural and analog-like, rather than robotic.
56
65
57
-
-**User Interaction**: The frontend captures mouse or touch coordinates within a designated "play area".
58
-
-**Real-time Communication**: These coordinates are sent to the Python backend in real-time using the `web_ui` Brick's WebSocket channel.
59
-
-**Audio Synthesis**: The backend maps the X-coordinate to **frequency** and the Y-coordinate to **amplitude**, then updates the `wave_generator` brick's state. The brick handles smooth transitions using configurable envelope parameters (attack, release, glide).
60
-
-**Audio Output**: The `wave_generator` brick runs continuously in a background thread, generating audio blocks and streaming them to the **USB** audio device with minimal latency.
66
+
-**User Interaction**: The frontend captures mouse/touch coordinates and sends them to the backend.
67
+
-**Real-time Communication**: Input data is sent via the `web_ui` Brick's WebSocket channel.
68
+
-**Audio Synthesis**: The `wave_generator` Brick runs in the background. It takes the target frequency and amplitude and applies a **glide (portamento) algorithm** to transition smoothly between notes.
69
+
-**Audio Output**: The Brick streams the generated sine wave directly to the **USB** audio device.
61
70
62
-
High-level data flow:
71
+
**High-level data flow:**
63
72
```
64
-
Web Browser Interaction → WebSocket → Python Backend → WaveGenerator Brick → USB Audio Device Output
73
+
Web Browser Interaction ──► WebSocket ──► Python Backend
74
+
▲ │
75
+
│ ▼
76
+
(Visual Updates) (Glide & Synthesis)
77
+
│ │
78
+
└─ WebSocket ◄── State ◄── Sine Wave Generation
79
+
│
80
+
▼
81
+
USB Audio Output
65
82
```
66
83
67
84
68
85
## Understanding the Code
69
86
70
87
### 🔧 Backend (`main.py`)
71
88
72
-
The Python code manages the web server, handles real-time user input, and controls the audio generation brick.
89
+
The Python code is simplified by using the `WaveGenerator` Brick, which encapsulates the audio logic.
73
90
74
-
-`ui = WebUI()` – Initializes the web server that serves the HTML interface and handles WebSocket communication.
75
-
-`wave_gen = WaveGenerator(...)` – Creates the wave generator brick with configured envelope parameters (attack=0.01s, release=0.03s, glide=0.02s). The brick automatically manages the USB speaker connection and audio streaming in a background thread.
76
-
-`ui.on_message('theremin:move', on_move)` – Registers a handler that fires whenever the frontend sends new coordinates. This function updates the wave generator's frequency and amplitude using `wave_gen.set_frequency()` and `wave_gen.set_amplitude()`.
77
-
- The `wave_generator` brick handles all audio generation and streaming automatically, including smooth transitions between frequency and amplitude changes, continuous audio output with ~**30 ms** blocks, and non-blocking playback without cracks or pops.
91
+
-`wave_gen = WaveGenerator(...)` – Initializes the audio engine. It configures the **wave type** (sine), **sample rate** (16kHz), and **envelope parameters** (attack, release, glide). This Brick automatically connects to the USB audio device and starts a background thread for streaming.
92
+
-`ui.on_message('theremin:move', on_move)` – When the frontend sends new coordinates, this function calculates the target frequency and calls `wave_gen.set_frequency()` and `wave_gen.set_amplitude()`.
93
+
-`App.run()` – Starts the application. Since the audio generation is handled by the Brick in the background, no custom loop is required here.
78
94
79
95
### 💻 Frontend (`main.js`)
80
96
@@ -84,61 +100,56 @@ The web interface provides the interactive play area and controls for the user.
84
100
-**Event listeners** capture `mousedown`, `mousemove`, `mouseup` (and touch equivalents) to track user interaction in the play area.
85
101
-`socket.emit('theremin:move', { x, y })` – Sends normalized (0.0–1.0) X and Y coordinates to the backend; emissions are **throttled to ~80 Hz (≈12 ms)** to avoid overload.
86
102
-`socket.on('theremin:state', ...)` – Receives state updates from the backend (like the calculated frequency and amplitude) and updates the values displayed on the webpage.
87
-
-`socket.emit('theremin:set_volume', { volume })` – Sends a **0-100** hardware volume value to control the USB speaker's output level.
88
-
-`socket.emit('theremin:power', { on })` – Toggles synth power (**On/Off**). After turning **On**, move/tap in the play area to resume sound.
89
-
90
-
103
+
-**Visual Waveform Generation:** Using the amplitude and frequency data received from the backend, the frontend draws a real-time sine wave animation on the HTML Canvas, providing immediate visual feedback.
-`socket.emit('theremin:power', { on })` – Toggles synth power (**On/Off**).
91
106
92
107
## Troubleshooting
93
108
94
109
### "No USB speaker found" error
95
110
96
-
If the application fails to start and you see the following error in the logs, it means the required audio hardware is missing or not detected.
97
-
```
98
-
arduino.app_peripherals.speaker.SpeakerException: No USB speaker found.
99
-
```
100
-
**Fix:**
101
-
1. Make sure a **powered USB-C® hub** is connected to the UNO Q and its **5 V / 3 A** power supply is plugged in.
102
-
2. Verify the **USB audio device** (USB speaker, wireless USB receiver, or USB-C→3.5 mm dongle) is **connected to the hub** and, if it has a switch, **turned on**.
111
+
If the application fails to start and you see an error regarding the speaker (e.g., inside the `WaveGenerator` initialization), it means the required audio hardware is missing.
112
+
113
+
**Fix:**
114
+
1. Make sure a **powered USB-C® hub** is connected to the UNO Q and its **5 V / 3 A** power supply is plugged in.
115
+
2. Verify the **USB audio device** (USB speaker or wireless USB receiver) is **connected to the hub** and, if it has a switch, **turned on**.
103
116
3. Restart the application.
104
117
105
118
### No Sound Output
106
119
107
-
-**Power Button:** Make sure the button in the web UI shows **On**.
108
-
-**Volume Slider:**Increase the volume slider in the web UI.
109
-
-**Pointer Position:** Move your mouse/finger toward the top of the play area (the bottom corresponds to zero volume).
110
-
-**Speaker/Headphone Volume:** Check the physical volume control and mute status on your speaker or headphones.
120
+
-**Power Button:** Make sure the button in the web UI shows **On**.
121
+
-**Volume Knob:**Check the visual knob on the web UI.
122
+
-**Pointer Position:** Move your mouse/finger toward the top of the play area (the bottom corresponds to zero volume).
123
+
-**Speaker/Headphone Volume:** Check the physical volume control and mute status on your speaker or headphones.
111
124
-**Output Path:** Remember that **HDMI audio** and **Bluetooth® speakers** are not supported; use a **USB** audio device.
112
125
113
126
### Choppy or Crackling Audio
114
127
115
-
-**CPU Load:** Close any other applications running on the Arduino UNO Q that may be consuming significant resources.
128
+
-**CPU Load:** Close any other applications running on the Arduino UNO Q that may be consuming significant resources.
116
129
-**Power Supply:** Ensure you are using a stable, adequate power supply (5 V, 3 A) for the USB-C® hub, as insufficient power can affect USB peripheral performance.
117
130
118
-
119
-
120
131
## Technical Details
121
132
122
-
-**Sample rate:** 16,000 Hz
123
-
-**Audio format:** 32-bit float, little-endian
124
-
-**Block duration:**~30 ms (≈480 samples per block)
125
-
-**Frequency range:**~20 Hz to ~8,000 Hz
126
-
-**Update rate:** Frontend throttled to ~80 Hz (≈12 ms minimum between updates)
0 commit comments