You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: examples/theremin/README.md
+89-74Lines changed: 89 additions & 74 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,73 +2,79 @@
2
2
3
3
The **Theremin Simulator** example lets you create and control a virtual theremin instrument using an interactive web interface, producing synthesized audio output through a connected **USB** audio device with low latency.
4
4
5
-
> **Note**: This example must be run in**[Network Mode](learn/network-mode)** or **[SBC Mode](learn/single-board-computer)**, since it requires a **USB-C® hub** and a **USB speaker**.
5
+
**Note:** This example requires to be run using**[Network Mode](learn/network-mode)** or **[Single-Board Computer (SBC) Mode](learn/single-board-computer)**, since it requires a **USB-C® hub** and a **USB speaker**.
This example generates real-time audio by creating sine waves at varying frequencies and amplitudes based on user input from the web interface. The workflow involves receiving mouse/touch coordinates from the frontend and updating a **Wave Generator** Brick, which handles the audio synthesis, smoothing, and streaming to the USB device automatically.
9
+
This App creates a virtual instrument that generates real-time audio by creating sine waves at varying frequencies and amplitudes based on user input. The workflow involves receiving mouse or touch coordinates from the frontend and updating a **Wave Generator** Brick, which handles the audio synthesis, smoothing, and streaming to the **USB device** automatically.
10
+
11
+
Key features include:
12
+
13
+
- Real-time audio synthesis with low latency
14
+
- Interactive web interface for pitch and volume control
15
+
- Visual waveform display showing frequency and amplitude
16
+
- Automatic envelope smoothing (attack, release, glide) for natural sound
17
+
- Support for USB speakers and wireless USB audio receivers
10
18
11
19
## Bricks Used
12
20
21
+
The theremin simulator example uses the following Bricks:
22
+
13
23
-`web_ui`: Brick that provides the web interface and a WebSocket channel for real-time control of the theremin.
14
24
-`wave_generator`: Brick that handles audio synthesis, envelope control (smoothing), and streaming to the USB audio device.
15
25
16
26
## Hardware and Software Requirements
17
27
18
28
### Hardware
19
29
20
-
-[Arduino UNO Q](https://store.arduino.cc/products/uno-q) (x1)
- A **power supply** (5 V, 3 A) for the USB hub (e.g. a phone charger)
28
37
29
-
> **Not supported:****HDMI audio** and **Bluetooth® Speakers** are not supported by this App.
30
-
31
38
### Software
32
39
33
40
- Arduino App Lab
34
41
35
-
**Note:** A **USB-C® hub is mandatory** for this example. The UNO Q's single port must be used for the hub, which provides the necessary connections for both the power supply and the USB audio device. consequently, this example must be run in **[Network Mode](learn/network-mode)** or **[SBC Mode](learn/single-board-computer)**.
42
+
**Important:** A **USB-C® hub is mandatory** for this example. The UNO Q's single port must be used for the hub, which provides the necessary connections for both the power supply and the USB audio device. Consequently, this example must be run in **[Network Mode](learn/network-mode)** or **[SBC Mode](learn/single-board-computer)**.
43
+
44
+
**Note:****HDMI audio** and **Bluetooth® Speakers** are not supported by this App.
36
45
37
46
## How to Use the Example
38
47
39
-
1.**Hardware Setup:** Connect your **USB audio device** (e.g., USB speaker, wireless USB receiver) to a powered **USB-C® hub** attached to the UNO Q.
Connect your **USB audio device** (e.g., USB speaker, wireless USB receiver) to a powered **USB-C® hub** attached to the UNO Q. Ensure the hub is powered.
41
50
42
-
2.**Launch:** Launch the App by clicking the **Play** button in the top-right corner. Wait until the App has launched.
43
-
51
+
2.**Run the App**
52
+
Launch the App from Arduino App Lab. Wait until the App has launched completely.
44
53
45
-
3.**Open Interface:** Open the App in your browser at `<UNO-Q-IP-ADDRESS>:7000`*(typically 192.168.x.x, e.g., http://192.168.1.11:7000)*.
54
+
3.**Access the Web Interface**
55
+
Open the App in your browser at `<UNO-Q-IP-ADDRESS>:7000` (typically `192.168.x.x`).
46
56
47
-
### Interacting with the Interface
57
+
4.**Turn on Power**
58
+
Locate the orange control panel at the bottom of the interface. Click the **POWER** switch to toggle it **ON** (the small LED indicator will light up).
59
+
*Note: No sound will be produced if this switch is OFF.*
48
60
49
-
Once the web page loads, follow these steps to make sound:
61
+
5.**Set Master Volume**
62
+
Use the **+** and **-** buttons near the **VOL** indicator to adjust the master volume. This sets the maximum output limit for the application.
50
63
51
-
1.**Turn on Power:** Locate the orange control panel at the bottom. Click the **POWER** switch to toggle it **ON** (the small LED indicator will light up).
52
-
**Note: No sound will be produced if this switch is OFF.*
53
-
2.**Set Master Volume:** Adjust the **VOL** knob to a comfortable level (e.g., 70-80%). This sets the maximum output limit for the application.
54
-
3.**Play the Instrument:** Drag your mouse (or use your finger on a touchscreen) inside the large gray background area:
55
-
***Horizontal (Left ↔ Right):** Controls **Pitch**. Moving right increases the frequency (higher notes).
56
-
***Vertical (Bottom ↕ Top):** Controls **Note Volume**. Moving up increases the amplitude (louder). Moving to the very bottom silences the note.
57
-
4.**Waveform Display:** The screen in the center of the panel visualizes the real-time sine wave, frequency (Hz), and amplitude data.
64
+
6.**Play the Instrument**
65
+
Drag your mouse (or use your finger on a touchscreen) inside the large gray background area:
66
+
-**Horizontal (Left ↔ Right):** Controls **Pitch**. Moving right increases the frequency (higher notes).
67
+
-**Vertical (Bottom ↕ Top):** Controls **Note Volume**. Moving up increases the amplitude (louder). Moving to the very bottom silences the note.
58
68
59
-
#### Additional Controls
60
-
***GRID Switch:** Toggles a background grid overlay. Use this to visually reference specific pitch intervals or volume levels.
69
+
7.**Visualize Audio**
70
+
Observe the screen in the center of the panel, which visualizes the real-time sine wave, frequency (Hz), and amplitude data. You can also toggle the **GRID** switch to visually reference specific pitch intervals.
61
71
62
72
## How it Works
63
73
64
-
The application uses the `wave_generator` Brick to create a continuous audio stream. The Python backend receives user coordinates via WebSocket and updates the generator's frequency and amplitude. The Brick automatically handles **envelope smoothing** (attack, release, glide) to ensure the audio changes sound natural and analog-like, rather than robotic.
65
-
66
-
-**User Interaction**: The frontend captures mouse/touch coordinates and sends them to the backend.
67
-
-**Real-time Communication**: Input data is sent via the `web_ui` Brick's WebSocket channel.
68
-
-**Audio Synthesis**: The `wave_generator` Brick runs in the background. It takes the target frequency and amplitude and applies a **glide algorithm** to transition smoothly between notes.
69
-
-**Audio Output**: The Brick streams the generated sine wave directly to the **USB** audio device.
74
+
The application relies on a continuous data pipeline between the web interface and the audio synthesis engine.
70
75
71
76
**High-level data flow:**
77
+
72
78
```
73
79
Web Browser Interaction ──► WebSocket ──► Python Backend
-**User Interaction**: The frontend captures mouse/touch coordinates and sends them to the backend via the `web_ui` Brick's WebSocket channel.
91
+
-**Audio Synthesis**: The `wave_generator` Brick runs in the background. It takes the target frequency and amplitude and applies a **glide algorithm** to transition smoothly between notes.
92
+
-**Envelope Smoothing**: The Brick automatically handles attack, release, and glide to ensure the audio changes sound natural and analog-like, rather than robotic.
93
+
-**Audio Output**: The Brick streams the generated sine wave directly to the **USB** audio device.
84
94
85
95
## Understanding the Code
86
96
87
97
### 🔧 Backend (`main.py`)
88
98
89
-
The Python code is simplified by using the `WaveGenerator` Brick, which encapsulates the audio logic.
99
+
The Python script simplifies audio logic by utilizing the `WaveGenerator` Brick.
90
100
91
-
-`wave_gen = WaveGenerator(...)` – Initializes the audio engine. It configures the **wave type**(sine), **sample rate** (16kHz), and **envelope parameters** (attack, release, glide). This Brick automatically connects to the USB audio device and starts a background thread for streaming.
92
-
-`ui.on_message('theremin:move', on_move)` – When the frontend sends new coordinates, this function calculates the target frequency and calls `wave_gen.set_frequency()` and `wave_gen.set_amplitude()`.
93
-
-`App.run()` – Starts the application. Since the audio generation is handled by the Brick in the background, no custom loop is required here.
101
+
-**Initialization**: Configures the audio engine with specific parameters (sine wave, 16kHz sample rate) and envelope settings (attack, release, glide).
102
+
-**Frequency Calculation**: Maps the X-axis input (0.0 to 1.0) exponentially to a frequency range of 20 Hz to ~8000 Hz.
103
+
-**Event Handling**: Listens for `theremin:move` events from the frontend to update frequency and amplitude.
94
104
95
-
### 💻 Frontend (`main.js`)
105
+
```python
106
+
wave_gen = WaveGenerator(sample_rate=16000, ...)
96
107
97
-
The web interface provides the interactive play area and controls for the user.
108
+
def_freq_from_x(x):
109
+
# Exponential mapping from 20Hz up to Nyquist frequency
110
+
return20.0* ((SAMPLE_RATE/2.0/20.0) ** x)
98
111
99
-
-**Socket.IO connection** to the backend to send and receive data in real time.
100
-
-**Event listeners** capture `mousedown`, `mousemove`, `mouseup` (and touch equivalents) to track user interaction in the play area.
101
-
-`socket.emit('theremin:move', { x, y })` – Sends normalized (0.0–1.0) X and Y coordinates to the backend; emissions are **throttled to ~80 Hz (≈12 ms)** to avoid overload.
102
-
-`socket.on('theremin:state', ...)` – Receives state updates from the backend (like the calculated frequency and amplitude) and updates the values displayed on the webpage.
103
-
-**Visual Waveform Generation:** Using the amplitude and frequency data received from the backend, the frontend draws a real-time sine wave animation on the HTML Canvas, providing immediate visual feedback.
-`socket.emit('theremin:power', { on })` – Toggles synth power (**On/Off**).
112
+
defon_move(sid, data):
113
+
# Calculate target frequency and amplitude based on coordinates
114
+
freq = _freq_from_x(data.get("x"))
115
+
amp =max(0.0, min(1.0, 1.0-float(data.get("y"))))
116
+
117
+
wave_gen.set_frequency(freq)
118
+
wave_gen.set_amplitude(amp)
119
+
```
106
120
107
-
##Troubleshooting
121
+
### 🔧 Frontend (`main.js`)
108
122
109
-
### "No USB speaker found" error
123
+
The web interface handles user input and visualization.
110
124
111
-
If the application fails to start and you see an error regarding the speaker (e.g., inside the `WaveGenerator` initialization), it means the required audio hardware is missing.
125
+
-**Input Capture**: Event listeners track `mousemove`, `touchmove`, and `touchstart` to capture user interaction.
126
+
-**Throttling**: Emissions to the backend are throttled to approximately 80 Hz (~12 ms) to prevent network overload while maintaining responsiveness.
127
+
-**Visual Feedback**: The canvas draws a real-time sine wave animation based on the amplitude and frequency data received back from the server.
128
+
129
+
```javascript
130
+
// Send normalized coordinates (0.0 - 1.0) to backend
131
+
socket.emit('theremin:move', { x, y });
132
+
133
+
// Receive state for visualization
134
+
socket.on('theremin:state', (data) => {
135
+
updateStateDisplay(data.freq, data.amp);
136
+
});
137
+
```
112
138
139
+
## Troubleshooting
140
+
141
+
### "No USB speaker found" error
142
+
If the application fails to start and you see an error regarding the speaker:
113
143
**Fix:**
114
-
1.Make sure a **powered USB-C® hub** is connected to the UNO Q and its **5 V / 3 A** power supply is plugged in.
115
-
2. Verify the **USB audio device**(USB speaker or wireless USB receiver) is **connected to the hub** and, if it has a switch, **turned on**.
144
+
1.Ensure a **powered USB-C® hub** is connected to the UNO Q.
145
+
2. Verify the **USB audio device** is connected to the hub andturned on.
116
146
3. Restart the application.
117
147
118
148
### No Sound Output
119
-
120
-
-**Power Button:**Make sure the button in the web UI shows**On**.
121
-
-**Volume Knob:**Check the visual knob on the web UI.
122
-
-**Pointer Position:**Move your mouse/finger toward the top of the play area (the bottom corresponds to zero volume).
123
-
-**Speaker/Headphone Volume:** Check the physical volume control and mute status on your speaker or headphones.
124
-
-**Output Path:** Remember that **HDMI audio** and **Bluetooth® speakers** are not supported; use a **USB** audio device.
149
+
If the interface works but there is no sound:
150
+
-**Power Button:**Ensure the **POWER** switch in the web UI is**ON**.
151
+
-**Pointer Position:**Ensure you are interacting with the upper part of the play area (bottom is zero volume).
152
+
-**Volume Controls:**Increase the volume using the **+** button in the UI.
153
+
-**Hardware Volume:** Check the physical volume control on your speaker.
154
+
-**Audio Device:** Remember that **HDMI audio** and **Bluetooth® speakers** are not supported.
125
155
126
156
### Choppy or Crackling Audio
127
-
128
-
-**CPU Load:** Close any other applications running on the Arduino UNO Q that may be consuming significant resources.
129
-
-**Power Supply:** Ensure you are using a stable, adequate power supply (5 V, 3 A) for the USB-C® hub, as insufficient power can affect USB peripheral performance.
157
+
-**CPU Load:** Close other applications running on the Arduino UNO Q.
158
+
-**Power Supply:** Ensure you are using a stable 5 V, 3 A power supply for the USB-C® hub. Insufficient power often degrades USB audio performance.
0 commit comments