Music Animation

A web-based audio visualizer with Three.js animations that react to sound. Features multiple audio input modes including a lyrics player, system audio capture, and live coding with Strudel.

Features

• 🎵 Real-time audio visualization - 3D mesh animations react to audio frequencies
• 📝 Lyrics Mode - Play audio with synchronized lyrics display
• 🔊 System Audio Mode - Capture and visualize any audio playing on your system
• 🎹 Strudel Mode - Live code music patterns with Strudel (TidalCycles for the web)

Installation

# Clone the repository
git clone <repository-url>
cd music-animation

# Install dependencies
npm install

# Start development server
npm run dev

Open http://localhost:5173 in your browser.

Modes

Use the dropdown menu in the top-right corner to switch between modes.

1. Lyrics Mode (Default)

Play audio with synchronized lyrics displayed on screen.

Controls:

Action	Control
Play/Pause	Click the Play button
Seek	Click on the progress bar
Edit Lyrics	Click "Edit Lyrics" button

Lyrics Editor:

• Each line has a timestamp (in seconds) and the word/phrase
• Edit timestamps to sync with your audio
• Changes are saved in real-time
• Click "Close" to return to playback view

2. System Audio Mode

Capture and visualize any audio playing on your computer (Spotify, YouTube, etc.).

How to use:

1. Select "System Audio" from the dropdown
2. A browser prompt will appear to share your screen
3. Important: Check the "Share audio" checkbox
4. Select the tab or window with audio
5. The animation will react to the captured audio

Note: The captured audio is only used for visualization—it won't play through your speakers again.

3. Strudel Mode

Live code music patterns using Strudel—a JavaScript port of TidalCycles.

Keyboard Shortcuts:

Action	Shortcut
Play / Update pattern	Ctrl + Enter
Stop	Ctrl + .

Example Pattern:

s("bd sd [~ bd] sd, hh*16, misc")

How to run:

1. Select "Strudel" from the dropdown
2. Type your pattern in the editor
3. Press Ctrl + Enter to play
4. The animation reacts to the sound
5. Press Ctrl + . to stop

More examples:

// Simple beat
s("bd sd hh*4")

// With effects
s("bd sd").room(0.5).delay(0.3)

// Melodic pattern
note("c4 e4 g4 b4").s("piano")

Learn more at strudel.cc/learn

Tech Stack

• React + TypeScript
• Three.js - 3D graphics and shader-based visualization
• Vite - Fast development and build
• Strudel - Live coding audio patterns
• Web Audio API - Audio analysis and processing

Project Structure

src/
├── components/
│   ├── ThreeCanvas.tsx      # 3D visualization
│   ├── LyricsOverlay.tsx    # Lyrics display
│   ├── PlayerControls.tsx   # Audio player controls
│   ├── LyricEditor.tsx      # Lyrics timing editor
│   └── StrudelOverlay.tsx   # Strudel live coding editor
├── logic/
│   ├── AudioManager.ts      # Audio handling and analysis
│   └── Visuals.ts           # Shader-based visuals
├── context/
│   └── AudioContext.tsx     # React context for audio state
└── App.tsx                  # Main application

License

AGPL-3.0 (due to Strudel integration)