WIP: Final Report, GSOC 2025

content/news/XXXX-XX-XX-gsoc-2025-report-armaan-chowfin.md

@@ -0,0 +1,45 @@
+title: GSoC 2025 Work Product - Resampling Options for Mixxx
+authors: Armaan Chowfin
+status: draft
+tags: gsoc, gsoc-2025
+comments: yes
+
+#### Introduction
+Compressions and rarefactions of the air surrounding us cause the human eardrum to vibrate and generate a continuous electrical signal. In audio-engineering terms, this electrical signal represents "analog audio", and our ear represents a (biological) "audio interface", i.e. a gateway for audio to enter or exit a processing system. The hearing sense is the result of our brain processing analog audio to create the perception of sound. In the past, physical systems for processing audio were all analog - but today, the availability of billions of transistors in general-purpose digital processing systems (laptops, mobiles, etc.) has made the so-called "digital audio representation" almost ubiquitous.
+
+While an analog signal is represented by its amplitude as continuous function in continuous time, the digital representation of that signal is a finite series of amplitude values generated by noting the value of the analog signal at fixed, discrete time intervals. The process of generating a digital audio representation from an analog signal is termed "sampling", and the length of the time interval is termed the "sample-period (its reciprocal - the "sampling rate" - is more commonly used while describing digital audio). In practice, analog audio from a source (traditionally a singer) enters a mic, is converted to a continuous electrical signal, and finally, a component called the "ADC - Analog to Digital Converter" samples the electrical signal at a fixed sampling rate (ex. 44.1KHz, 48Khz, 96Khz, etc.), generating a series of amplitude values, i.e. digital audio. To allow this audio to be widely distributed, the digital audio is encoded to a standard digital format (ex. MP3, WAV, AAC, etc.) using well-known algorithms. This allows sampled audio, i.e. music records to be (lossy or losslessly - depending on the encoding scheme) stored on digital hard-drives. The analog equivalent - vinyl records - skip the sampling stage entirely. Instead, a series of concentric grooves is etched into the disc, "encoding" the continuous change in the amplitude of the analog electrical signal. 
+
+For an audio record to be played back, there must exist a processing system that understands the original encoding scheme. For vinyl records, we have turntables connected to amplifiers. Moving the stylus along the vinyl grooves generates a continuous electrical signal, which is sent to a speaker. The speaker, being an analog device, responds to the continuous electrical signal by moving its membrane, creating air vibrations that we hear as sound. To playback digital records, however, we need the right software. The standardization of audio formats ensures that any piece of software that adheres to certain conventions can "decode" and play a digital record. This is one key principle behind the audio playback feature of production-grade software such as VLC, Windows Media Player, Apple Music, Spotify, and even Mixxx. A second requirement of this playback chain is the accurate conversion of digital audio to the original analog signal, to drive the speaker membrane. Audio playback software communicates with a digital audio interface that contains a "DAC - Digital to Analog Converter", which reconstructs the analog signal, and supplies it to the speaker. The audio playback software must ensure the DAC receives the samples it needs for reconstruction at the right time. 
+
+### The DAC and Sample-Rate Conversion
+The DAC is a device with a configurable clock, that requests processing software (in this case, Mixxx) for fixed-size sample buffers at regular intervals, i.e. at a certain sample rate. Depending on the software, each audio sample is represented by a fixed-precision floating point number (Mixxx uses 32-bit floats). The sample-rate of the input track gives the number of samples that represent a sample-period of track playback time.  
+
+
+
+
+
+#### Emulated Vinyl Scratching
+While Mixxx supports standard playback of digital records, its main purpose is to allow DJs to manipulate these loaded tracks in creative ways. 
+This could mean playing several tracks simultaneously, ramping track tempo, beatmatching etc. Traditionally, this is done by hand using multiple turntables and vinyl records. DJ Software such as Mixxx takes advantage of digital audio to offer far more flexiblity compared to vinyl DJing. For example, Mixxx allows a DJ to add digital effects - flanger, reverb, moog, etc. - using a technique known as digital filtering. Other aspects of DJing, such as vinyl scratching, are less compatible with digital audio.
+
+**Vinyl scratching causes a sudden acceleration or deceleration in playback speed of the loaded record(s)**, creating the audible scratching sound. On a turntable, scratching is performed by moving the stylus by hand - causing it to follow grooves in the vinyl that correspond to the analog audio waveform. In this case, no distortions are generated. 
+
+All DJ software, Mixxx included, must offer the ability to emulate vinyl scratching when records are stored digitally. Here, a MIDI controller's jog wheels can be spun to emulate the motion of the turntable stylus during scratching. However, since digitally stored audio is not a perfect representation of the analog waveform (i.e. it is a series of amplitude values sampled from the analog waveform at a fixed sample rate), *the software implementation of **playback speed change** is also imperfect*. More specifically, a resample of the input audio file is required to ensure the DAC receives samples in such a way that the reconstructed signal is in fact sped up/down. Sub-optimal resampling occasionally leads to unwanted, audible distortions during scratching.
+
+At present, the resample during scratching uses a fast, handcrafted linear interpolation algorithm. Mixxx uses the SoundTouch and RubberBand libraries to perform general audio time-stretching. These libraries implement additional interpolation algorithms, but as of June 2025, Mixxx has no support for using these while scratching. Therefore, the project goal is defined:
+- To determine whether alternative interpolation algorithms result in a *noticable reduction in scratching artifacts or a latency improvement over linear interpolation* and **providing quantitative supporting evidence** in the form of C++ unit tests under varying scratching scenarios.
+
+## Implementation
+I implemented resampling options to the Mixxx audio playback path through two branches:
+- A longstanding issue requesting custom resample rates during recording and broadcast (issue #)
+- Individual branches implementing pluggable C++17 resampler backends built on the libzita and libsamplerate libraries.
+
+
+#### The libsamplerate and libzita libraries
+
+
+
+
+
+# Footnotes.
+"and generate a continuous electrical signal": Not exactly. The cochlea has hair cells that are mapped to certain frequencies, and the actual signal is an impulse train. But for our model, it is sufficiently to view the cochlear signal as a continuous electrical signal.