Introduction complete with new sections. Need to elaborate on implementation details, add media.

Author: armaan

content/news/XXXX-XX-XX-gsoc-2025-report-armaan-chowfin.md

@@ -4,30 +4,38 @@ status: draft
 tags: gsoc, gsoc-2025
 comments: yes
 
-#### Introduction
-Compressions and rarefactions of the air surrounding us cause the human eardrum to vibrate and generate a continuous electrical signal. In audio-engineering terms, this electrical signal represents "analog audio", and our ear represents a (biological) "audio interface", i.e. a gateway for audio to enter or exit a processing system. The hearing sense is the result of our brain processing analog audio to create the perception of sound. In the past, physical systems for processing audio were all analog - but today, the availability of billions of transistors in general-purpose digital processing systems (laptops, mobiles, etc.) has made the so-called "digital audio representation" almost ubiquitous.
+### Introduction: Analog and Digital Audio
+Vibrations of the surrounding air in turn cause the human eardrum to vibrate, and generate a continuous electrical signal. In audio-engineering terms, this electrical signal represents "analog audio", and our ear represents a (biological) "audio interface", i.e. a gateway for audio to enter or exit a processing system. The hearing sense is the result of our brain processing analog audio to create the perception of sound.
 
-While an analog signal is represented by its amplitude as continuous function in continuous time, the digital representation of that signal is a finite series of amplitude values generated by noting the value of the analog signal at fixed, discrete time intervals. The process of generating a digital audio representation from an analog signal is termed "sampling", and the length of the time interval is termed the "sample-period (its reciprocal - the "sampling rate" - is more commonly used while describing digital audio). In practice, analog audio from a source (traditionally a singer) enters a mic, is converted to a continuous electrical signal, and finally, a component called the "ADC - Analog to Digital Converter" samples the electrical signal at a fixed sampling rate (ex. 44.1KHz, 48Khz, 96Khz, etc.), generating a series of amplitude values, i.e. digital audio. To allow this audio to be widely distributed, the digital audio is encoded to a standard digital format (ex. MP3, WAV, AAC, etc.) using well-known algorithms. This allows sampled audio, i.e. music records to be (lossy or losslessly - depending on the encoding scheme) stored on digital hard-drives. The analog equivalent - vinyl records - skip the sampling stage entirely. Instead, a series of concentric grooves is etched into the disc, "encoding" the continuous change in the amplitude of the analog electrical signal. 
+While an analog signal is represented by its amplitude as continuous function in continuous time, the digital representation of that signal is a finite series of amplitude values generated by noting the value of the analog signal at fixed, discrete time intervals. The process of generating a digital audio representation from an analog signal is termed "sampling", and the length of the time interval is termed the "sample-period (its reciprocal - the "sampling rate" - is more commonly used while describing digital audio). In practice, analog audio from a sound source enters a mic, is converted to a continuous electrical signal, and finally, a component called the "ADC - Analog to Digital Converter" samples the electrical signal at a fixed sampling rate (ex. 44.1KHz, 48Khz, 96Khz, etc.), generating a series of amplitude values, i.e. digital audio. To allow this audio to be widely distributed, the digital audio is encoded to a standard digital format (ex. MP3, WAV, AAC, etc.) using well-known algorithms. This allows sampled audio, i.e. music records to be stored on digital hard-drives. The analog equivalent - vinyl records - skip the sampling stage entirely. Instead, a series of concentric grooves is etched into the disc, "encoding" the continuous change in the amplitude of the analog electrical signal. 
 
-For an audio record to be played back, there must exist a processing system that understands the original encoding scheme. For vinyl records, we have turntables connected to amplifiers. Moving the stylus along the vinyl grooves generates a continuous electrical signal, which is sent to a speaker. The speaker, being an analog device, responds to the continuous electrical signal by moving its membrane, creating air vibrations that we hear as sound. To playback digital records, however, we need the right software. The standardization of audio formats ensures that any piece of software that adheres to certain conventions can "decode" and play a digital record. This is one key principle behind the audio playback feature of production-grade software such as VLC, Windows Media Player, Apple Music, Spotify, and even Mixxx. A second requirement of this playback chain is the accurate conversion of digital audio to the original analog signal, to drive the speaker membrane. Audio playback software communicates with a digital audio interface that contains a "DAC - Digital to Analog Converter", which reconstructs the analog signal, and supplies it to the speaker. The audio playback software must ensure the DAC receives the samples it needs for reconstruction at the right time. 
+For an audio record to be subsequently played back, there must exist a processing system that understands the original encoding scheme. For vinyl records, we have turntables connected to amplifiers. Moving the stylus along the vinyl grooves generates a continuous electrical signal, which is sent to a speaker. The speaker, being an analog device, responds to the continuous electrical signal by moving its membrane, creating air vibrations that we hear as sound. To playback digital records, however, we need the right software. The standardization of audio formats ensures that any piece of software that adheres to certain conventions can "decode" and play a digital record. This is one key principle behind the audio playback feature of production-grade software such as VLC, Windows Media Player, Apple Music, Spotify, and even Mixxx. A second requirement of this playback chain is the accurate conversion of digital audio to the original analog signal, to drive the speaker membrane. Audio playback software communicates with a digital audio interface that contains a "DAC - Digital to Analog Converter", which reconstructs the analog signal, and supplies it to the speaker.
 
-### The DAC and Sample-Rate Conversion
-The DAC is a device with a configurable clock, that requests processing software (in this case, Mixxx) for fixed-size sample buffers at regular intervals, i.e. at a certain sample rate. Depending on the software, each audio sample is represented by a fixed-precision floating point number (Mixxx uses 32-bit floats). The sample-rate of the input track gives the number of samples that represent a sample-period of track playback time.  
+The accurate reconstruction of an analog signal from a digital record is mathematically guaranteed under certain conditions. The Fourier theorem - a famous mathematical result - states that any analog signal can be represented by the sum of sinusoidal components of varying frequency and amplitude. The set of frequencies and their amplitudes gives the spectrum of the signal. In this model, the Shannon-Nyquist Sampling theorem states that the frequency at which analog audio is sampled (i.e. sampling rate) is twice its highest-resolvable frequency component -the Nyquist frequency. As a consequence, given that the human hearing range is roughly 20Hz-20kHz, the standard sampling rates 44.1Khz, 48Khz and 96Khz, with Nyquist frequencies 22Khz, 24Khz and 48Khz respectively, ensure that a digital record can represent every audible human frequency. The actual signal reconstruction is carried out in DAC hardware by circuits that implement so-called "digital reconstruction filters". These filters perform mathematical transformations to the discrete sample sequence to recreate the original analog signal.
 
+### Introduction: Sample-Buffers, DAC, ALSA
+Recall that digital audio is represented by a finite sequence of amplitude values, with each amplitude value stored as a fixed-precision floating point number (Mixxx uses 32-bit floats). These samples are more generally treated as logical "frames" for multichannel audio. An audio frame is an array containing #channels copies of the current sample value. There are three fundamental components in the audio playback chain: Mixxx (user-space audio processing software), ALSA (kernel subsystem with userspace API), and the DAC (device with transducer to convert digital to analog audio). The Mixxx preferences page [[insert mixxx sound preferences image]] allows the user to configure a `Sample Rate (Hz)`, and `Audio Buffer (ms)`. These parameters together influence the quality of output sound.
 
+The audio playback chain has three key buffers: A larger, userspace buffer containing outbound, perhaps processed frames of the digital record, an ALSA-managed, DMA-registered ring buffer of configurable size, and a hardware FIFO buffer in the DAC itself. The DAC has a configurable clock, whose clock rate (a.k.a. DAC sample-rate) specifies the fixed frequency at which frames in its hardware FIFO are consumed by the reconstruction filter. For example, a 44.1Khz DAC expects a frame to be available every 1/44.1K = 22.6us, for a total of 44.1K frames every second. Achieving this condition, however, is determined by upstream subsytems such as ALSA and parameters set by Mixxx. The `Sample Rate (Hz)` setting configures the DAC sample-rate, and the `Audio Buffer (ms)` sets the size in frames of the ALSA ring buffer for the selected `Sample Rate (Hz)`.
+ 
+The ALSA ring-buffer is of size `ring_buffer_size = Audio Buffer (ms)/1000 * Sample Rate (Hz)` frames, set via Mixxx preferences. Mixxx writes processed audio frames to the ring-buffer in units of atmost `ring_buffer_size` frames, and the ring-buffer is emptied in units of `period_size` frames (usually = `Audio Buffer/2`, see https://0pointer.de/blog/projects/all-about-periods.html), negotiated between Mixxx and the audio driver. During playback, a software interrupt is triggered each time `period_size` frames are consumed from the ring buffer and DMA-transferred to the DAC FIFO. The interrupt is handled by a high-priority callback thread in Mixxx, which refills the ring-buffer with more frames. While technically, only `period_size` frames are written to the DAC on every callback, Mixxx prepares `ring_buffer_size` frames in that duration. We can therefore simplify our model by noting that on average, `ring_buffer_size` frames are written to the DAC every callback.
 
+From the `period_size` and DAC sample rate, we can calculate `period_time = period_size/Sample Rate` as the hard real-time constraint on the user-space callback function. In other words, for the DAC hardware FIFO to never starve, the userspace callback thread must prepare atleast `period_size` worth of frames in atmost `period_time` time. Whether the constraint is met depends on various factors such as the amount of processing performed in the real-time thread, OS memory pressure, scheduling latency, etc. None of these kernel procedures have strict real-time guarantees, therefore on occasion, if `period_time` is too short, one hears pops during playback. This audio distortion is called a buffer underrun - highly undesireable in any live-DJ software. Since the main aim of Mixxx is to allow DJs to manipulate loaded tracks in creative ways -  via mixing, ramping track tempo, digital effects, scratching, etc., the realtime thread must perform far more processing than a standard audio player. This requires low-latency implementations of audio processing workflows without degrading audio quality.
 
+### Introduction: Sample Rate Conversion
+While buffer underruns are a result of DAC starvation agnostic of the buffer contents, another class of audio distortions is caused by the DAC not having the *right set* of frames in its FIFO. Resampling outbound audio is a procedure that ensures the DAC is receives the set of frames required to generate the analog playback characteristics we desire. The primary use-case is to resolve a sample-rate mismatch between DAC and the digital record.
 
-#### Emulated Vinyl Scratching
-While Mixxx supports standard playback of digital records, its main purpose is to allow DJs to manipulate these loaded tracks in creative ways. 
-This could mean playing several tracks simultaneously, ramping track tempo, beatmatching etc. Traditionally, this is done by hand using multiple turntables and vinyl records. DJ Software such as Mixxx takes advantage of digital audio to offer far more flexiblity compared to vinyl DJing. For example, Mixxx allows a DJ to add digital effects - flanger, reverb, moog, etc. - using a technique known as digital filtering. Other aspects of DJing, such as vinyl scratching, are less compatible with digital audio.
+The input sample rate defines how many frames of a digital record represent one second of analog audio. Meanwhile the DAC sample rate specifies how many outbound frames are consumed per second of real-world (wall-clock) time during playback. For example, a digital record sampled from analog at 48 kHz stores 48,000 frames for every second of analog sound. If this record is played back on a DAC operating at 44.1 kHz, assuming no underruns, only 44,100 frames are processed each second—meaning less than a full second of the outbound audio is played back per second. This results in an unintended slowdown. Conversely, if the DAC sample rate exceeds the input sample rate, more than one second of the original recording is heard every second, creating the perception of sped-up playback.
 
-**Vinyl scratching causes a sudden acceleration or deceleration in playback speed of the loaded record(s)**, creating the audible scratching sound. On a turntable, scratching is performed by moving the stylus by hand - causing it to follow grooves in the vinyl that correspond to the analog audio waveform. In this case, no distortions are generated. 
+Embed audio: [[playback kisses back wav@44.1Khz on 48K]], [[playback kisses back wav@48Khz on 44.1K]], [[playback kisses back wav@48Khz on 48K]]
 
-All DJ software, Mixxx included, must offer the ability to emulate vinyl scratching when records are stored digitally. Here, a MIDI controller's jog wheels can be spun to emulate the motion of the turntable stylus during scratching. However, since digitally stored audio is not a perfect representation of the analog waveform (i.e. it is a series of amplitude values sampled from the analog waveform at a fixed sample rate), *the software implementation of **playback speed change** is also imperfect*. More specifically, a resample of the input audio file is required to ensure the DAC receives samples in such a way that the reconstructed signal is in fact sped up/down. Sub-optimal resampling occasionally leads to unwanted, audible distortions during scratching.
+Mixxx offers features besides simple audio playback. Another use-case for resampling is during an intentional tempo ramp, when there is no sample-rate mismatch.  For a record sampled at 44.1kHz with a DAC also at 44.1kHz, scaling tempo by a factor of 3 means we want to pass 3x frames to the DAC on each callback than we would during standard playback. Without resampling, attempting to write 3x frames per callback would overfill the Mixxx-ALSA buffer. In the worst case, the excess frames would be dropped. Either way, the DAC would still consume only 44.1k frames per second—nullifying the intended tempo increase. It is clear that we need to create a situation where we can represent a longer amount of track duration using less frames, while ensuring that the resampled frames are still capable of being reconstructed. That is, every second, we need to represent 3 *44.1k frames using 44.1k frames only. This resample is achieved by a procedure called digital decimation, wherein frames are actually removed from a longer sequence before being written to the DAC. Conversely, digital interpolation is used when increasing track tempo, whereby new frames are generated between true samples using various algorithms. `SoundTouch`, `RubberBand`, `libzita` and `libsamplerate` are examples of open-source C++ libraries that implement standard interpolation algorithms to perform time-stretching on streaming data.
 
-At present, the resample during scratching uses a fast, handcrafted linear interpolation algorithm. Mixxx uses the SoundTouch and RubberBand libraries to perform general audio time-stretching. These libraries implement additional interpolation algorithms, but as of June 2025, Mixxx has no support for using these while scratching. Therefore, the project goal is defined:
-- To determine whether alternative interpolation algorithms result in a *noticable reduction in scratching artifacts or a latency improvement over linear interpolation* and **providing quantitative supporting evidence** in the form of C++ unit tests under varying scratching scenarios.
+### Emulated Vinyl Scratching
+Most commercial DJ software, Mixxx included, offers the ability to emulate vinyl scratching when records are stored digitally. Here, a MIDI controller's jog wheels can be spun to emulate the motion of the turntable stylus during scratching. Vinyl scratching causes a sudden acceleration or deceleration in tempo of the loaded record(s). On a turntable, scratching is performed by moving the stylus by hand - causing it to follow grooves in the vinyl that correspond to the analog audio waveform. However, while working with dital audio, faithful emulation of scratching requires a highly accurate and fast implementation of tempo ramping. In particular, Mixxx needs to implement fast and accurate resampling of input audio. Sub-optimal resampling often leads to audible distortions during scratching, particularly underflows. While there is technically also a chance for phase distortion due to incorrect interpolation algorithms, it is far less noticeable.
+
+At present, the resample during scratching uses a fast, handcrafted linear interpolation algorithm. Mixxx uses the `SoundTouch` and `RubberBand` libraries to perform general audio time-stretching. These libraries implement additional interpolation algorithms, but their performance is not optimal for fast changing speed and pitch which is the case when scratching. Therefore, my project goal is interpreted as follows:
+> To determine whether alternative interpolation algorithms result in a *noticable reduction in scratching artifacts or a latency improvement  over linear interpolation* and **providing quantitative supporting evidence** in the form of C++ unit tests under varying scratching scenarios
 
 ## Implementation
 I implemented resampling options to the Mixxx audio playback path through two branches:
@@ -37,9 +45,3 @@ I implemented resampling options to the Mixxx audio playback path through two br
 
 #### The libsamplerate and libzita libraries
 
-
-
-
-
-# Footnotes.
-"and generate a continuous electrical signal": Not exactly. The cochlea has hair cells that are mapped to certain frequencies, and the actual signal is an impulse train. But for our model, it is sufficiently to view the cochlear signal as a continuous electrical signal.