WIP: Final Report, GSOC 2025

[ar-in0]

This PR introduces a blog-post describing the implementation of additional resamplers for Mixxx - a deliverable for the GSOC 2025 final evaluation.

Preview

[daschuer]

nice start

[daschuer]

generate a continuous electrical signal.

This is not quite correct. Even our ear does a sampling of the signal by the small hears, in the cochlea.
https://en.wikipedia.org/wiki/Cochlea (whatch the video). There is also quantisation of in the frequency domain involved.
The value of the bins is than analog voltage, where it is digital in case it is a also quantized.

@danferns has posted here another nice video:
https://mixxx.zulipchat.com/#narrow/channel/109171-development/topic/Track.20Key.20.26.20BPM.20Color.20Coding

How a bout add a small picture of the inner ear here? Maybe you find a copy-left one on?

In the past, physical systems for processing audio were all analog - but today, the availability of billions of transistors in general-purpose digital processing systems (laptops, mobiles, etc.) has made the so-called "digital audio representation" almost ubiquitous.

I would remove that it is a bit It is a bit missleading.

I like the ear analogy to your resample very much, where the analog sound is recreated in liquid and than resampled by the hears.

Ear: Input signal -> drum -> bones -> recreate analog sound in liquid -> hears -> electrical signal.
resampler: Input signal -> recreation of the analog sound by sinc -> resampling -> numbers.

[ar-in0]

"and generate a continuous electrical signal": Not exactly. The cochlea has hair cells that are mapped to certain frequencies, and the actual signal is an impulse train. But for our model, it is sufficiently to view the cochlear signal as a continuous electrical signal.

• I had included this in the footnotes, maybe i can add it to the main passage.

[daschuer]

Maybe I would mention the Nyquist frequency.

[daschuer]

A mayor issue is also that 48 kHz and 44.1 kHz even exists. 44.1 for audio only 48 for video. Just discovered an interesting fact.
44.1 comes from video tapes used to record first digital audio ...

[daschuer]

Not sure if this sentences is a good fit in this paragraph.

[daschuer]

Suggested change

At present, the resample during scratching uses a fast, handcrafted linear interpolation algorithm. Mixxx uses the SoundTouch and RubberBand libraries to perform general audio time-stretching. These libraries implement additional interpolation algorithms, but as of June 2025, Mixxx has no support for using these while scratching. Therefore, the project goal is defined:

At present, the resample during scratching uses a fast, handcrafted linear interpolation algorithm. Mixxx uses the SoundTouch and RubberBand libraries to perform general audio time-stretching. These libraries implement additional interpolation algorithms, but their performance is not optimal for fast changing speed and pitch which is the case when scratching. Therefore, the project goal is defined:

[daschuer]

not sure if it is a smart move to city this literally. Your project did not solve all this. Instead we now have a recording re-sampler and a working scratching engine draft with libsamplerate without any benchmarks.

[daschuer]

Suggested change

	Mixxx offers the ability to emulate vinyl scratching when records are stored digitally. Here, a MIDI controller's jog wheels can be spun to emulate the motion of the turntable stylus during scratching. The action of scratching causes a sudden acceleration or deceleration in tempo of the loaded record(s).
	Mixxx offers the ability to emulate vinyl scratching when records are stored digitally. Here, a MIDI controller's jog wheels can be spun to emulate the motion of the viny record during scratching. The action of scratching causes a sudden acceleration or deceleration in tempo of the loaded record..

The turntable stylus only follows the carves in the vinyl, This is a minimal movement. Also turntable is moving at a constant speed eg. 33 rounds/min.

DJs have a so called slip pad between the vinyl and the turntable. This allow to stop or accelerate the vinyl on the turntable or play it backwards. The vinyl slips on the turntable.

[daschuer]

some more comments.

[daschuer]

Suggested change

On a turntable, scratching is performed by moving the stylus by hand - causing it to follow grooves in the vinyl that correspond to the analog audio waveform. However, while working with dital audio, faithful emulation of scratching requires a highly accurate and fast implementation of tempo ramping. In particular, Mixxx needs to implement fast and accurate resampling of input audio. Sub-optimal resampling often leads to audible distortions during scratching, particularly underflows. While there is technically also a chance for phase distortion due to incorrect interpolation algorithms, it is far less noticeable.

On a real turntable, scratching is performed by moving the vinyl record back and forth by hand - causing the stylus to follow grooves in the vinyl that correspond to the music's sound and tempo. However, while working with digital audio, realistic emulation of scratching requires a highly accurate and fast implementation of tempo ramping. In particular, Mixxx needs to implement fast and accurate resampling of input audio with gradually changing sample rates. Sub-optimal resampling often leads to audible distortions during scratching. It might introduce clicks or pops, especially if your computer can't keep up. While there is technically also a chance for phase distortion due to incorrect interpolation algorithms, it is far less noticeable.

[daschuer]

Suggested change

At present, the Mixxx resampler for scratching uses a fast, handcrafted linear interpolation algorithm. Mixxx also uses the `SoundTouch` and `RubberBand` libraries to perform general audio time-stretching. These libraries implement interpolation algorithms, but their performance is not optimal for fast changing speed and pitch which is the case when scratching. Therefore, my project goal is defined as follows:

At present, the Mixxx resampler for scratching uses a fast, handcrafted linear interpolation algorithm. Mixxx also uses the `SoundTouch` and `RubberBand` libraries to perform general audio time-stretching. These libraries implement high quality algorithms, but their performance is not optimal for fast changing speed and pitch which is the case when scratching. Therefore, my project goal is defined as follows:

[daschuer]

Suggested change

	Contributing to this project involves fundamental ideas in audio engineering, digital signal processing and realtime programming. To make this report accessible to those who have not worked with audio systems, I provide all necessary theoretical background, before describing my implementation.
	Contributing to this project required fundamental ideas in audio engineering, digital signal processing and realtime programming. To make this report accessible to those who have not worked with audio systems, I provide all necessary theoretical background, before describing my implementation in the following.

[daschuer]

Suggested change

Vibrations of the surrounding air in turn cause the human eardrum to vibrate, and generate a continuous electrical signal. In audio-engineering terms, this electrical signal represents "analog audio", and our ear represents a (biological) "audio interface", i.e. a gateway for audio to enter or exit a processing system. The hearing sense is the result of our brain processing analog audio to create the perception of sound.

Vibrations of the surrounding air, cause the human eardrum to vibrate. This movement is turned into electrical signals by tiny structures inside the ear. In audio-engineering terms, this electrical signal represents "analog audio", and our ear represents a (biological) "audio interface" to our brain. The hearing sense is the result of our brain processing analog audio to create the perception of sound.

[daschuer]

Suggested change

While an analog signal is represented by its amplitude as continuous function in continuous time, the digital representation of that signal is a finite series of amplitude values generated by noting the value of the analog signal at fixed, discrete time intervals. The process of generating a digital audio representation from an analog signal is termed "sampling", and the length of the time interval is termed the "sample-period (its reciprocal - the "sampling rate" - is more commonly used while describing digital audio). In practice, analog audio from a sound source enters a mic, is converted to a continuous electrical signal, and finally, a component called the "ADC - Analog to Digital Converter" samples the electrical signal at a fixed sampling rate (ex. 44.1KHz, 48Khz, 96Khz, etc.), generating a series of amplitude values, i.e. digital audio. To allow this audio to be widely distributed, the digital audio is encoded to a standard digital format (ex. MP3, WAV, AAC, etc.) using well-known algorithms. This allows sampled audio, i.e. music records to be stored on digital hard-drives. The analog equivalent - vinyl records - skip the sampling stage entirely. Instead, a series of concentric grooves is etched into the disc, "encoding" the continuous change in the amplitude of the analog electrical signal.

While an analog signal is represented by its amplitude as continuous function in continuous time, the digital representation of that signal is a finite series of amplitude values generated by noting the value of the analog signal at fixed, discrete time intervals. The process of generating a digital audio representation from an analog signal is termed "sampling", and the length of the time interval is termed the "sample-period (its reciprocal - the "sampling rate" - is more commonly used while describing digital audio). In practice, analog audio from a sound source enters a mic, is converted to a continuous electrical signal, and finally, a component called the "ADC - Analog to Digital Converter" samples the electrical signal at a fixed sampling rate (e.g. 44.1 kHz, 48 kHz, 96 kHz, etc.), generating a series of amplitude values, i.e. digital audio. To allow this audio to be widely distributed, the digital audio is encoded to a standard digital format (ex. MP3, WAV, AAC, etc.) using well-known algorithms. This allows sampled audio, i.e. music records to be stored on digital media. The analog equivalent - vinyl records - does not requires sampling. Instead, a series of concentric grooves is etched into the disc, "encoding" the continuous change in the amplitude of the analog electrical signal.

[daschuer]

Suggested change

For an audio record to be subsequently played back, there must exist a processing system that understands the original encoding scheme. For vinyl records, we have turntables connected to amplifiers. Moving the stylus along the vinyl grooves generates a continuous electrical signal, which is sent to a speaker. The speaker, being an analog device, responds to the continuous electrical signal by moving its membrane, creating air vibrations that we hear as sound. To playback digital records, however, we need the right software. The standardization of audio formats ensures that any piece of software that adheres to certain conventions can "decode" and play a digital record. This is one key principle behind the audio playback feature of production-grade software such as VLC, Windows Media Player, Apple Music, Spotify, and even Mixxx. A second requirement of this playback chain is the accurate conversion of digital audio to the original analog signal, to drive the speaker membrane. Audio playback software communicates with a digital audio interface that contains a **DAC - Digital to Analog Converter**, which reconstructs the analog signal, and supplies it to the speaker.

For an audio record to be subsequently played back, there must exist a processing system that understands the original encoding scheme. For vinyl records, we have turntables connected to amplifiers. Moving the stylus along the vinyl grooves generates a continuous electrical signal, which is sent to a speaker. The speaker, being an analog device, responds to the continuous electrical signal by moving its membrane, creating air vibrations that we hear as sound. To playback digital records, however, we need the right software. The standardization of audio formats ensures that any piece of software that adheres to certain conventions can "decode" and play a digital record. This is one key principle behind the audio playback feature of production-grade software such as VLC, Windows Media Player, Apple Music, Spotify, and even Mixxx. A second requirement of this playback chain is the accurate conversion of digital audio to the original analog signal, to drive the speaker membrane. Audio playback software communicates with a digital audio interface that contains a **DAC - Digital to Analog Converter**, which reconstructs the analog signal, and supplies it to the speaker.

[daschuer]

Suggested change

The audio playback chain has three key buffers: A larger, userspace buffer containing outbound, perhaps processed frames of the digital record, an ALSA-managed, DMA-registered ring buffer of configurable size, and a hardware FIFO buffer in the DAC itself. The DAC has a configurable clock, whose clock rate (a.k.a. DAC sample-rate) specifies the fixed frequency at which frames in its hardware FIFO are consumed by the reconstruction filter. For example, a 44.1Khz DAC expects a frame to be available every 1/44.1K = 22.6us, for a total of 44.1K frames every second. Achieving this condition, however, is determined by upstream subsytems such as ALSA and parameters set by Mixxx. The `Sample Rate (Hz)` setting configures the DAC sample-rate, and the `Audio Buffer (ms)` sets the size in frames of the ALSA ring buffer for the selected `Sample Rate (Hz)`.

The audio playback chain has three buffers: A larger, read ahead buffer containing the decoded frames from of the digital record in source sample rate. A small engine buffer, containing processed outbound frames in hardware sample rate of configurable size, and a hardware FIFO buffer in the DAC itself. The DAC has a configurable clock, whose clock rate (a.k.a. DAC samplerate) specifies the fixed frequency at which frames in its hardware FIFO are consumed by the reconstruction filter. For example, a 44.1 kHz DAC expects a frame to be available every 1/44.1 kHz = 22.6 µs, for a total of 44.1 k frames every second. Achieving this condition, however, is determined by upstream subsytems such as ALSA on Linux and parameters set by Mixxx. The `Sample Rate (Hz)` setting configures the DAC sample-rate, and the `Audio Buffer (ms)` sets the duration of the engine buffer defining the time Mixxx has to produce the number of samples to fill this buffer at the selected samplerate.

[daschuer]

Suggested change

The accurate reconstruction of an analog signal from a digital record is mathematically guaranteed under certain conditions. The Fourier theorem - a famous mathematical result - states that any analog signal can be represented by the sum of sinusoidal components of varying frequency and amplitude. The set of frequencies and their amplitudes gives the spectrum of the signal. In this model, the Shannon-Nyquist Sampling theorem states that the frequency at which analog audio is sampled (i.e. sampling rate) is twice its highest-resolvable frequency component -the Nyquist frequency. As a consequence, given that the human hearing range is roughly 20Hz-20kHz, the standard sampling rates 44.1Khz, 48Khz and 96Khz, with Nyquist frequencies 22Khz, 24Khz and 48Khz respectively, ensure that a digital record can represent every audible human frequency. The actual signal reconstruction is carried out in DAC hardware by circuits that implement so-called "digital reconstruction filters". These filters perform mathematical transformations to the discrete sample sequence to recreate the original analog signal.

The accurate reconstruction of an analog signal from a digital record is mathematically guaranteed under certain conditions. The Fourier theorem - a famous mathematical result - states that any analog signal can be represented by the sum of sinusoidal components of varying frequency and amplitude. The set of frequencies and their amplitudes gives the spectrum of the signal. In this model, the Shannon-Nyquist Sampling theorem states that the frequency at which analog audio is sampled (i.e. sampling rate) is twice its highest-resolvable frequency component - the Nyquist frequency. As a consequence, given that the human hearing range is roughly 20 Hz - 20 kHz, the standard sampling rates 44.1 kHz, 48 kHz and 96 kHz, with Nyquist frequencies 22 kHz, 24 kHz and 48 kHz respectively, ensure that a digital record can represent every audible human frequency. The actual signal reconstruction is carried out in DAC hardware by circuits that implement so-called "digital reconstruction filters". These filters perform mathematical transformations to the discrete sample sequence to recreate the original analog signal.

[daschuer]

How?
Samplerate: Avoid resampling ...
Audio Buffer: CPU load vs responsivity

[daschuer]

This is how other audio applications may work. Mixxx is optimized for low latency and runs directly on a swapped buffer. While one chung kis passed to the DAC, the other is prepared by Mixxx. After a period both buffers are swapped.

Suggested change

The ALSA ring-buffer is of size `ring_buffer_size = Audio Buffer (ms)/1000 * Sample Rate (Hz)` frames, set via Mixxx preferences. Mixxx writes processed audio frames to the ring-buffer in units of atmost `ring_buffer_size` frames, and the ring-buffer is emptied in units of `period_size` frames (usually = `Audio Buffer/2`)[[1]](#1), negotiated between Mixxx and the audio driver. During playback, a software interrupt is triggered each time `period_size` frames are consumed from the ring buffer and DMA-transferred to the DAC FIFO. The interrupt is handled by a high-priority callback thread in Mixxx, which refills the ring-buffer with more frames. While technically, only `period_size` frames are written to the DAC on every callback, Mixxx prepares `ring_buffer_size` frames in that duration. We can therefore simplify our model by noting that on average, `ring_buffer_size` frames are written to the DAC every callback.

During playback, a software interrupt is triggered each time the buffered frames are consumed from the engine buffer and DMA-transferred to the DAC FIFO. The interrupt is handled by a high-priority callback thread in Mixxx, which refills the ring-buffer. In case Mixxx fails to fill it before the samples are needed to old buffer is played again, this causes an audible pop.```

[daschuer]

This paragraph is long without many useful information, I would remove it. Maybe rework the paragraph above a bit with these infos.

[Eve00000]

You write:
The set of frequencies and their amplitudes gives the spectrum of the signal. In this model, the Shannon-Nyquist Sampling theorem states that the frequency at which analog audio is sampled (i.e. sampling rate) is twice its highest-resolvable frequency component -the Nyquist frequency.

That isn't correct:
... that the frequency at which analog audio is sampled (i.e. sampling rate) is twice its highest-resolvable ..
must be
... that the sample rate must be at least twice the bandwidth of the signal to avoid aliasing ...

Add: if you mention theories please supply a link to the source.

[daschuer]

Thank you for the Blog post. There are still some misleading information in this blog, but I don't want to risk a formal deadline miss of @6charm GSoC project on certain details.
I will merge it now but we need urgently a fixed version to not confuse other contributors. I will hold @6charm responsible ;-)

[ar-in0]

Thank you for the Blog post. There are still some misleading information in this blog, but I don't want to risk a formal deadline miss of @6charm GSoC project on certain details. I will merge it now but we need urgently a fixed version to not confuse other contributors. I will hold @6charm responsible ;-)

Thank you for your consideration. Will continue the conversation in a new PR.