Skip to content

Commit 7a0075a

Browse files
committed
Audio fingerprinting and recognition in Go
Signed-off-by: Quang Nguyen <quang.nguyen@hey.com> Signed-off-by: Quang Nguyen <nguyenquang@microsoft.com>
0 parents  commit 7a0075a

48 files changed

Lines changed: 6342 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.dockerignore

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
.git
2+
.gitignore
3+
.github
4+
.claude
5+
*.prfp
6+
*.wav
7+
audio/
8+
testSamples/
9+
deploy/
10+
README.md
11+
CLAUDE.md
12+
gopher.svg
13+
Dockerfile
14+
.dockerignore

.gitignore

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Binaries for programs and plugins
2+
*.exe
3+
*.exe~
4+
*.dll
5+
*.so
6+
*.dylib
7+
8+
# Test binary, built with `go test -c`
9+
*.test
10+
11+
# Output of the go coverage tool, specifically when used with LiteIDE
12+
*.out
13+
14+
# Dependency directories (remove the comment below to include it)
15+
# vendor/
16+
17+
/presto
18+
/audio
19+
/testSamples

.markdownlint.json

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"MD013": false,
3+
"MD033": false,
4+
"MD041": false
5+
}

CLAUDE.md

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Project Overview
6+
7+
Presto is an audio fingerprinting and recognition tool written in Go. It identifies song tracks from audio snippets using two selectable algorithms — constellation peak-pair hashing (Wang 2003, default) and Haitsma-Kalker sub-band mel-band energy comparison — with a persistent file-backed library and vote-based lookup that scales to thousands of songs. See [docs/algorithm.md](docs/algorithm.md) for the full algorithm walkthrough.
8+
9+
## Build, Run, and Test
10+
11+
```bash
12+
# Build
13+
go build ./cmd/presto
14+
15+
# Index an audio directory into a library file
16+
./presto index <audio_dir> <store_file> <winSize> <hopSize> [windowFunc]
17+
18+
# Index with sub-band algorithm instead of constellation
19+
./presto index <audio_dir> <store_file> <winSize> <hopSize> [windowFunc] --algo subband
20+
21+
# Match a sample against an indexed library (one-shot CLI)
22+
./presto match <store_file> <sample_file>
23+
24+
# Run the HTTP server (configured via PRESTO_* env vars)
25+
./presto serve
26+
27+
# Example
28+
./presto index ./songs/ library.prfp 1024 512 hann
29+
./presto match library.prfp clip.wav
30+
PRESTO_STORE_PATH=./library.prfp ./presto serve
31+
32+
# Window function options: hann, hamming, bartlett (omit for none)
33+
34+
# Container build
35+
docker build -t presto:latest .
36+
37+
# Run all tests
38+
go test ./...
39+
40+
# Run a single test
41+
go test ./internal/fingerprint/ -run TestExactClipMatch -v
42+
43+
# Run benchmarks
44+
go test ./internal/fingerprint/ -bench=. -benchmem
45+
go test ./internal/store/ -bench=. -benchmem
46+
```
47+
48+
Input files must be WAV format (PCM, 8/16/24/32-bit).
49+
50+
## Architecture
51+
52+
```text
53+
internal/
54+
audio/ WAV file I/O
55+
wav.go Signal type, ReadWAV, DecodeWAV, EncodeWAV, WriteWAV
56+
57+
dsp/ Signal processing primitives
58+
fft.go Cooley-Tukey radix-2 FFT, per-call scratch buffer
59+
window.go Window functions (hann, hamming, bartlett)
60+
spectrogram.go STFT spectrogram, signal normalization, noise injection
61+
peaks.go 2D time-frequency local maximum detection
62+
mel.go Mel-frequency banding, sub-band energy comparison
63+
64+
fingerprint/ FP type, Strategy interface, registry
65+
fingerprint.go FP tagged union, Strategy interface, Get/Register, convenience funcs
66+
constellation/ Constellation strategy (peak-pair hashing, init-registers)
67+
subband/ Sub-band strategy (mel-band energy bits, init-registers)
68+
69+
store/ Persistent library with vote-based lookup
70+
store.go Store type, Load (mmap), Save, Match API, StoreStrategy dispatch
71+
strategy.go StoreStrategy interface, algorithm byte mapping
72+
format.go Binary file format (PRFP v2 with algorithm byte)
73+
constellation_index.go Constellation hash inverted index + store strategy
74+
subband_index.go LSH inverted index + sub-band store strategy
75+
76+
metrics/ Stdlib-only Prometheus metrics (Counter/Gauge/Histogram/Registry)
77+
metrics.go Metric types, registry, runtime collector
78+
79+
cmd/presto/
80+
main.go Subcommand dispatch: index / match / serve
81+
server.go HTTP server (POST /v1/match, /v1/stats, /healthz, /readyz, /metrics)
82+
server_test.go httptest-based handler tests
83+
84+
deploy/k8s/ Plain YAML manifests for Kubernetes deployment
85+
Dockerfile Multi-stage build (golang:1.26-alpine -> distroless)
86+
docs/algorithm.md Algorithm walkthrough covering both strategies
87+
```
88+
89+
**Dependency graph**: `cmd/presto` -> `store`, `metrics` -> `fingerprint` -> `audio`, `dsp`
90+
91+
**Strategy pattern**: Both algorithms register via `init()` + blank import (like `database/sql` drivers). The `Strategy` interface covers fingerprinting and similarity; the `StoreStrategy` interface covers indexing, matching, and serialization. The store header's algorithm byte auto-selects at load time.
92+
93+
**Fingerprinting pipeline** (shared + per-algorithm):
94+
95+
1. Read/decode WAV (`audio.ReadWAV` or `audio.DecodeWAV`) -> float64 samples
96+
2. Normalize + window + FFT (`dsp.Spectrogram`) -> magnitude spectrogram
97+
3a. *Constellation*: `dsp.FindPeaks` -> `constellation.GenerateHashes` -> `FP{Hashes, NumFrames}`
98+
3b. *Sub-band*: `dsp.MelBandsInto` -> `dsp.SubBandFPInto` -> `FP{Data, Stride, Frames}`
99+
100+
## HTTP server (`presto serve`)
101+
102+
Loads one `.prfp` library at startup (async — readyz reflects state). Auto-selects algorithm from header. Concurrent reads are safe (store is read-only after Load).
103+
104+
**Endpoints**: `POST /v1/match` (JSON), `GET /v1/stats`, `GET /healthz`, `GET /readyz`, `GET /metrics`
105+
106+
**Configuration** via environment variables:
107+
108+
- `PRESTO_STORE_PATH` (default `/var/lib/presto/library.prfp`)
109+
- `PRESTO_LISTEN_ADDR` (default `:8080`)
110+
- `PRESTO_MAX_UPLOAD_BYTES` (default 10 MiB)
111+
112+
**Graceful shutdown**: `SIGINT` / `SIGTERM` -> drain with 10 s timeout. Panic recovery middleware on all routes.
113+
114+
## Store
115+
116+
**File format** (single binary file, little-endian, version 2):
117+
118+
```text
119+
Header (32 bytes):
120+
magic "PRFP", version (2), songCount,
121+
winSize, hopSize, windowFunc, algorithm (0=constellation, 1=subband), reserved
122+
123+
Per song:
124+
nameLen uint16, name, numFrames uint32, dataLen uint32, fpData
125+
```
126+
127+
`Load` memory-maps the file and zero-copy slices per-song data via `unsafe.Slice` (with runtime alignment guard). Index is rebuilt on load (not serialized).
128+
129+
**Matching**: direct hash lookup (constellation) or LSH candidate filter + sliding-window Compare (sub-band), both using `(songID, delta)` vote accumulation.
130+
131+
**Size limits**: 65,535 songs (uint16 songID); song length limited only by uint32 frame offset.

Dockerfile

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# syntax=docker/dockerfile:1.7
2+
3+
# ---- Build stage ----
4+
FROM golang:1.26-alpine AS build
5+
WORKDIR /src
6+
7+
# Prime the module cache first so repeated builds with no dep changes are fast.
8+
# go.sum is copied alongside go.mod so module downloads are verified against
9+
# checksums instead of silently trusting whatever the proxy returns.
10+
COPY go.mod go.sum* ./
11+
RUN go mod download
12+
13+
# Copy sources and build a static, stripped binary.
14+
COPY . .
15+
RUN --mount=type=cache,target=/root/.cache/go-build \
16+
CGO_ENABLED=0 GOOS=linux go build \
17+
-trimpath \
18+
-ldflags="-s -w" \
19+
-o /out/presto \
20+
./cmd/presto
21+
22+
# ---- Runtime stage ----
23+
FROM gcr.io/distroless/static-debian12:nonroot
24+
25+
COPY --from=build /out/presto /usr/local/bin/presto
26+
27+
EXPOSE 8080
28+
29+
# Default configuration — override via env vars in the deployment.
30+
ENV PRESTO_LISTEN_ADDR=:8080 \
31+
PRESTO_STORE_PATH=/var/lib/presto/library.prfp \
32+
PRESTO_MAX_UPLOAD_BYTES=10485760
33+
34+
ENTRYPOINT ["/usr/local/bin/presto"]
35+
CMD ["serve"]

LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2026 Quang Nguyen
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

0 commit comments

Comments
 (0)