Security: Heap buffer overflow in vendored stb_vorbis via setup_malloc integer truncation

[Tomer-PL]

Summary

A crafted 205-byte Ogg Vorbis file triggers a heap buffer overflow in the vendored stb_vorbis.c when processed by whisper-cli or the whisper server. The root cause is an implicit size_t-to-int truncation in setup_malloc.

Upstream report: nothings/stb#1947

Impact

• whisper-cli: Processing a malicious .ogg file via -f triggers the overflow
• whisper server: Uploading a crafted Ogg file to the /inference endpoint triggers the overflow (miniaudio decodes Ogg via stb_vorbis before whisper processing)

Root Cause

setup_malloc in examples/stb_vorbis.c takes int sz. Callers compute sizes with size_t arithmetic (sizeof(float) * entries * dimensions), but the result is truncated to int at the call boundary.

A codebook with entries=16,519,105 and dimensions=65 produces sizeof(float) * entries * dims = 4,294,967,300 — correct as size_t, but truncates to 4 as int32. After alignment, malloc(8) is called. The subsequent expansion loop writes 124 bytes into the 8-byte buffer.

Reproduction

# Generate the 205-byte exploit OGG (see PoC script in upstream issue)
python3 poc_generate.py

# Build with ASan
cmake -B build -DCMAKE_C_FLAGS="-fsanitize=address" -DCMAKE_CXX_FLAGS="-fsanitize=address"
cmake --build build

# Trigger
ASAN_OPTIONS=detect_leaks=0:halt_on_error=1 ./build/bin/whisper-cli -m models/ggml-tiny.bin -f exploit.ogg

ASan output:

ERROR: AddressSanitizer: heap-buffer-overflow on address 0x...
WRITE of size 4 at ... thread T0
    #0 ... in start_decoder stb_vorbis.c:3889
0x... is located 0 bytes to the right of 8-byte region
allocated by setup_malloc stb_vorbis.c:960

Suggested Fix

Update the vendored stb_vorbis.c to change setup_malloc (and setup_temp_malloc) to accept size_t instead of int. See the upstream issue for the full fix.

Related

CVE-2023-45661 through CVE-2023-45667 (same vulnerability family, different call sites).