transforms.MFCC: top_db clamp reduces over the time axis and cannot be disabled

[isaacsight]

Summary

amplitude_to_DB's top_db reduction includes the time axis, so a frame's MFCC coefficients depend on every other frame in the same tensor. transforms.MFCC hardcodes self.top_db = 80.0 and exposes no way to change or disable it, so this is not avoidable through that API.

This is a different axis from the one fixed in #1113. #994 and #3448 concern the batch/channel dimensions, and after #1113 each batch element does get its own maximum. The time axis was never the subject of those threads and is still inside the reduction:

x_db = torch.max(x_db, (x_db.amax(dim=(-3, -2, -1)) - top_db).view(-1, 1, 1, 1))

For a 2-D input [n_mels, time], packed_channels = 1 and the reshape gives [1, 1, n_mels, time], so amax(dim=(-3, -2, -1)) reduces over mel and time together: one scalar for the whole clip. Batched 4-D input is per-sample but still whole-clip in time.

Reading it as "the peak of this spectrogram" is entirely defensible, and I am not claiming the reduction is wrong in itself. The reportable part is that transforms.MFCC gives no way out of it, and the consequence is larger than I expected.

Reproduction

Self-contained, torchaudio 2.11.0:

import numpy as np, torch, torchaudio

SR, HOP, WIN, NFFT = 16000, 160, 400, 512
rng = np.random.default_rng(0)
quiet = torch.from_numpy(0.001 * rng.standard_normal(HOP * 200 + WIN)).float()

mfcc = torchaudio.transforms.MFCC(
    sample_rate=SR, n_mfcc=13,
    melkwargs=dict(n_fft=NFFT, win_length=WIN, hop_length=HOP,
                   center=False, n_mels=26))
alone = mfcc(quiet)

print(f"{'appended burst':>16}{'max|diff|':>12}{'c1..c12 energy':>18}{'distinct frames':>17}")
for amp in [0.0, 0.05, 0.9, 5.0, 50.0]:
    y = quiet if amp == 0 else torch.cat(
        [quiet, torch.from_numpy(amp * rng.standard_normal(HOP * 100)).float()])
    got = mfcc(y)[:, : alone.shape[1]]
    distinct = len({tuple(c.tolist()) for c in got.T})
    label = "none" if amp == 0 else f"+{20*np.log10(amp/0.001):.0f} dB"
    print(f"{label:>16}{(alone-got).abs().max():>12.4f}"
          f"{got[1:].abs().sum():>18.4g}{distinct:>13d}/{got.shape[1]}")

center=False with a length that is an exact multiple of hop_length means the 200 compared frames are byte-identical between runs. Only what follows them changes.

  appended burst   max|diff|    c1..c12 energy  distinct frames
            none      0.0000              8274          200/200
          +34 dB      0.0000              8274          200/200
          +59 dB      1.4035              8203          200/200
          +74 dB     23.3449              2544          200/200
          +94 dB    127.8718          0.003715            1/200

The last row is the part worth attention. Once the quiet frames fall more than top_db below the file peak, every mel bin in them clamps to the same floor, the frame becomes constant across mel, and the DCT of a constant is zero except at c0. All 200 frames collapse to one distinct vector. The remaining 0.003715 is float32 rounding, not signal: the largest surviving coefficient is 5.73e-06 against eps32 * |c0| = 4.39e-06, and c0 is identical to four decimal places across every frame.

A door slam or a cough in an otherwise quiet recording is roughly +94 dB above the noise floor.

Confirming the clamp is the sole cause, since log_mels=True takes the natural log of the mel energies and never calls AmplitudeToDB:

log_mels=True (AmplitudeToDB bypassed): max|diff| = 0.000000000

Same frames, same everything, clamp bypassed, bitwise identical.

Why this may be worth fixing rather than documenting

• Streaming and batch inference compute different features from identical audio by default, so a model trained on whole files and served on chunks sees a silent distribution shift.
• One loud transient degrades features everywhere else in the same file.
• log_mels=True avoids it but also changes the log base, so it is not a drop-in workaround for anyone matching an existing feature pipeline.

This is the same class as #4205 (Conformer output depending on padding), which is open and being treated as a real bug: an output that depends on context it should not see, failing silently rather than loudly.

Proposal

Accept top_db in transforms.MFCC.__init__ (and LFCC, which has the same hardcode), forwarding to AmplitudeToDB, with the current 80.0 kept as the default. That is backward compatible and makes the behaviour opt-out for anyone who needs frame-local features.

Optionally, allowing the reduction axes to be selected in amplitude_to_DB would let callers ask for a per-frame floor without giving up the clamp, but that is a larger change and I would not want to presume the design.

Happy to send a PR for the top_db passthrough if that would be useful.

For cross-reference, librosa has the same behaviour and the same lack of an override on its MFCC entry point; I have filed librosa/librosa#2094 there. Their maintainer proposed the equivalent fix in librosa/librosa#1734 back in 2023.

Versions: torchaudio 2.11.0, torch 2.13.0, numpy 2.5.2, Python 3.12, macOS.