I've been porting parakeet-tdt to another runtime and comparing against parakeet-cli token by token, which turned up an odd corner in the decoder.
parakeet_decode picks the TDT duration by argmax over pstate.logits[n_vocab_logits + i], but pstate.logits holds log-probs — the joint graph ends in log(softmax(...)) over the whole 1030-wide row. The duration slots carry a vanishing share of that softmax (their log-probs sit around -100 and below on real speech), so exp(logit - max) underflows fp32 to exactly 0 and the log gives -inf. When all five duration slots are -inf, the argmax loop (best_duration_logit = -1e10f with a strict >) never updates and silently returns duration index 0 — a value that came from the sentinel, not the model.
On samples/gb1.wav this happens at 16 of 786 emitted tokens. Computing the same argmax analytically (logit − logsumexp, which can't underflow) gives a different winner at those steps with gaps of 6–20 nats, so it's not a near-tie. Token ids happen to come out the same on that clip, but the affected timestamps shift, and in principle the decode path can diverge since t advances differently.
Two easy fixes: argmax the durations over the raw logits tensor (it's already ggml_set_output in the joint graph), or compute the log-softmax as logit − logsumexp instead of log(softmax).
I've been porting parakeet-tdt to another runtime and comparing against parakeet-cli token by token, which turned up an odd corner in the decoder.
parakeet_decodepicks the TDT duration by argmax overpstate.logits[n_vocab_logits + i], butpstate.logitsholds log-probs — the joint graph ends inlog(softmax(...))over the whole 1030-wide row. The duration slots carry a vanishing share of that softmax (their log-probs sit around -100 and below on real speech), soexp(logit - max)underflows fp32 to exactly 0 and the log gives -inf. When all five duration slots are -inf, the argmax loop (best_duration_logit = -1e10fwith a strict>) never updates and silently returns duration index 0 — a value that came from the sentinel, not the model.On samples/gb1.wav this happens at 16 of 786 emitted tokens. Computing the same argmax analytically (logit − logsumexp, which can't underflow) gives a different winner at those steps with gaps of 6–20 nats, so it's not a near-tie. Token ids happen to come out the same on that clip, but the affected timestamps shift, and in principle the decode path can diverge since t advances differently.
Two easy fixes: argmax the durations over the raw
logitstensor (it's alreadyggml_set_outputin the joint graph), or compute the log-softmax as logit − logsumexp instead of log(softmax).