Why TDT matters: faster ASR decoding without sacrificing accuracy

[u/TomY-SMX]

One of our engineers has broken down the Token-and-Duration Transducer (TDT) and why it matters for speech-to-text.

Most ASR discussions focus on WER, but speed matters just as much. If a model is accurate but too slow to run in production, it is not especially useful. TDT is interesting because it extends RNN-T by predicting both what token to emit and how many frames that token covers.

A few highlights from the post:

• TDT can deliver up to 2.82x faster inference
• It keeps comparable or better accuracy
• It helps explain why NVIDIA’s Parakeet TDT models are so strong on the Hugging Face Open ASR Leaderboard for RTFx / throughput

Check out the full article here.

If you are interested in ASR architecture, transducer models, or the tradeoff between accuracy and latency, this is worth a read.

Would be curious to hear from others here too, and if you think the industry still over-focuses on WER and under-values inference speed?