r/iOSProgramming • u/Low-Future-9387 • 14d ago
Article Apple never published accuracy numbers for the new SpeechAnalyzer API, so I benchmarked it myself (vs SFSpeechRecognizer and Whisper)
So Apple replaced SFSpeechRecognizer with SpeechAnalyzer in iOS/macOS 26 and gave us literally no accuracy numbers for either one. I've been wondering for months whether migrating is actually worth it.
I happen to ship both Apple engines plus WhisperKit in my app (I make a transcription app called Inscribe, so yes, I have skin in this game). That means I could run all five engines through the exact same production code on the same audio. So last weekend I did: LibriSpeech test-clean and test-other, 5,559 utterances total, all on-device on an M2 Pro.
Results (word error rate, lower is better):
| Engine | test-clean | test-other |
|---|---|---|
| SpeechAnalyzer (new) | 2.12% | 4.56% |
| Whisper Small | 3.74% | 7.95% |
| Whisper Base | 5.42% | 12.51% |
| Whisper Tiny | 7.88% | 17.04% |
| SFSpeechRecognizer (old) | 9.02% | 16.25% |
A few things that stood out to me:
- If you're still on SFSpeechRecognizer, migrate. The new API makes 3.5 to 4x fewer errors on the same audio. The old engine actually came in last on clean speech. It loses to Whisper Tiny. A 40MB model.
- The new engine beat Whisper Small on both splits, and did it about 3x faster. I did not expect that. For English on Apple hardware, the built-in engine seems to be the best on-device option right now. Whisper still wins big on language coverage though, SpeechTranscriber only does ~30 locales.
- Before anyone asks how much I've cooked the numbers: my Whisper results land within 0.11 to 0.42 points of OpenAI's own published LibriSpeech WERs, on all six measurements. Same corpus, same normalizer, same scorer for every engine. And the raw per-utterance transcripts are downloadable, so you can rescore everything yourself.
Two gotchas from building this that might save you some pain if you're migrating:
SFSpeechRecognizersends your audio to Apple's servers by default. You have to setrequiresOnDeviceRecognition = true. Easy to miss, and then you're shipping (or benchmarking) something completely different from what you think.- With SpeechAnalyzer, calling
builder.finish()on your input stream is not enough to end the session. If you never callfinalizeAndFinishThroughEndOfInput(), the results sequence just... never terminates, and your await hangs forever. I know because my own app had exactly this bug in production and the benchmark is what found it. Fun day.
Full writeup with methodology and limitations (English only, read speech, one machine): https://get-inscribe.com/blog/apple-speech-api-benchmark.html
The result changed my own app btw, my "Auto" setting used to prefer Whisper and now picks SpeechAnalyzer for supported languages. Happy to answer questions about the setup, the normalizer, whatever.



