That is why the Speech Accessibility Project is so important. Led by the University of Illinois Urbana-Champaign, with participation from organizations across academia and industry, the project was created to address a persistent gap in speech technology. Its dataset includes about 1,000 hours of contributed speech from people with diverse and non-standard speech patterns.
The project asks a better and more demanding question than many traditional benchmarks:
Who does speech AI still fail to understand?
That shift matters. It moves the focus from average performance to individual experience—from what works under common conditions to what works for people who have historically been underrepresented in training data and evaluation.
Internally, we evaluated the production Zoom AI Services Scribe API on this benchmark exactly as it is available today. There was no benchmark-specific tuning, and no separate model created for the evaluation.
The result was a word error rate of 5.72%.
Scribe's word error rate was approximately 52% lower than that of the next-closest system:
| Model |
Word Error Rate ↓ |
| Zoom Scribe API |
|
| Microsoft/Azure-LLM-speech |
|
| Elevenlabs/Scribe_v2 |
|
| Microsoft/Azure-fast-transcription |
|
| Assemblyai/Universal-3.5-pro |
|
| Deepgram/Nova-3 |
|
| Cartesia/Ink2 |
|
| Openai/gpt-realtime-whisper |
|
Lower WER = better accuracy. Benchmark: Speech Accessibility Project dataset.
On a benchmark this demanding, reducing recognition errors by half is more than an incremental improvement. It suggests that the frontier is beginning to move on a problem our field has wrestled with for decades.
We are proud of the result. But I have also learned to approach benchmarks with humility. No single dataset captures the full complexity of human communication. No benchmark represents every speaker, language, environment, or use case.
What matters is whether a result changes our understanding of what can be built—and whether that progress reaches the people who need it.