Hume AI

Today Hume is introducing Real World VoiceEQ: a benchmark for measuring the human quality of voice AI across 40+ proprietary and open-source models, 15+ dimensions, and 60+ metrics—developed from more than 1 million individual human ratings. The biggest takeaway: there is no single “best” voice model. The field is becoming specialized. Some models are strong at expressive speech. Others are better at precision. Some understand emotion but struggle to respond naturally. Voice models are getting better at speaking than truly listening. Many systems still reason from transcripts instead of vocal cues like tone, pacing, hesitation, and emphasis. That is a major gap for real-world voice agents. A confident “yeah” and a hesitant “…yeah” can mean very different things responding to a fraud alert or weighing health advice. This is why voice AI needs evaluation that looks beyond words. Traditional benchmarks are starting to overestimate real-world performance. Models can perform well in clean test settings, then struggle with accented speech, noisy environments, overlapping speakers, emotional expression, and long conversations. Speech-to-speech remains one of the least mature frontiers in voice AI. Some systems recognize emotion well but do not respond naturally. Others sound fluid but miss the emotional context. Understanding and responding are still separate capabilities. Human evaluation remains essential for voice AI. Automated judges can help, but voice quality, emotional fit, believability, and conversational appropriateness are ultimately experienced by people. The next phase of voice AI will need a measurement layer: public benchmarks, private evaluations, and production monitoring that show how models perform in the real world, not just under ideal test conditions.