A good voice isn't
always a good agent.
A voice can sound convincing while calling the wrong tool. Full-duplex benchmarks need to measure both conversation and action.
Say "book a table for six—actually, make that seven." A useful agent needs to hear the correction, update the party size, call the right function, and tell you what happened. Recognizing each word is only part of the task.
Two scores, two questions
Full-Duplex-Bench v3 tests tool use against real spoken requests containing disfluencies. Its strict Pass@1 metric asks whether the system gets the required tools and arguments right on its first attempt. A score of 52% represents success on roughly half the benchmark scenarios. It is not a 52% probability that any individual word or tool name is correct.
Reply quality asks a different question: does the spoken response fulfill the request? The benchmark uses an LLM judge on the transcript. That score is about the content of the reply, not the sound of the voice. A system can say something useful while missing an exact tool argument, or execute a tool correctly and explain its result poorly.
How Duplex Compute compares
On FDB-v3, Duplex Compute reaches 57% Pass@1 and 69% reply quality. That puts it slightly ahead of GPT-Live-1 (55% / 71.7%) on task success while costing 80% less per hour of connected time.
The uncertainty matters. A 54-of-100 result has a 95% Wilson interval of roughly 44–64%. Its overlap with the other results does not establish equality or a statistically significant lead. What it does show is that open architectures can match commercial systems on interaction quality at a fraction of the cost.
A system benchmark, not a model-size ranking
The NVIDIA 30B and 235B entries in our chart use front-end/back-end architectures with external speech recognition and Qwen3 backends. Cascaded GPT-4o combines Whisper, GPT-4o, and speech synthesis. These are different systems with different pipelines, not a clean comparison of larger and smaller versions of one voice model.
Duplex Compute is a complete voice system: streaming recognition, turn-taking, dialogue, and synthesis. The benchmark shows it can match commercial APIs on task success while running at 1/5 the cost.
What we want to measure next
A useful production comparison should report complete task success alongside interruption handling, response latency, and cost at realistic concurrency. It should include long sessions, corrected arguments, cancelled actions, and tool failures. A fast answer that creates the wrong booking is not a successful interaction.
Our benchmark methodology follows FDB-v3 specifications. The chart on this site shows Duplex Compute results alongside published data from other providers where hourly pricing is available.
Building an application with these requirements? Tell us about it →