Listens while it speaks
Every second, Duopra decides whether to listen or to speak. It takes the turn when you finish, holds back in a mid-sentence pause and gives way when you cut in.
It listens and speaks at the same time, and decides on its own when to act.
It's actually 150 euros, not 100.Duopra hears the correction, decides on its own to call the tool, and sends the call while it is still speaking.
Full-duplex models already know when to listen and when to speak. Duopra also knows when to act — it calls tools itself, mid-sentence, through actions that are never voiced.
Every second, Duopra decides whether to listen or to speak. It takes the turn when you finish, holds back in a mid-sentence pause and gives way when you cut in.
Tool calls go out on a private channel while Duopra is still talking. The call itself is never spoken — only its result is.
Change a value halfway through a request and only the final value reaches the tool. The abandoned 100 never does.
Two changes asked for in one hesitant breath become two calls, and each is reported only after its result comes back.
Connect a backend model and it becomes one more tool, query(task). Duopra hands work over the way it calls any tool: mid-conversation, with no pause and no hand-off you can hear.
A tool is a JSON schema: a name, a description and its parameters, supplied at run time. Write your own — your calendar, your database, your home — and Duopra can call it.
Two sessions from the evaluation, replayed in recorded time.
Every second, Duopra reads the new audio and any tool results, then emits three things at once.
One causal policy, conditioned on the audio so far, the observations injected so far and its own past outputs. Conversation and tool use are decided together, not handed between components.
A background agent is exposed as query(task), whose argument is a self-contained description of the user's task. Calling a tool directly or delegating is the same kind of decision, made by the same policy — there is no external dispatcher.
A runtime executes each call asynchronously and injects the result at the first one-second boundary after it returns. The model keeps listening and speaking while it waits, and reports a value only once it has arrived.
query(task) included — asks for both values and calls flight.status itself, then again with the new date.One stage for each problem: the form of an action, the decision to act, and a voice that drifts when the thinker changes.
Teaches the form of an action.
Supervised fine-tuning of the thinker on synthetic sessions that demonstrate the protocol: call tools by schema, wait for the result before reporting it, ask only for missing values.
Teaches the decision to act.
Dr. GRPO on a thinker LoRA, eight rollouts per task, in a sandbox that plays the same scripted user to every rollout. Time-anchored deadlines and outcome tiers reward calls that are timely, complete and grounded.
Keeps the voice intelligible.
Post-training shifts the thinker's hidden states, which condition the talker. Only the projector between them is retrained, on cached features; the text, control tokens and calls are untouched.
The highest pass rate among open-source systems on Full-Duplex-Bench v3.
A session passes only when every executed call matches the gold call: right tool, every argument, no extra call.
Full-Duplex-Bench v3: 100 real, disfluent English recordings and 12 simulated APIs. Duopra runs in our harness — three seeds, 300 sessions per configuration, judged by Qwen3.8-27B; ± is the standard deviation across seeds. Hatched bars are reported numbers, measured with each source's own harness and judge (mostly GPT-4o), so they compare only approximately. * Commercial system.
The outcome of each of 300 sessions, Duopra models calling tools themselves.
Multi-call requests become solvable: Duopra passes 28 of 102 sessions that need two or three calls, against 2 after SFT. The remaining headroom is in arguments — wrong arguments (19.3% of sessions) are now as common as sessions without a call (18.7%).
| Run (300 sessions) | Pass | Wrong arg. | Missing | Extra | No call |
|---|---|---|---|---|---|
| Duopra-SFT | 78 | 17 | 38 | 1 | 166 |
| Duopra-RL1 | 80 | 17 | 72 | 0 | 131 |
| Duopra | 143 | 58 | 40 | 3 | 56 |
FDB v1.0 takeover rates of Duopra and the model it is built on, Realtime-Venus-Audio, in the same pipeline (%).
Not every behaviour survives: Duopra answers fewer interruptions than Duopra-RL1 (96.5%) and, like every post-trained stage, no longer backchannels.
| System | Tool | Arg. | Resp. | Pass@1 |
|---|---|---|---|---|
| Commercial systems*, as reported | ||||
| GPT-Realtime | 87.6 | 68.0 | 79.2 | 60.0 |
| Gemini Live 3.1 | 81.7 | 58.8 | 71.8 | 54.0 |
| MOSS-Live | n/a | n/a | n/a | 51.0 |
| Gemini Live 2.5 | 78.6 | 59.3 | 55.4 | 49.0 |
| Cascaded | 80.3 | 56.2 | 60.0 | 45.0 |
| Grok | 79.7 | 54.2 | 61.7 | 43.0 |
| Ultravox v0.7 | 79.4 | 51.3 | 51.0 | 41.0 |
| Open-source systems, as reported | ||||
| Hu et al., Qwen3-235B backend | 71.7 | 55.2 | 67.0 | 48.0 |
| DuplexAgent-RSI | 97.4 | 57.0 | n/a | 47.0 |
| Realtime-Venus-Audio, offline | 82.0 | 52.2 | n/a | 42.0 |
| Gander, GPT-5.6 | 75.9 | 50.3 | 49.0 | 40.0 |
| NemotronLabs VoiceChat | 82.5 | 42.2 | n/a | 33.0 |
| Duopra, our harness | ||||
| Duopra, calling tools itself | 76.7 | 56.2 | 66.3 | 47.7 ± 3.5 |
| Duopra, Qwen3-8B backend | 81.8 | 57.0 | 70.6 | 44.7 ± 5.7 |
| Duopra, DeepSeek backend | 71.8 | 55.0 | 69.0 | 49.3 ± 3.5 |
Tool: tool-selection F1, which involves no judge (Hu et al. report tool-selection accuracy). Arg.: argument accuracy. Resp.: response quality. n/a: not reported. Bold: best within each group. Reported rows use each source's harness and judge, mostly GPT-4o; Duopra's use Qwen3.8-27B. DuplexAgent-RSI is measured on VoiceChat (49.0 on Qwen-Realtime) with exact-string argument matching; Realtime-Venus-Audio offline has the tool definitions in the prompt; MOSS-Live reports a tool-call success rate. Further open-source systems that we ran in our harness are listed in the paper's Table 1.
| Metric | Value |
|---|---|
| Takes the turn after the user finishes ↑ FDB v1.0 | 75.6 |
| Takes the turn during a pause ↓ FDB v1.0 | 5.4 |
| Responds after an interruption ↑ FDB v1.0 | 91.3 |
| Takes the turn at all ↑ FDB v3, calling tools itself | 93.0 |
| VoiceBench, mean of nine categories ↑ | 55.9 |
| Task-completion latency ↓ domain tools and backend both visible, Qwen3-8B | 10.42 s |
Pass@1 is read from the call log rather than the transcript. ± is the standard deviation across three seeds. Judged metrics use Qwen3.8-27B; tool-selection F1 involves no judge. Delegating every request instead raises the task-completion latency to 14.85 s.
Full-duplex interaction models exhibit strong conversational intelligence, enabling natural, low-latency turn-taking, backchannels, and interruptions. However, successful task completion further requires task agency—timely tool invocation and argument revision following user corrections. In frontend–backend architectures, delegation and execution errors are often entangled, obscuring the primary bottleneck. We first investigate this question on Full-Duplex-Bench (FDB) v3. Realtime-Venus-Audio with its original Harness delegates only 11.0% of sessions and passes 4.7%; even a perfect backend could improve performance by at most 6.3 points. We then introduce Duopra, which directly performs non-verbalized tool calls and treats backend delegation as tool selection. Its three-stage training combines supervised fine-tuning, full-duplex reinforcement learning with time-anchored deadlines and tiered rewards, and projector-only talker repair. With a fixed backend, Duopra raises the pass rate from 4.7% to 44.7%, compared with a 4.7-point gain from a much larger backend. It achieves 47.7% standalone and 49.3% with an open backend, outperforming all compared open-source systems. Complementary evaluations show improved turn-taking control and overlap handling over the base model, though with reduced backchanneling, showing that task agency can build on conversational agency.
@misc{duopra2026,
title = {Duopra: From Full-Duplex Conversation
to Full-Duplex Agency},
author = {Chen, Chiyu and Cheng, Pengzhou and Ma, Yuezhao and Li, Xiaocheng
and Liu, Can and Hu, Haonan and Yao, Yao and Yang, Yifei
and Lu, Yijie and Wang, Taolei and Li, Yansi and Wu, Muhuan
and Zhang, Zhuosheng and Liu, Gongshen and Sun, Chenglu},
year = {2026},
note = {Preprint forthcoming},
url = {https://cychen2003.github.io/duopra/}
}
The entry will be updated with a link once the preprint is public.