Listens while it speaks
Every second, Duopra decides whether to listen or to speak. It takes the turn when you finish, holds back in a mid-sentence pause and gives way when you cut in.
It listens and speaks at the same time, and decides on its own when to act.
It's actually 150 euros, not 100.Duopra hears the correction, decides on its own to call the tool, and sends the call while it is still speaking.
Full-duplex models already know when to listen and when to speak. Duopra also knows when to act — it calls tools itself, mid-sentence, through actions that are never voiced.
Every second, Duopra decides whether to listen or to speak. It takes the turn when you finish, holds back in a mid-sentence pause and gives way when you cut in.
Tool calls go out on a private channel while Duopra is still talking. The call itself is never spoken — only its result is.
Change a value halfway through a request and only the final value reaches the tool. The abandoned 100 never does.
Two changes asked for in one hesitant breath become two calls, and each is reported only after its result comes back.
Connect a backend model and it becomes one more tool, query(task). Duopra hands work over the way it calls any tool: mid-conversation, with no pause and no hand-off you can hear.
A tool is a JSON schema: a name, a description and its parameters, supplied at run time. Write your own — your calendar, your database, your home — and Duopra can call it.
Two sessions from the evaluation, replayed in recorded time.
Every second, Duopra reads the new audio and any tool results, then emits three things at once.
One causal policy, conditioned on the audio so far, the observations injected so far and its own past outputs. Conversation and tool use are decided together, not handed between components.
A background agent is exposed as query(task), whose argument is a self-contained description of the user's task. Calling a tool directly or delegating is the same kind of decision, made by the same policy — there is no external dispatcher.
A runtime executes each call asynchronously and injects the result at the first one-second boundary after it returns. The model keeps listening and speaking while it waits, and reports a value only once it has arrived.
ecommerce_19; schematic). The user corrects both the product (laptop to tablet) and the quantity (two to one); add_to_cart depends on the product ID returned by search_products. (a) When domain schemas are available only to the backend, the frontend may need successive hand-offs to convey the evolving request, making it harder to preserve corrections and tool dependencies. (b) With schemas visible, Duopra can issue the corrected domain-tool calls directly without a hand-off, while query(task) remains available. Crossed-out values and curved arrows denote superseded requests. Table 18(a) of the paper reports an observed delegated success on this request.One stage for each problem: the form of an action, the decision to act, and a voice that drifts when the thinker changes.
Teaches the form of an action.
Supervised fine-tuning of the thinker on synthetic sessions that demonstrate the protocol: call tools by schema, wait for the result before reporting it, ask only for missing values.
Teaches the decision to act.
Dr. GRPO on a thinker LoRA, eight rollouts per task, in a sandbox that plays the same scripted user to every rollout. Time-anchored deadlines and outcome tiers reward calls that are timely, complete and grounded.
Keeps the voice intelligible.
Post-training shifts the thinker's hidden states, which condition the talker. Only the projector between them is retrained, on cached features; the text, control tokens and calls are untouched.
Pass@1 on Full-Duplex-Bench v3: 49.3% with a backend, 47.7% calling tools itself.
A session passes only when every executed call matches the gold call: right tool, every argument, no extra call.
Full-Duplex-Bench v3: 100 real, disfluent English recordings and 12 simulated APIs. Duopra runs in our harness — three seeds, 300 sessions per configuration, judged by Qwen3.8-27B; ± is the standard deviation across seeds. Hatched bars are published results, measured with each source's own harness and judge (mostly GPT-4o): reference points, not a common-protocol ranking. * Commercial system.
The outcome of each of 300 sessions, Duopra models calling tools themselves.
Multi-call requests become solvable: Duopra passes 28 of 102 sessions that need two or three calls, against 2 after SFT. The remaining headroom is in arguments — wrong arguments (19.3% of sessions) are now as common as sessions without a call (18.7%).
| Run (300 sessions) | Pass | Wrong arg. | Missing | Extra | No call |
|---|---|---|---|---|---|
| Duopra-SFT | 78 | 17 | 38 | 1 | 166 |
| Duopra-RL1 | 80 | 17 | 72 | 0 | 131 |
| Duopra | 143 | 58 | 40 | 3 | 56 |
FDB v1.0 takeover rates of Duopra and the model it is built on, Realtime-Venus-Audio, in the same pipeline (%).
| System | Tool | Arg. | Resp. | Pass@1 |
|---|---|---|---|---|
| Commercial systems*, as reported | ||||
| GPT-Realtime | 87.6 | 68.0 | 79.2 | 60.0 |
| Gemini Live 3.1 | 81.7 | 58.8 | 71.8 | 54.0 |
| MOSS-Live | n/a | n/a | n/a | 51.0 |
| Gemini Live 2.5 | 78.6 | 59.3 | 55.4 | 49.0 |
| Cascaded | 80.3 | 56.2 | 60.0 | 45.0 |
| Grok | 79.7 | 54.2 | 61.7 | 43.0 |
| Ultravox v0.7 | 79.4 | 51.3 | 51.0 | 41.0 |
| Open-source systems, as reported | ||||
| Hu et al., Qwen3-235B backend | 71.7 | 55.2 | 67.0 | 48.0 |
| DuplexAgent-RSI | 97.4 | 57.0 | n/a | 47.0 |
| Realtime-Venus-Audio, offline | 82.0 | 52.2 | n/a | 42.0 |
| Gander, GPT-5.6 | 75.9 | 50.3 | 49.0 | 40.0 |
| NemotronLabs VoiceChat | 82.5 | 42.2 | n/a | 33.0 |
| Duopra, our harness | ||||
| Duopra, calling tools itself | 76.7 | 56.2 | 66.3 | 47.7 ± 3.5 |
| Duopra, Qwen3-8B backend | 81.8 | 57.0 | 70.6 | 44.7 ± 5.7 |
| Duopra, DeepSeek backend | 71.8 | 55.0 | 69.0 | 49.3 ± 3.5 |
Tool: tool-name F1, which involves no judge (Hu et al. report tool-selection accuracy). Arg.: argument accuracy. Resp.: response quality. n/a: not reported. Bold: best within each group. Published rows use each source's harness and judge, mostly GPT-4o, and are reference points, not a common-protocol ranking; Duopra's use Qwen3.8-27B. DuplexAgent-RSI is measured on VoiceChat (49.0 on Qwen-Realtime) with exact-string argument matching; Realtime-Venus-Audio offline has the tool definitions in the prompt; MOSS-Live reports a tool-call success rate. Calling tools itself is the paper's Direct setting; a named backend is its Backend setting, in which only query(task) is visible. Further open-source systems that we ran in our harness are in the paper's Tables 2 and 11.
| Metric | Value |
|---|---|
| Takes the turn after the user finishes ↑ FDB v1.0 | 75.6 |
| Takes the turn during a pause ↓ FDB v1.0 | 5.4 |
| Responds after an interruption ↑ FDB v1.0 | 91.3 |
| Takes the turn at all ↑ FDB v3, calling tools itself | 93.0 |
| VoiceBench, mean of nine categories ↑ | 55.9 |
| Factual-answer latency ↓ domain tools and backend both visible, Qwen3-8B | 10.42 s |
Pass@1 is read from the call log rather than the transcript. ± is the standard deviation across three seeds. Judged metrics use Qwen3.8-27B; tool-name F1 involves no judge. Factual-answer latency is measured until the factual answer, including runtime and tool execution.
Full-duplex interaction models exhibit strong conversational agency, enabling natural, low-latency turn-taking, backchannels, and interruptions. However, successful task completion further requires task agency that timely tool invocation and argument revision following user corrections. Frontend–backend agents separate initiation from execution: the frontend decides whether and what to request, while the backend determines how. Execution can be offloaded, but unsent requests cannot be executed, and incomplete ones can misdirect the backend. We investigate this issue on FD-Bench-v3, most failures of Realtime-Venus-Audio + Harness in our streaming setting occur before a domain tool executes.
We introduce Duopra, a full-duplex interaction model that learns task execution alongside conversational control. Duopra treats the backend as a tool, accepting a self-contained natural-language request. Direct calls and backend delegation share an action interface and requirements for faithful initiation and correct execution. Its three-stage training combines protocol SFT, full-duplex RL rewarding timely and complete execution, and projector-only repair of talker conditioning after thinker updates. With a fixed backend, Pass@1 rises from 4.7% to 44.7%; Duopra achieves 47.7% under Direct (26.0% after SFT) and 49.3% under Backend with DeepSeek. It improves three FDB v1.0 takeover rates over the base model. These results show that task agency can coexist with key conversational agency.
@misc{duopra2026,
title = {Duopra: From Full-Duplex Conversation
to Full-Duplex Agency},
author = {Chen, Chiyu and Cheng, Pengzhou and Ma, Yuezhao and Li, Xiaocheng
and Liu, Can and Hu, Haonan and Yao, Yao and Yang, Yifei
and Lu, Yijie and Wang, Taolei and Li, Yansi and Wu, Muhuan
and Zhang, Zhuosheng and Liu, Gongshen and Sun, Chenglu},
year = {2026},
note = {Preprint forthcoming},
url = {https://cychen2003.github.io/duopra/}
}
The entry will be updated with a link once the preprint is public.