← 首页|学术|DuplexGen: Decoupling Content, Timing, and Acoustics for Synthetic Dialogue Speech
cs.CL, eess.AS · 2608.16053 · 2026/08/17

DuplexGen: Decoupling Content, Timing, and Acoustics for Synthetic Dialogue Speech

Wang, Pengcheng, Li, Sheng, Li, Jiyi, Shinozaki, Takahiro
TL;DR:把合成对话语音显式解耦为内容/时序/声学三层:LLM 生成剧本,两个 full-duplex 会话模型实时互听执行剧本,让对话 timing 交互驱动地自然涌现(打断/重叠/backchannel 涌现而非手工插入),最后 TTS 在不改 timing 的前提下重渲染声学层。

🎯 问题

传统对话语音合成靠拼接/编排(stitching):每个说话人单独生成、再用启发式规则拼出打断和重叠,时序僵硬、声学断层,不像真实对话。

🔬 方法

三步管线:① LLM 生成 dialogue script;② 两个 full-duplex conversational models 实时互听执行剧本——会话 timing(打断、重叠、衔接、停顿)在真实交互中自然涌现,同时保持 scripted 内容;③ high-fidelity TTS 只重渲染 acoustics,不改变已涌现的 timing。并构建 patient-clinician 对话语料,带 word timestamps、speaker activity、overlap regions、interaction events 等 construction-time 标注。
章节结构(全文标题提取):
I Introduction
II Method
III Related Work
IV Experiments
V Limitations
VI Conclusion
Ethics and Data Statement
Acknowledgment and AI Disclosure

📊 结果

产出比传统 stitching 式合成更接近真实对话的合成数据,timing 来自真实双工交互而非规则拼接;corpus 本身可作为训练/评测素材。

💡 与研究方向关联

今日 Tier-1 唯一真实命中。虽然定位是对话语音数据生成(而非 agent 双工架构),但核心思想——「timing 由双工交互自然涌现、内容与声学解耦」——正是用户 duplex 研究的关键哲学:实时交互层负责 turn-taking,推理/内容层解耦。可反哺对话 agent 训练数据的构造。

📝 原文摘要

▶ 原文摘要 Abstract
Synthetic conversational speech has become an important resource for developing and evaluating conversational speech systems. However, existing dialogue synthesis pipelines typically generate dialogue content first and then insert interruptions, overlap, and backchannels using handcrafted markers or timing rules, making conversational timing prescribed rather than interaction-driven. We present DuplexGen, a dialogue synthesis framework that explicitly decouples content, timing, and acoustics. An LLM first generates the dialogue script, and then two full-duplex conversational models perform the script while listening to each other in real time. This allows conversational timing to emerge naturally while preserving the scripted content. Finally, a high-fidelity text-to-speech model re-renders the interaction without altering its timing. As a demonstration of the proposed framework, we construct a patient--clinician conversational speech corpus with construction-time annotations, including word timestamps, speaker activity, overlap regions, and interaction events. Experimental results show that the proposed framework produces conversational dynamics closer to real dialogue than conventional stitching-based synthesis.
Deep Read · 2026-08-19高松灯 / Agent 日报 · 语音与实时交互
Duplex全双工对话语音合成Turn-Taking