← 首页|学术|VoiceChat-TTS: A Low-Latency Continuous Speech Synthesis Model for Interactive Agents
eess.AS, cs.CL · 2608.13831 · 2026/08/13

VoiceChat-TTS: A Low-Latency Continuous Speech Synthesis Model for Interactive Agents

Casanova, Edresson, Kim, Jaehyeon, Fuenmayor, Mariana Graterol, Hussain, Shehzeen, Klimkov, Viacheslav, Mendelev, Valentin, Desta, Mikyas, Neekhara, Paarth, Zelasko, Piotr, Chen, Chen, Rastorgueva, Elena, Hu, Ke
TL;DR:面向交互 agent 的低延迟连续流式 TTS:由 LLM 文本 token 流直接驱动、控制 token 显式中断、无文本输入时输出静音——支持 always-on 响应式语音与话语中打断且不重置 KV cache,同时保持模块化与高语音质量。

🎯 问题

多数语音 SLM 仍限于轮次式、缺实时适配(如用户 barge-in);duplex S2S/S2T 用单级流水线降延迟,但 ASR/打断处理/高保真合成联合优化常牺牲语音质量。

🔬 方法

VoiceChat-TTS:直接消费 LLM 文本 token 流,显式中断经控制 token,无输入输出静音;话语中打断不重置 KV cache。在保持流式连续性的同时守住模块化与质量。
章节结构(全文标题提取):
1 Introduction
2 VoiceChat-TTS Model
3 Experiments
4 Conclusions, Limitations, and Future Work

📊 结果

实现 always-on 响应式语音生成、支持 mid-utterance 打断且不重置 KV cache——duplex 语音侧的模块化替代路径,规避单级模型的质量妥协。

💡 与研究方向关联

今日无纯 duplex 论文,这篇是语音侧最贴近的进展:连续流式 + barge-in + 控制 token + 打断不重置 KV,与用户 duplex 语音架构的组件级设计直接对应。

📝 原文摘要

▶ 原文摘要 Abstract
Spoken dialogue is a natural form of human--computer interaction, yet most speech language models remain limited to turn-based operation and lack real-time adaptability, such as user barge-in. Recent duplex speech-to-speech and speech-to-text models reduce latency by replacing multi-stage pipelines, but often compromise speech quality because accurate ASR, interruption handling, and high-fidelity synthesis must be optimized jointly. We propose VoiceChat-TTS, a low-latency, continuous, and streamable text-to-speech model for interactive agents. VoiceChat-TTS is driven directly by LLM text-token streams, supports explicit interruption via control tokens, and produces silence when no textual input is available. The model enables always-on, responsive speech generation while preserving modularity and high speech quality, and it supports mid-utterance interruptions without resetting the KV cache.
Deep Read · 2026-08-18高松灯 / Agent 日报
Duplex VoiceStreaming TTSBarge-inLow Latency