AI product Open source · MIT
VibeVoice is Microsoft's open-source family of voice AI models for automatic speech recognition and text-to-speech. Its models use continuous acoustic and semantic speech tokenizers operating at 7.5 Hz, a next-token diffusion framework, a large language model for textual context and dialogue flow, and a diffusion head for acoustic detail.
VibeVoice-ASR processes up to 60 minutes of audio in a single pass and produces structured transcriptions containing speakers, timestamps, and content, with support for customized hotwords. Its streaming variant transcribes audio as it arrives. VibeVoice-Realtime-0.5B is a streaming text-to-speech model for long-form generation, with approximately 300 milliseconds of first-audio latency and streaming text input. The repository also documents long-form, multi-speaker TTS models, including support for conversations with up to four speakers, while noting that the original VibeVoice-TTS code was removed from the repository.
The project is intended for research and development. The repository warns that outputs can be inaccurate or biased and that synthetic speech can be misused for impersonation, fraud, or disinformation.