AI product Open source
Haru is a low-latency, multilingual, full-duplex spoken-dialogue model that listens and speaks simultaneously. It processes one 133-millisecond audio frame at a time and produces an audio frame and text, combining user audio, model audio, and model text in a single language-model position so listening and speaking do not wait for each other.
The model generates a 64-dimensional continuous audio latent per frame through a small MLP head using MeanFlow sampling, without a separate audio transformer or discrete audio tokens. A text character card controls the character, while a short reference recording controls the speaking voice. It supports English, Spanish, Portuguese, German, French, Chinese, Japanese, Russian, and Korean, and uses the causal VibeVoice acoustic tokenizer for 24 kHz audio.
The repository provides PyTorch inference and a browser demo, an MLX implementation with a native Apple Silicon macOS app, and a vLLM-Omni plugin for serving concurrent conversations. Haru-4B is released as a pretrained base model for research under the Apache License 2.0; the README notes that it is intended for natural conversation rather than reliable factual answers or instruction following.
1 use taken from transcripts — each links to the moment in the video.
A multilingual voice conversation model that listens and speaks at the same time, with a text card setting its character and a reference recording setting the voice. The repo includes a browser demo, PyTorch implementation, an MLX version for Apple Silicon, and serving through VLM Omni.
1 in the library.