AI product
Muse Realtime Avatar is Meta’s real-time avatar system, which turns Muse Realtime Voice into expressive, interactive characters conditioned on reference media. It can animate portraits, illustrations, animals, and everyday objects with facial, hand, and full-body movement while preserving their appearance and mannerisms across conversational turns.
The system uses the same speech-token stream for audio and visual generation, keeping spoken content, lip motion, and expression synchronized. Its audio-driven Diffusion Transformer generates video in short causal chunks, feeding newly generated video latents forward as motion context for subsequent chunks. Meta describes a distillation process that converts a bidirectional teacher using guided diffusion into a two-step causal student, and an inference stack using persistent KV caches, dynamic batching, quantization-aware training, fused kernels, and CUDA Graph capture for subsecond interactive latency. Generated video includes an invisible Meta Video Seal watermark for traceability.
1 use taken from transcripts — each links to the moment in the video.
A real-time avatar system from Meta that turns its real-time voice model into interactive characters, using the same speech token stream so lip motion and expression stay synchronized. It animates portraits, illustrations, animals and objects with video responses in under a second.
1 in the library.