LiveKit Inference

LiveKit is an open-source framework and developer platform for building, testing, deploying, scaling, and observing real-time voice, video, and physical AI agents. Its agent pipeline streams user speech from an app, browser, or phone call to an agent, which applies custom business logic and returns a response; the platform supports automatic turn detection and interruption handling, speech-to-text, language-model, and text-to-speech providers, web and mobile applications, and telephony through phone numbers and SIP integrations. LiveKit Cloud provides deployment and scaling on LiveKit's real-time infrastructure, alongside an inference gateway and full-stack observability for agent sessions.

LiveKit Agents is an open-source Python framework for building programmable, real-time multimodal voice agents that run as server-side participants. It combines speech-to-text, large language models, text-to-speech, and realtime APIs with LiveKit's WebRTC clients, telephony stack, RPC and data APIs, and MCP tool support. The framework provides agent sessions, server-side job scheduling and dispatch, semantic turn detection based on a transformer model, multi-agent handoffs, and tools for voice, text-only, transcription, vision, and video-avatar applications. Agents can be tested with native test integration, including event assertions and LLM-based judges, and can run locally in console mode, in development mode with hot reloading, or in production mode. The framework can use LiveKit Cloud or a self-hosted LiveKit server and is distributed as a Python package with plugins for model providers. The Agents framework is licensed under Apache-2.0; LiveKit turn-detection models use the LiveKit Model License.

A voice-tuned Gemma 4 31B model presented as part of LiveKit Inference for production voice agents. The video says it passes voice evaluations at 88% and begins speaking in about 380 milliseconds.

LiveKit Agents documentation is the official documentation resource for building realtime voice, video, and physical AI agents with LiveKit. Maintained as part of LiveKit's documentation, it covers Python and Node.js SDKs, Agent Builder, realtime media and data pipelines, STT-LLM-TTS processing, turn detection, interruptions, tool use, multimodal interaction, agent handoffs, telephony, deployment, observability, and model-provider integrations. It also explains how agent servers register with LiveKit, receive dispatch requests, run jobs in rooms over WebRTC, and connect to users through LiveKit Cloud or self-hosted infrastructure. The framework and LiveKit ecosystem are documented as open source under the Apache 2.0 license.

An open-source benchmark and evaluation harness for voice agents. It runs hundreds of simulated callers against a complete agent and scores outcomes such as successful bookings, caller cutoffs, invented tool calls, and data leaks. Its methodology and evaluation sets are open, allowing teams to run the benchmark against their own data.