LiveKit Inference (voice-tuned Gemma 4 31B)

A voice-tuned version of the Gemma 4 31B language model hosted on LiveKit Inference for production voice agents. The talk describes it as designed for long prompts, many tool calls, and tight latency budgets; it reports an 88% voice-evaluation score and speech beginning in about 380 milliseconds.

An open-source benchmark and evaluation harness for voice agents. It runs hundreds of simulated callers against a complete agent and scores outcomes such as successful bookings, caller cutoffs, invented tool calls, and data leaks. Its methodology and evaluation sets are open, allowing teams to run the benchmark against their own data.

LiveKit's audio-based turn detector listens to raw speech audio, including rhyme, pitch, and intonation, to determine when a caller has finished speaking instead of waiting for a transcript. The video reports that it cuts users off 4.5% of the time at a given latency budget, compared with 9.9% for the best alternative; its evaluation harness is open source.

LiveKit Agents is an open-source Python framework for building programmable, real-time multimodal voice agents that run as server-side participants. It combines speech-to-text, large language models, text-to-speech, and realtime APIs with LiveKit's WebRTC clients, telephony stack, RPC and data APIs, and MCP tool support. The framework provides agent sessions, server-side job scheduling and dispatch, semantic turn detection based on a transformer model, multi-agent handoffs, and tools for voice, text-only, transcription, vision, and video-avatar applications. Agents can be tested with native test integration, including event assertions and LLM-based judges, and can run locally in console mode, in development mode with hot reloading, or in production mode. The framework can use LiveKit Cloud or a self-hosted LiveKit server and is distributed as a Python package with plugins for model providers. The Agents framework is licensed under Apache-2.0; LiveKit turn-detection models use the LiveKit Model License.

LiveKit is an open-source framework and developer platform for building, testing, deploying, scaling, and observing real-time voice, video, and physical AI agents. Its agent pipeline streams user speech from an app, browser, or phone call to an agent, which applies custom business logic and returns a response; the platform supports automatic turn detection and interruption handling, speech-to-text, language-model, and text-to-speech providers, web and mobile applications, and telephony through phone numbers and SIP integrations. LiveKit Cloud provides deployment and scaling on LiveKit's real-time infrastructure, alongside an inference gateway and full-stack observability for agent sessions.