LiveKit

LiveKit Agents is an open-source Python framework for building programmable, real-time multimodal voice agents that run as server-side participants. It combines speech-to-text, large language models, text-to-speech, and realtime APIs with LiveKit's WebRTC clients, telephony stack, RPC and data APIs, and MCP tool support. The framework provides agent sessions, server-side job scheduling and dispatch, semantic turn detection based on a transformer model, multi-agent handoffs, and tools for voice, text-only, transcription, vision, and video-avatar applications. Agents can be tested with native test integration, including event assertions and LLM-based judges, and can run locally in console mode, in development mode with hot reloading, or in production mode. The framework can use LiveKit Cloud or a self-hosted LiveKit server and is distributed as a Python package with plugins for model providers. The Agents framework is licensed under Apache-2.0; LiveKit turn-detection models use the LiveKit Model License.

LiveKit Agents documentation is the official documentation resource for building realtime voice, video, and physical AI agents with LiveKit. Maintained as part of LiveKit's documentation, it covers Python and Node.js SDKs, Agent Builder, realtime media and data pipelines, STT-LLM-TTS processing, turn detection, interruptions, tool use, multimodal interaction, agent handoffs, telephony, deployment, observability, and model-provider integrations. It also explains how agent servers register with LiveKit, receive dispatch requests, run jobs in rooms over WebRTC, and connect to users through LiveKit Cloud or self-hosted infrastructure. The framework and LiveKit ecosystem are documented as open source under the Apache 2.0 license.

An open-source benchmark and evaluation harness for voice agents. It runs hundreds of simulated callers against a complete agent and scores outcomes such as successful bookings, caller cutoffs, invented tool calls, and data leaks. Its methodology and evaluation sets are open, allowing teams to run the benchmark against their own data.

Kit for AI is a service that provides a persistent memory and grounded-knowledge layer for AI agents. It exposes APIs and native memory tools that let agents call stored knowledge directly, supports ingesting files and URLs, and is intended to avoid building a separate retrieval-augmented generation stack. The site states the product works with any model and offers a free-to-start tier.

LiveKit's audio-based turn detector listens to raw speech audio, including rhyme, pitch, and intonation, to determine when a caller has finished speaking instead of waiting for a transcript. The video reports that it cuts users off 4.5% of the time at a given latency budget, compared with 9.9% for the best alternative; its evaluation harness is open source.