Pydantic

Pydantic AI is a Python SDK for building typed, extensible agents with model selection, function tools, dependency injection, and structured outputs. An agent's tool signatures and output types are validated so its runs return typed results; the same agent can run in applications, terminals, voice sessions, background workflows, and GitHub Actions. Pydantic Graph is provided as a separate package for typed graph-based control flow, while Pydantic Evals supports testing agent behavior.

Pydantic Evals is a code-first Python framework from Pydantic for evaluating LLM applications and agents. It organizes evaluations into datasets of cases, tasks, experiments, and evaluators: each case runs against a task, and evaluators score its outputs or tool-call trajectory using deterministic checks, custom logic, LLM judges, or span-based analysis of OpenTelemetry traces. Evaluations can run over defined datasets or sampled live production traffic, with results available in Python reports, the terminal, or Pydantic Logfire for visualization and comparison. The package is distributed as `pydantic-evals`; Logfire integration is optional.

Pipecat is an open-source Python framework maintained by Daily and the community for building real-time voice and multimodal conversational agents. It orchestrates audio and video, transports such as WebSockets and WebRTC, speech recognition and text-to-speech services, speech preprocessing, turn detection, model calls, tool calls, and streamed audio through composable conversation pipelines. Pipelines can operate as single agents or as multi-agent systems whose specialists hand off, fan out in parallel, or coordinate over a shared bus locally or across processes and machines. The project also provides a CLI for scaffolding, monitoring, and deploying agents, along with client SDKs and related tools for structured conversations and pipeline debugging.

DSPy is a framework for defining language-model program contracts—inputs, output schemas, and scoring functions—so prompt and instruction optimization becomes a search problem. In the described workflow, it searches for shorter instructions while preserving the application's interface and evaluation contract, with candidate changes checked through repeated model runs, shadow evaluation, and human review of disagreements.