OpenAI Agents SDK

AgentOps is an open-source observability and developer-tools platform for building, evaluating, and monitoring AI agents. Its Python SDK records agent sessions and LLM calls for dashboard-based session replays, step-by-step execution graphs, debugging, analytics, cost tracking, and benchmarking. Developers instrument code with decorators for sessions, agents, operations, tasks, and workflows; these support nested span hierarchies, input and output recording, exception handling, asynchronous functions, generators, and custom attributes. The SDK provides integrations with agent frameworks and LLM systems including CrewAI, AG2/AutoGen, Agno, LangGraph, and the OpenAI Agents SDK, and the platform can be self-hosted with its dashboard and API backend. The app is released under the MIT license.

OpenInference is an open-source set of OpenTelemetry-complementary semantic conventions, instrumentation libraries, and plugins for tracing AI applications. Its specification models LLM invocations and surrounding context such as vector-store retrieval and external tool use in a transport- and file-format-agnostic schema, while its instrumentations normalize traces from model, agent, and application frameworks across Python, JavaScript, Java, and Go. The project also defines conventions for real-time audio and voice-agent traces, allowing audio, transcripts, and tool calls to be represented in a queryable session trace. It is natively supported by Arize Phoenix and Arize AX and can send data to any OpenTelemetry-compatible backend.

Project Rosetta Stone is a workshop and comparison repository for instrumenting the same AI agent with different agent frameworks and observability platforms. Its Wonder Toys application is a chat-to-purchase toy-store assistant with semantic vector search, product browsing, purchasing, order tracking, and cancellation; framework variants are provided without observability, with Arize Phoenix instrumentation, and with Arize AX instrumentation. Readers can compare the instrumentation footprint by diffing each framework's baseline and instrumented directories, then run the applications locally to send traces to Phoenix or AX. The repository also includes shared synthetic requests, evaluation harnesses, voice testing for the OpenAI voice tier, product data, and instructions for adding frameworks and running the end-to-end workflows; it is licensed under MIT.