Phoenix

AgentOps is an open-source observability and developer-tools platform for building, evaluating, and monitoring AI agents. Its Python SDK records agent sessions and LLM calls for dashboard-based session replays, step-by-step execution graphs, debugging, analytics, cost tracking, and benchmarking. Developers instrument code with decorators for sessions, agents, operations, tasks, and workflows; these support nested span hierarchies, input and output recording, exception handling, asynchronous functions, generators, and custom attributes. The SDK provides integrations with agent frameworks and LLM systems including CrewAI, AG2/AutoGen, Agno, LangGraph, and the OpenAI Agents SDK, and the platform can be self-hosted with its dashboard and API backend. The app is released under the MIT license.

Arize AI is an AI agent observability, evaluation, and improvement platform. It provides tracing, evaluation, and experimentation, using production signals to support continual improvement of agents. The platform is available through Arize AX and a terminal-based setup.

HyperProbe is an AI-native production debugger for investigating live application state without code changes, redeployment, or service restarts. It reads logs and distributed traces, uses a coding agent to locate a suspected file and line, and places a temporary read-only virtual breakpoint there. When the breakpoint fires on live traffic, it captures the variable state asynchronously without pausing requests, then uses the evidence to confirm a root cause. Probes are non-blocking, cannot write memory or execute code, disappear after capture, and are recorded in an immutable audit trail. The service is designed for silent failures, exceptions whose causes occur earlier in the call chain, incorrect behavior without thrown errors, race conditions, third-party contract changes, and business-metric failures. It supports JavaScript, TypeScript, Java, Kotlin, Python, and Ruby, and can run in a managed cloud, self-hosted deployment, or private VPC. The page states that it integrates with PagerDuty, Datadog, Slack, Cursor, Claude Code, Codex, and Opencode, with approval gates and agent-side PII redaction.

OrcaReplay is an open-source replay and debugging tool for AI-agent executions, built by the OrcaRouter.ai team. It records an agent run as a local trace, including model requests and responses, tool calls and results, shell exit codes and timing, filesystem snapshots, MCP traffic, and selected network activity. It runs as a local proxy and capture layer around an unmodified agent. A proxy records model traffic, while PATH shims, JSON-RPC interception, filesystem snapshots, fetch hooks, and optional TLS interception capture effects that model transcripts cannot show. The resulting timeline can be viewed, exported, queried through JSON or MCP, and analyzed with recorded versus inferred causal edges. The same recorded stream supports offline replay with the network blocked, forking from a derived checkpoint onto another model, and comparison of several models with an optional verification command. Replay uses the recorded conversation and workspace state; interactive terminal sessions can be approximate because prompts and interactive-only tools may not exist on the wire. The CLI is installed with npm, requires Node 20 or newer, stores runs under `.orca/runs/`, and the code, CLI, viewer, adapters, and trace format implementation are released under Apache-2.0; the trace specification is CC BY 4.0. Traces are local and redacted on write, but the project describes them as sensitive and treats redaction as best-effort.