Weights & Biases

Weights & Biases (W&B) is the platform account used in the evaluation workshop, providing inference credits and sandboxes for running evaluation workflows. The workflow uses W&B's Weave tooling to preserve agent trajectories, including commands, messages, and failures, so reviewers can inspect the evidence behind leaderboard scores.

W&B Inference is a serverless model-serving endpoint for running agents against selected open-weight models.

The Mastering AI Observability Workshop repository is a hands-on workshop resource for building a trace-backed support-agent improvement workflow with Braintrust. It provides a local Python workshop app, setup instructions, a support agent using tool calls, seeded evaluation cases, production-like trace generation, Braintrust datasets and evaluations, hosted scorers, Topics facets, sample-trace imports, and optional draft evaluation cases derived from traces. The workflow connects tracing and online analysis to offline evaluations, custom scoring, failure investigation, regression cases, and reviewable code changes. The repository includes a public WORKSHOP.md guide, environment setup through uv and Make commands, local database seeding, a web interface, and configuration for Braintrust Gateway model calls.

W&B Weave is an AI evaluation and observability tool that stores and displays per-run trajectories behind leaderboard results. In the WolfBench workflow, it preserves inspectable traces of commands, messages, failures, and other run evidence so reviewers can trace a score back to the conditions that produced it.

OpenObserve is an open-source, self-hosted observability platform for logs, metrics, traces, dashboards, frontend monitoring, pipelines, and incidents. It accepts OpenTelemetry data and provides SQL queries for logs and traces and SQL or PromQL queries for metrics. The platform is built in Rust and supports single-binary deployment, using Parquet columnar storage with an S3-native architecture; it also offers a unified web interface and cloud-hosted edition.