W&B Serverless Sandboxes

Weights & Biases (W&B) is the platform account used in the evaluation workshop, providing inference credits and sandboxes for running evaluation workflows. The workflow uses W&B's Weave tooling to preserve agent trajectories, including commands, messages, and failures, so reviewers can inspect the evidence behind leaderboard scores.

E2B is open-source cloud sandbox infrastructure for securely running AI-generated or user-specific code, tools, and agents in isolated environments. Sandboxes can be started and controlled through JavaScript/TypeScript or Python SDKs, supporting command execution, code interpretation, desktop interaction, sandbox templates, and snapshots that preserve a running environment's filesystem and memory for pausing and resuming. The platform manages sandbox lifetimes, idle timeouts, workspace assignment, runtime filesystem writes, storage, orchestration, and network restrictions. It can be self-hosted using Embed deployment and Terraform on AWS, Google Cloud, Azure, or a general Linux machine.

Docker Sandboxes is a Docker-developed proprietary tool for running AI coding agents and other workloads inside isolated microVMs on a local machine or Docker-managed cloud infrastructure. Each sandbox has its own kernel and filesystem, controls access to the host filesystem, network, and tools, provides a private Docker daemon for test containers, uses secret placeholders, blocks network access by default, and records an audit trail. It supports agents including Claude Code, Codex, Gemini CLI, and OpenCode, with configurable network and filesystem rules, read-only mounts, MCP catalogs, and policy-based governance controls. It is distributed as the `sbx` command-line tool for macOS, Windows, and Linux, with stable, release-candidate, and nightly builds.

WolfBench is an open-source Apache-2.0 toolkit and evaluation methodology by Wolfram Ravenwolf for collecting, curating, analyzing, and publishing AI-agent benchmark results. It evaluates agents on Terminal-Bench 2.0 through Harbor orchestration and Daytona sandboxes, preserving configuration metadata, per-task results, token usage, timing, failures, and trajectory traces. Rather than reporting only an average, WolfBench calculates five metrics: the union of tasks solved across runs (Answers), the best single-run score, the mean across valid runs, the worst single-run score, and the tasks solved in every run (Solid). Repeated runs under consistent benchmark, runner, sandbox, agent, model-service, and settings configurations expose variance and distinguish dependable performance from isolated successes. The repository includes a Marimo dashboard that scans local and remote run directories over SSH, deduplicates and classifies runs, filters incomplete or invalid results, downloads data, saves curated JSON snapshots, uploads evaluations and trajectory traces to W&B Weave, and generates interactive HTML leaderboard charts. The core collector and chart generator use only the Python standard library; the dashboard requires Python 3.12+, Marimo, Weave/W&B, and SSH access for initial collection. Benchmark data is licensed CC-BY-4.0.

W&B Weave is an AI evaluation and observability tool that stores and displays per-run trajectories behind leaderboard results. In the WolfBench workflow, it preserves inspectable traces of commands, messages, failures, and other run evidence so reviewers can trace a score back to the conditions that produced it.